--- /dev/null
+<?xml version="1.0" encoding="UTF-8"?>
+<chapter version="5.0" xml:id="lustressk" xml:lang="en-US"
+ condition='l29'
+ xmlns="http://docbook.org/ns/docbook">
+ <title xml:id="lustressk.title">Configuring Shared-Secret Key
+ (SSK) Security</title>
+ <para>This chapter describes how to configure Shared-Secret Key security
+ and includes the following sections:</para>
+ <itemizedlist>
+ <listitem>
+ <para><xref linkend="sskoverview"/></para>
+ </listitem>
+ <listitem>
+ <para><xref linkend="ssksecurityflavors"/></para>
+ </listitem>
+ <listitem>
+ <para><xref linkend="sskkeyfiles"/></para>
+ </listitem>
+ <listitem>
+ <para><xref linkend="ssklustregsskeyring"/></para>
+ </listitem>
+ <listitem>
+ <para><xref linkend="ssknodemaprole"/></para>
+ </listitem>
+ <listitem>
+ <para><xref linkend="sskexamples"/></para>
+ </listitem>
+ <listitem>
+ <para><xref linkend="ssksptlrpcctx"/></para>
+ </listitem>
+ </itemizedlist>
+ <section xml:id="sskoverview">
+ <title>SSK Security Overview</title>
+ <para>The SSK feature ensures integrity and data protection for Lustre
+ PtlRPC traffic. Key files containing a shared secret and session-specific
+ attributes are distributed to Lustre hosts. This authorizes Lustre hosts
+ to mount the file system and optionally enables secure data transport,
+ depending on which security flavor is configured. The administrator handles
+ the generation, distribution, and installation of SSK key files, see
+ <xref linkend="sskkeyfilemanagement"/>.</para>
+ <section remap="h3">
+ <title>Key features</title>
+ <para>SSK provides the following key features:</para>
+ <itemizedlist>
+ <listitem>
+ <para>Host-based authentication</para>
+ </listitem>
+ <listitem>
+ <para>Data Transport Privacy</para>
+ <itemizedlist>
+ <listitem>
+ <para>Encrypts Lustre RPCs</para>
+ </listitem>
+ <listitem>
+ <para>Prevents eavesdropping</para>
+ </listitem>
+ </itemizedlist>
+ </listitem>
+ <listitem>
+ <para>Data Transport Integrity - Keyed-Hashing Message
+ Authentication Code (HMAC)</para>
+ <itemizedlist>
+ <listitem>
+ <para>Prevents man-in-the-middle attacks</para>
+ </listitem>
+ <listitem>
+ <para>Ensures RPCs cannot be altered undetected</para>
+ </listitem>
+ </itemizedlist>
+ </listitem>
+ </itemizedlist>
+ </section>
+ </section>
+ <section xml:id="ssksecurityflavors">
+ <title>SSK Security Flavors</title>
+ <para>SSK is implemented as a Generic Security Services (GSS) mechanism
+ through Lustre's support of the GSS Application Program Interface (GSSAPI).
+ The SSK GSS mechanism supports five flavors that offer varying levels of
+ protection.</para>
+ <para>Flavors provided:</para>
+ <itemizedlist>
+ <listitem>
+ <para><literal>skn</literal> - SSK Null (Authentication)</para>
+ </listitem>
+ <listitem>
+ <para><literal>ska</literal> - SSK Authentication and Integrity for
+ non-bulk RPCs</para>
+ </listitem>
+ <listitem>
+ <para><literal>ski</literal> - SSK Authentication and Integrity</para>
+ </listitem>
+ <listitem>
+ <para><literal>skpi</literal> - SSK Authentication, Privacy, and
+ Authentication</para>
+ </listitem>
+ <listitem>
+ <para><literal>gssnull</literal> - Provides no protection. Used for
+ testing purposes only</para>
+ </listitem>
+ </itemizedlist>
+ <para>The table below describes the security characteristics of each
+ flavor:</para>
+ <table xml:id="ex.sskflavortable" frame='all'>
+ <title>SSK Security Flavor Protections</title>
+ <tgroup cols='5' align='center' colsep='1' rowsep='1'>
+ <colspec colname='c1' align='left' colwidth='3*'/>
+ <colspec colname='c2' colwidth='3*'/>
+ <colspec colname='c3' colwidth='3*'/>
+ <colspec colname='c4' colwidth='3*'/>
+ <colspec colname='c5' colwidth='3*'/>
+ <thead>
+ <row>
+ <entry>
+ <para></para>
+ </entry>
+ <entry>
+ <para>skn</para>
+ </entry>
+ <entry>
+ <para>ska</para>
+ </entry>
+ <entry>
+ <para>ski</para>
+ </entry>
+ <entry>
+ <para>skpi</para>
+ </entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>
+ <para>Required to mount file system</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#00cc00" ?><?dbfo bgcolor="#00cc00" ?>
+ <para>Yes</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#00cc00" ?><?dbfo bgcolor="#00cc00" ?>
+ <para>Yes</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#00cc00" ?><?dbfo bgcolor="#00cc00" ?>
+ <para>Yes</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#00cc00" ?><?dbfo bgcolor="#00cc00" ?>
+ <para>Yes</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para>Provides RPC Integrity</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#ff3300" ?><?dbfo bgcolor="#ff3300" ?>
+ <para>No</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#00cc00" ?><?dbfo bgcolor="#00cc00" ?>
+ <para>Yes</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#00cc00" ?><?dbfo bgcolor="#00cc00" ?>
+ <para>Yes</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#00cc00" ?><?dbfo bgcolor="#00cc00" ?>
+ <para>Yes</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para>Provides RPC Privacy</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#ff3300" ?><?dbfo bgcolor="#ff3300" ?>
+ <para>No</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#ff3300" ?><?dbfo bgcolor="#ff3300" ?>
+ <para>No</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#ff3300" ?><?dbfo bgcolor="#ff3300" ?>
+ <para>No</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#00cc00" ?><?dbfo bgcolor="#00cc00" ?>
+ <para>Yes</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para>Provides Bulk RPC Integrity</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#ff3300" ?><?dbfo bgcolor="#ff3300" ?>
+ <para>No</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#ff3300" ?><?dbfo bgcolor="#ff3300" ?>
+ <para>No</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#00cc00" ?><?dbfo bgcolor="#00cc00" ?>
+ <para>Yes</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#00cc00" ?><?dbfo bgcolor="#00cc00" ?>
+ <para>Yes</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para>Provides Bulk RPC Privacy</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#ff3300" ?><?dbfo bgcolor="#ff3300" ?>
+ <para>No</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#ff3300" ?><?dbfo bgcolor="#ff3300" ?>
+ <para>No</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#ff3300" ?><?dbfo bgcolor="#ff3300" ?>
+ <para>No</para>
+ </entry>
+ <entry><?dbhtml bgcolor="#00cc00" ?><?dbfo bgcolor="#00cc00" ?>
+ <para>Yes</para>
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>Valid non-GSS flavors include:</para>
+ <para><literal>null</literal> - Provides no protection. This is the
+ default flavor.</para>
+ <para><literal>plain</literal> - Plaintext with a hash on each RPC.</para>
+ <section remap="h3">
+ <title>Secure RPC Rules</title>
+ <para>Secure RPC configuration rules are written to the Lustre log
+ (llog) with the <literal>lctl</literal> command. Rules are processed
+ with the llog and dictate the security flavor that is used for a
+ particular Lustre network and direction.</para>
+ <note>
+ <para>Rules take affect in a matter of seconds and impact both existing
+ and new connections.</para>
+ </note>
+ <para>Rule format:</para>
+ <para><replaceable>target</replaceable>.srpc.flavor.<replaceable>network</replaceable>[.<replaceable>direction</replaceable>]=<replaceable>flavor</replaceable></para>
+ <itemizedlist>
+ <listitem>
+ <para><replaceable>target</replaceable> - This could be the
+ file system name or a specific
+ MDT/OST device name.</para>
+ </listitem>
+ <listitem>
+ <para><replaceable>network</replaceable> - LNet network
+ name of the RPC initiator. For example
+ <literal>tcp1</literal> or <literal>o2ib0</literal>. This can also
+ be the keyword <literal>default</literal> that applies to all
+ networks otherwise specified.</para>
+ </listitem>
+ <listitem>
+ <para><replaceable>direction</replaceable> - Direction is
+ optional. This could be one of
+ <literal>mdt2mdt</literal>, <literal>mdt2ost</literal>,<literal>
+ cli2mdt</literal>, or <literal>cli2ost</literal>.</para>
+ </listitem>
+ </itemizedlist>
+ <note>
+ <para>To secure the connection to the MGS use the
+ <literal>mgssec=</literal><replaceable>flavor</replaceable>
+ mount option. This is required because security rules are unknown to
+ the initiator until after the MGS connection has been established.
+ </para>
+ </note>
+ <para>The examples below are for a test Lustre file system named
+ <emphasis>testfs</emphasis>.</para>
+ <section remap="h4">
+ <title>Defining Rules</title>
+ <para>Rules can be defined and deleted in any order. The rule with the
+ greatest specificity for a given connection is applied. The
+ <replaceable>fsname</replaceable><literal>.srpc.flavor.default</literal>
+ rule is the broadest rule as it applies to all non-MGS connections for
+ the file system in the absence of a more specific rule. You may tailor
+ SSK security to your needs by further specifying a specific
+ <literal>target</literal>, <literal>network</literal>, and/or
+ <literal>direction</literal>.</para>
+ <para>The following example illustrates an approach to configuring SSK
+ security for an environment consisting of three LNet networks. The
+ requirements for this example are:</para>
+ <itemizedlist>
+ <listitem>
+ <para>All non-MGS connections must be authenticated.</para>
+ </listitem>
+ <listitem>
+ <para>PtlRPC traffic on LNet network <literal>tcp0</literal> must
+ be encrypted.</para>
+ </listitem>
+ <listitem>
+ <para>LNet networks <literal>tcp1</literal> and
+ <literal>o2ib0</literal> are local physically secure networks that
+ require high performance. Do not encrypt PtlRPC traffic on these
+ networks.</para>
+ </listitem>
+ </itemizedlist>
+ <orderedlist>
+ <listitem>
+ <para>Ensure that all non-MGS connections are authenticated and
+ encrypted by default.</para>
+ <screen>mgs# lctl conf_param testfs.srpc.flavor.default=skpi</screen>
+ </listitem>
+ <listitem>
+ <para>Override the file system default security flavor on LNet
+ networks <literal>tcp1</literal> and <literal>o2ib0</literal> with
+ <literal>ska</literal>. Security flavor <literal>ska</literal>
+ provides authentication but without the performance impact of
+ encryption and bulk RPC integrity.</para>
+ <screen>mgs# lctl conf_param testfs.srpc.flavor.tcp1=ska
+mgs# lctl conf_param testfs.srpc.flavor.o2ib0=ska</screen>
+ </listitem>
+ </orderedlist>
+ <note>
+ <para>Currently the "<literal>lctl set_param -P</literal>" format does
+ not work with sptlrpc.</para>
+ </note>
+ </section>
+ <section remap="h4">
+ <title>Listing Rules</title>
+ <para>To view the Secure RPC Config Rules, enter:</para>
+ <screen>mgs# lctl get_param mgs.*.live.testfs
+...
+Secure RPC Config Rules:
+testfs.srpc.flavor.tcp.cli2mdt=skpi
+testfs.srpc.flavor.tcp.cli2ost=skpi
+testfs.srpc.flavor.o2ib=ski
+...</screen>
+ </section>
+ <section remap="h4">
+ <title>Deleting Rules</title>
+ <para>To delete a security flavor for an LNet network use the
+ <literal>conf_param -d</literal> command to delete the flavor for that
+ network:</para>
+ <para>For example, to delete the
+ <literal>testfs.srpc.flavor.o2ib1=ski</literal> rule, enter:</para>
+ <screen>mgs# lctl conf_param -d testfs.srpc.flavor.o2ib1</screen>
+ </section>
+ </section>
+ </section>
+ <section xml:id="sskkeyfiles">
+ <title>SSK Key Files</title>
+ <para>SSK key files are a collection of attributes formatted as fixed
+ length values and stored in a file, which are distributed by the
+ administrator to client and server nodes. Attributes include:</para>
+ <itemizedlist>
+ <listitem>
+ <para><emphasis role='bold'>Version</emphasis> - Key file schema
+ version number. Not user-defined.</para>
+ </listitem>
+ <listitem>
+ <para><emphasis role='bold'>Type</emphasis> - A mandatory attribute
+ that denotes the Lustre role of the key file consumer. Valid key types
+ are:
+ <itemizedlist>
+ <listitem>
+ <para><emphasis role='bold'>mgs</emphasis> - for MGS when the
+ <literal>mgssec</literal> <literal>mount.lustre</literal> option
+ is used.</para>
+ </listitem>
+ <listitem>
+ <para><emphasis role='bold'>server</emphasis> - for MDS and OSS
+ servers</para>
+ </listitem>
+ <listitem>
+ <para><emphasis role='bold'>client</emphasis> - for clients as
+ well as servers who communicate with other servers in a client
+ context (e.g. MDS communication with OSTs).</para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </listitem>
+ <listitem>
+ <para><emphasis role='bold'>HMAC algorithm</emphasis> - The Keyed-Hash
+ Message Authentication Code algorithm used for integrity. Valid
+ algorithms are (Default: SHA256):
+ <itemizedlist>
+ <listitem>
+ <para>SHA256</para>
+ </listitem>
+ <listitem>
+ <para>SHA512</para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </listitem>
+ <listitem>
+ <para><emphasis role='bold'>Cryptographic algorithm</emphasis> - Cipher
+ for encryption. Valid algorithms are (Default: AES-256-CTR).</para>
+ <itemizedlist>
+ <listitem>
+ <para>AES-256-CTR</para>
+ </listitem>
+ </itemizedlist>
+ </listitem>
+ <listitem>
+ <para><emphasis role='bold'>Session security context expiration</emphasis> -
+ Seconds before session contexts generated from key expire and are
+ regenerated (Default: 604800 seconds (7 days)).</para>
+ </listitem>
+ <listitem>
+ <para><emphasis role='bold'>Shared key length</emphasis> - Shared key
+ length in bits (Default: 256).</para>
+ </listitem>
+ <listitem>
+ <para><emphasis role='bold'>Prime length</emphasis> - Length of prime
+ (p) in bits used for the Diffie-Hellman Key Exchange (DHKE).
+ (Default: 2048). This is generated only for client keys and can take a
+ while to generate. This value also sets the minimum prime length that
+ servers and MGS will accept from a client. Clients attempting to
+ connect with a prime length less than the minimum will be rejected.
+ In this way servers can guarantee the minimum encryption level that
+ will be permitted.</para>
+ </listitem>
+ <listitem>
+ <para><emphasis role='bold'>File system name</emphasis> - Lustre File
+ system name for key.</para>
+ </listitem>
+ <listitem>
+ <para><emphasis role='bold'>MGS NIDs</emphasis> - Comma-separated list
+ of MGS NIDs. Only required when <literal>mgssec</literal> is used
+ (Default: "").</para>
+ </listitem>
+ <listitem>
+ <para><emphasis role='bold'>Nodemap name</emphasis> - Nodemap name for
+ key (Default: "default"). See <xref linkend="ssknodemaprole"/></para>
+ </listitem>
+ <listitem>
+ <para><emphasis role='bold'>Shared key</emphasis> - Shared secret used
+ by all SSK flavors to provide authentication.</para>
+ </listitem>
+ <listitem>
+ <para><emphasis role='bold'>Prime (p)</emphasis> - Prime used for the
+ DHKE. This is only used for keys with <literal>Type=client</literal>.
+ </para>
+ </listitem>
+ </itemizedlist>
+ <note>
+ <para>Key files provide a means to authenticate Lustre connections;
+ always store and transfer key files securely. Key files must not be
+ world writable or they will fail to load.</para>
+ </note>
+ <section xml:id="sskkeyfilemanagement" remap="h3">
+ <title>Key File Management</title>
+ <para>The <literal>lgss_sk</literal> utility is used to write, modify,
+ and read SSK key files. <literal>lgss_sk</literal> can be used to load
+ key files singularly into the kernel keyring. <literal>lgss_sk</literal>
+ options include:</para>
+ <table xml:id="table.lgss_sk_params" frame='all'>
+ <title>lgss_sk Parameters</title>
+ <tgroup cols='3' align='left' colsep='1' rowsep='1'>
+ <colspec colname='c1' colwidth='2*'/>
+ <colspec colname='c2' colwidth='1*'/>
+ <colspec colname='c3' colwidth='3*'/>
+ <thead>
+ <row>
+ <entry>
+ <para>Parameter</para>
+ </entry>
+ <entry>
+ <para>Value</para>
+ </entry>
+ <entry>
+ <para>Description</para>
+ </entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>
+ <para><literal>-l|--load</literal></para>
+ </entry>
+ <entry>
+ <para><replaceable>filename</replaceable></para>
+ </entry>
+ <entry>
+ <para>Install key from file into user's session keyring. Must
+ be executed by <emphasis>root</emphasis>.</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-m|--modify</literal></para>
+ </entry>
+ <entry>
+ <para><replaceable>filename</replaceable></para>
+ </entry>
+ <entry>
+ <para>Modify a file's key attributes</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-r|--read</literal></para>
+ </entry>
+ <entry>
+ <para><replaceable>filename</replaceable></para>
+ </entry>
+ <entry>
+ <para>Show file's key attributes</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-w|--write</literal></para>
+ </entry>
+ <entry>
+ <para><replaceable>filename</replaceable></para>
+ </entry>
+ <entry>
+ <para>Generate key file</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-c|--crypt</literal></para>
+ </entry>
+ <entry>
+ <para><replaceable>cipher</replaceable></para>
+ </entry>
+ <entry>
+ <para>Cipher for encryption (Default: AES Counter mode)</para>
+ <para>AES-256-CTR</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-i|--hmac</literal></para>
+ </entry>
+ <entry>
+ <para><replaceable>hash</replaceable></para>
+ </entry>
+ <entry>
+ <para>Hash algorithm for intregrity (Default: SHA256)</para>
+ <para>SHA256 or SHA512</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-e|--expire</literal></para>
+ </entry>
+ <entry>
+ <para><replaceable>seconds</replaceable></para>
+ </entry>
+ <entry>
+ <para>Seconds before contexts from key expire (Default: 604800
+ (7 days))</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-f|--fsname</literal></para>
+ </entry>
+ <entry>
+ <para><replaceable>name</replaceable></para>
+ </entry>
+ <entry>
+ <para>File system name for key</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-g|--mgsnids</literal></para>
+ </entry>
+ <entry>
+ <para><replaceable>NID(s)</replaceable></para>
+ </entry>
+ <entry>
+ <para>Comma separated list of MGS NID(s). Only required when
+ mgssec is used (Default: "")</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-n|--nodemap</literal></para>
+ </entry>
+ <entry>
+ <para><replaceable>map</replaceable></para>
+ </entry>
+ <entry>
+ <para>Nodemap name for key (Default: "default")</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-p|--prime-bits</literal></para>
+ </entry>
+ <entry>
+ <para><replaceable>length</replaceable></para>
+ </entry>
+ <entry>
+ <para>Prime length (p) for DHKE in bits (Default: 2048)</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-t|--type</literal></para>
+ </entry>
+ <entry>
+ <para><replaceable>type</replaceable></para>
+ </entry>
+ <entry>
+ <para>Key type (mgs, server, client)</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-k|--key-bits</literal></para>
+ </entry>
+ <entry>
+ <para><replaceable>length</replaceable></para>
+ </entry>
+ <entry>
+ <para>Shared key length in bits (Default: 256)</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-d|--data</literal></para>
+ </entry>
+ <entry>
+ <para><replaceable>file</replaceable></para>
+ </entry>
+ <entry>
+ <para>Shared key random data source (Default: /dev/random)</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-v|--verbose</literal></para>
+ </entry>
+ <entry>
+ <para></para>
+ </entry>
+ <entry>
+ <para>Increase verbosity for errors</para>
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <section remap="h4">
+ <title>Writing Key Files</title>
+ <para>Key files are generated by the <literal>lgss_sk</literal>
+ utility. Parameters are specified on the command line followed by the
+ <literal>--write</literal> parameter and the filename to write
+ to. The <literal>lgss_sk</literal> utility will not overwrite files
+ so the filename must be unique. Mandatory parameters for generating
+ key files are <literal>--type</literal>, either
+ <literal>--fsname</literal> or <literal>--mgsnids</literal>, and
+ <literal>--write</literal>; all other parameters
+ are optional.</para>
+ <para><literal>lgss_sk</literal> uses <literal>/dev/random</literal>
+ as the default entropy data source; you may override this with the
+ <literal>--data</literal> parameter. When no hardware random
+ number generator is available on the system where
+ <literal>lgss_sk</literal> is executing, you may need to press keys on
+ the keyboard or move the mouse (if directly attached to the system)
+ or cause disk IO (if system is remote), in order to generate entropy
+ for the shared key. It is possible to use
+ <literal>/dev/urandom</literal> for testing purposes but this may
+ provide less security in some cases.</para>
+ <para>Example:</para>
+ <para>To create a <emphasis>server</emphasis> type key file for the
+ <emphasis>testfs</emphasis> Lustre file system for clients in the
+ <emphasis>biology</emphasis> nodemap, enter:</para>
+ <screen>server# lgss_sk -t server -f testfs -n biology \
+-w testfs.server.biology.key</screen>
+ </section>
+ <section remap="h4">
+ <title>Modifying Key Files</title>
+ <para>Like writing key files you modify them by specifying the
+ paramaters on the command line that you want to change. Only key file
+ attributes associated with the parameters provided are changed; all
+ other attributes remain unchanged.</para>
+ <para>To modify a key file's <emphasis>Type</emphasis> to
+ <emphasis>client</emphasis> and populate the
+ <emphasis>Prime (p)</emphasis> key attribute, if it is missing,
+ enter:</para>
+ <screen>client# lgss_sk -t client -m testfs.client.biology.key</screen>
+ <para>To add MGS NIDs <literal>192.168.1.101@tcp,10.10.0.101@o2ib</literal>
+ to server key file <literal>testfs.server.biology.key</literal> and
+ client key file <literal>testfs.client.biology.key</literal>, enter
+ </para>
+ <screen>server# lgss_sk -g 192.168.1.101@tcp,10.10.0.101@o2ib \
+-m testfs.server.biology.key
+
+client# lgss_sk -g 192.168.1.101@tcp,10.10.0.101@o2ib \
+-m testfs.client.biology.key</screen>
+ <para>To modify the <literal>testfs.server.biology.key</literal> on
+ the MGS to support MGS connections from <emphasis>biology</emphasis>
+ clients, modify the key file's <emphasis>Type</emphasis> to include
+ <emphasis>mgs</emphasis> in addition to <emphasis>server</emphasis>,
+ enter:</para>
+ <screen>mgs# lgss_sk -t mgs,server -m testfs.server.biology.key</screen>
+ </section>
+ <section remap="h4">
+ <title>Reading Key Files</title>
+ <para>Read key files with the <literal>lgss_sk</literal> utility and
+ <literal>--read</literal> parameter. Read the keys modified in the
+ previous examples:</para>
+ <screen>mgs# lgss_sk -r testfs.server.biology.key
+Version: 1
+Type: mgs server
+HMAC alg: SHA256
+Crypt alg: AES-256-CTR
+Ctx Expiration: 604800 seconds
+Shared keylen: 256 bits
+Prime length: 2048 bits
+File system: testfs
+MGS NIDs: 192.168.1.101@tcp 10.10.0.101@o2ib
+Nodemap name: biology
+Shared key:
+ 0000: 84d2 561f 37b0 4a58 de62 8387 217d c30a ..V.7.JX.b..!}..
+ 0010: 1caa d39c b89f ee6c 2885 92e7 0765 c917 .......l(....e..
+
+client# lgss_sk -r testfs.client.biology.key
+Version: 1
+Type: client
+HMAC alg: SHA256
+Crypt alg: AES-256-CTR
+Ctx Expiration: 604800 seconds
+Shared keylen: 256 bits
+Prime length: 2048 bits
+File system: testfs
+MGS NIDs: 192.168.1.101@tcp 10.10.0.101@o2ib
+Nodemap name: biology
+Shared key:
+ 0000: 84d2 561f 37b0 4a58 de62 8387 217d c30a ..V.7.JX.b..!}..
+ 0010: 1caa d39c b89f ee6c 2885 92e7 0765 c917 .......l(....e..
+Prime (p) :
+ 0000: 8870 c3e3 09a5 7091 ae03 f877 f064 c7b5 .p....p....w.d..
+ 0010: 14d9 bc54 75f8 80d3 22f9 2640 0215 6404 ...Tu...".&@..d.
+ 0020: 1c53 ba84 1267 bea2 fb05 37a4 ed2d 5d90 .S...g....7..-].
+ 0030: 84e3 1a67 67f0 47c7 0c68 5635 f50e 9cf0 ...gg.G..hV5....
+ 0040: e622 6f53 2627 6af6 9598 eeed 6290 9b1e ."oS&'j.....b...
+ 0050: 2ec5 df04 884a ea12 9f24 cadc e4b6 e91d .....J...$......
+ 0060: 362f a239 0a6d 0141 b5e0 5c56 9145 6237 6/.9.m.A..\V.Eb7
+ 0070: 59ed 3463 90d7 1cbe 28d5 a15d 30f7 528b Y.4c....(..]0.R.
+ 0080: 76a3 2557 e585 a1be c741 2a81 0af0 2181 v.%W.....A*...!.
+ 0090: 93cc a17a 7e27 6128 5ebd e0a4 3335 db63 ...z~'a(^...35.c
+ 00a0: c086 8d0d 89c1 c203 3298 2336 59d8 d7e7 ........2.#6Y...
+ 00b0: e52a b00c 088f 71c3 5109 ef14 3910 fcf6 .*....q.Q...9...
+ 00c0: 0fa0 7db7 4637 bb95 75f4 eb59 b0cd 4077 ..}.F7..u..Y..@w
+ 00d0: 8f6a 2ebd f815 a9eb 1b77 c197 5100 84c0 .j.......w..Q...
+ 00e0: 3dc0 d75d 40b3 6be5 a843 751a b09c 1b20 =..]@.k..Cu....
+ 00f0: 8126 4817 e657 b004 06b6 86fb 0e08 6a53 .&H..W........jS</screen>
+ </section>
+ <section remap="h4">
+ <title>Loading Key Files</title>
+ <para>Key files can be loaded into the kernel keyring with the
+ <literal>lgss_sk</literal> utility or at mount time with the
+ <literal>skpath</literal> mount option. The <literal>skpath</literal>
+ method has the advantage that it accepts a directory path and loads
+ all key files within the directory into the keyring. The
+ <literal>lgss_sk</literal> utility loads a single key file into the
+ keyring with each invocation. Key files must not be world writable or
+ they will fail to load.</para>
+ <para>Third party tools can also load the keys if desired. The only
+ caveat is that the key must be available when the request_key upcall
+ to userspace is made and they use the correct key descriptions for a
+ key so that it can be found during the upcall
+ (see Key Descriptions).</para>
+ <para>Examples:</para>
+ <para>Load the <literal>testfs.server.biology.key</literal> key file
+ using <literal>lgss_sk</literal>,
+ enter:</para>
+ <screen>server# lgss_sk -l testfs.server.biology.key</screen>
+ <para>Use the <literal>skpath</literal> mount option to load all of
+ the key files in the <literal>/secure_directory</literal> directory
+ when mounting a storage target, enter:</para>
+ <screen>server# mount -t lustre -o skpath=/secure_directory \
+/storage/target /mount/point</screen>
+ <para>Use the <literal>skpath</literal> mount option to load key files
+ into the keyring on a client, enter:</para>
+ <screen>client# mount -t lustre -o skpath=/secure_directory \
+mgsnode:/testfs /mnt/testfs</screen>
+ </section>
+ </section>
+ </section>
+ <section xml:id="ssklustregsskeyring">
+ <title>Lustre GSS Keyring</title>
+ <para>The Lustre GSS Keyring binary <literal>lgss_keyring</literal> is
+ used by SSK to handle the upcall from kernel space into user space via
+ <literal>request-key</literal>. The purpose of
+ <literal>lgss_keyring</literal> is to create a token that is passed as
+ part of the security context initialization RPC (SEC_CTX_INIT)</para>
+ <section remap="h3">
+ <title>Setup</title>
+ <para>The Lustre GSS keyring types of flavors utilize the Linux kernel
+ keyring infrastructure to maintain keys as well as to perform the
+ upcall from kernel space to userspace for key
+ negotiation/establishment. The GSS keyring establishes a key type
+ (see “request-key(8)”) named <literal>lgssc</literal> when the Lustre
+ <literal>ptlrpc_gss</literal> kernel module is loaded. When a security
+ context must be established it creates a key and uses the
+ <literal>request-key</literal> binary in an upcall to establish the
+ key. This key will look for the configuration file in
+ <literal>/etc/request-key.d</literal> with the name
+ <replaceable>keytype</replaceable>.conf, for Lustre this is
+ <literal>lgssc.conf</literal>.</para>
+ <para>Each node participating in SSK Security must have a
+ <literal>/etc/request-key.d/lgssc.conf</literal> file that contains the
+ following single line:</para>
+ <para><literal>
+ create lgssc * * /usr/sbin/lgss_keyring %o %k %t %d %c %u %g %T %P %S
+ </literal>
+ </para>
+ <para>The <literal>request-key</literal> binary will call
+ <literal>lgss_keyring</literal> with the arguments following it with
+ their substituted values (see <literal>request-key.conf(5)</literal>).
+ </para>
+ </section>
+ <section remap="h3">
+ <title>Server Setup</title>
+ <para>Lustre servers do not use the Linux
+ <literal>request-key</literal> mechanism as clients do. Instead
+ servers run a daemon that uses a pipefs with the kernel to trigger
+ events based on read/write to a file descriptor. The server-side
+ binary is <literal>lsvcgssd</literal>. It can be executed in the
+ foreground or as a daemon. Below are the parameters for the
+ <literal>lsvcgssd</literal> binary which requires various security
+ flavors (<literal>gssnull, krb5, sk</literal>) to be enabled
+ explicitly. This ensures that only required functionality is
+ enabled.</para>
+ <table xml:id="table.lsvcgssdparameters">
+ <title>lsvcgssd Parameters</title>
+ <tgroup cols="2" align='left' colsep='1' rowsep='1'>
+ <colspec colname='c1' colwidth='1*'/>
+ <colspec colname='c2' colwidth='1*'/>
+ <thead>
+ <row>
+ <entry>
+ <para>Parameter</para>
+ </entry>
+ <entry>
+ <para>Description</para>
+ </entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>
+ <para><literal>-f</literal></para>
+ </entry>
+ <entry>
+ <para>Run in foreground</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-n</literal></para>
+ </entry>
+ <entry>
+ <para>Do not establish Kerberos credentials</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-v</literal></para>
+ </entry>
+ <entry>
+ <para>Verbosity</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-m</literal></para>
+ </entry>
+ <entry>
+ <para>Service MDS</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-o</literal></para>
+ </entry>
+ <entry>
+ <para>Service OSS</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-g</literal></para>
+ </entry>
+ <entry>
+ <para>Service MGS</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-k</literal></para>
+ </entry>
+ <entry>
+ <para>Enable Kerberos support</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-s</literal></para>
+ </entry>
+ <entry>
+ <para>Enable Shared Key support</para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para><literal>-z</literal></para>
+ </entry>
+ <entry>
+ <para>Enable <literal>gssnull</literal> support</para>
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>A SysV style init script is installed for starting and stopping
+ the <literal>lsvcgssd</literal> daemon. The init script checks the
+ <literal>LSVCGSSARGS</literal> variable in the
+ <literal>/etc/sysconfig/lsvcgss</literal> configuration file for
+ startup parameters.</para>
+ <para>Keys during the upcall on the client and handling of an RPC on
+ the server are found by using a specific key description for each key
+ in the kernel keyring.</para>
+ <para>For each MGS NID there must be a separate key loaded. The format
+ of the key description should be:</para>
+ <table xml:id="table.sskkeydescription">
+ <title>Key Descriptions</title>
+ <tgroup cols='3' align='left' colsep='1' rowsep='1'>
+ <colspec colname='c1' colwidth='1*'/>
+ <colspec colname='c2' colwidth='3*'/>
+ <colspec colname='c3' colwidth='3*'/>
+ <thead>
+ <row>
+ <entry>
+ <para>Type</para>
+ </entry>
+ <entry>
+ <para>Key Description</para>
+ </entry>
+ <entry>
+ <para>Example</para>
+ </entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>
+ <para>MGC</para>
+ </entry>
+ <entry>
+ <para>lustre:MGC<replaceable>NID</replaceable></para>
+ </entry>
+ <entry>
+ <para><literal>lustre:MGC192.168.1.10@tcp</literal></para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para>MDC/OSC/OSP/LWP</para>
+ </entry>
+ <entry>
+ <para>lustre:<replaceable>fsname</replaceable></para>
+ </entry>
+ <entry>
+ <para><literal>lustre:testfs</literal></para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para>MDT</para>
+ </entry>
+ <entry>
+ <para>lustre:<replaceable>fsname</replaceable>:<replaceable>NodemapName</replaceable></para>
+ </entry>
+ <entry>
+ <para><literal>lustre:testfs:biology</literal></para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para>OST</para>
+ </entry>
+ <entry>
+ <para>lustre:<replaceable>fsname</replaceable>:<replaceable>NodemapName</replaceable></para>
+ </entry>
+ <entry>
+ <para><literal>lustre:testfs:biology</literal></para>
+ </entry>
+ </row>
+ <row>
+ <entry>
+ <para>MGS</para>
+ </entry>
+ <entry>
+ <para>lustre:MGS</para>
+ </entry>
+ <entry>
+ <para><literal>lustre:MGS</literal></para>
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>All keys for Lustre use the <literal>user</literal> type for
+ keys and are attached to the user’s keyring. This is not
+ configurable. Below is an example showing how to list the user’s
+ keyring, load a key file, read the key, and clear the key from the
+ kernel keyring.</para>
+ <screen><emphasis role='bold'>client#</emphasis> keyctl show
+Session Keyring
+ 17053352 --alswrv 0 0 keyring: _ses
+ 773000099 --alswrv 0 65534 \_ keyring: _uid.0
+
+<emphasis role='bold'>client#</emphasis> lgss_sk -l /secure_directory/testfs.client.key
+
+<emphasis role='bold'>client#</emphasis> keyctl show
+Session Keyring
+ 17053352 --alswrv 0 0 keyring: _ses
+ 773000099 --alswrv 0 65534 \_ keyring: _uid.0
+1028795127 --alswrv 0 0 \_ user: lustre:testfs
+
+<emphasis role='bold'>client#</emphasis> keyctl pipe 1028795127 | lgss_sk -r -
+Version: 1
+Type: client
+HMAC alg: SHA256
+Crypt alg: AES-256-CTR
+Ctx Expiration: 604800 seconds
+Shared keylen: 256 bits
+Prime length: 2048 bits
+File system: testfs
+MGS NIDs:
+Nodemap name: default
+Shared key:
+ 0000: faaf 85da 93d0 6ffc f38c a5c6 f3a6 0408 ......o.........
+ 0010: 1e94 9b69 cf82 d0b9 880b f173 c3ea 787a ...i.......s..xz
+Prime (p) :
+ 0000: 9c12 ed95 7b9d 275a 229e 8083 9280 94a0 ....{.'Z".......
+ 0010: 8593 16b2 a537 aa6f 8b16 5210 3dd5 4c0c .....7.o..R.=.L.
+ 0020: 6fae 2729 fcea 4979 9435 f989 5b6e 1b8a o.')..Iy.5..[n..
+ 0030: 5039 8db2 3a23 31f0 540c 33cb 3b8e 6136 P9..:#1.T.3.;.a6
+ 0040: ac18 1eba f79f c8dd 883d b4d2 056c 0501 .........=...l..
+ 0050: ac17 a4ab 9027 4930 1d19 7850 2401 7ac4 .....'I0..xP$.z.
+ 0060: 92b4 2151 8837 ba23 94cf 22af 72b3 e567 ..!Q.7.#..".r..g
+ 0070: 30eb 0cd4 3525 8128 b0ff 935d 0ba3 0fc0 0...5%.(...]....
+ 0080: 9afa 5da7 0329 3ce9 e636 8a7d c782 6203 ..]..)<..6.}..b.
+ 0090: bb88 012e 61e7 5594 4512 4e37 e01d bdfc ....a.U.E.N7....
+ 00a0: cb1d 6bd2 6159 4c3a 1f4f 1167 0e26 9e5e ..k.aYL:.O.g.&.^
+ 00b0: 3cdc 4a93 63f6 24b1 e0f1 ed77 930b 9490 <.J.c.$....w....
+ 00c0: 25ef 4718 bff5 033e 11ba e769 4969 8a73 %.G....>...iIi.s
+ 00d0: 9f5f b7bb 9fa0 7671 79a4 0d28 8a80 1ea1 ._....vqy..(....
+ 00e0: a4df 98d6 e20e fe10 8190 5680 0d95 7c83 ..........V...|.
+ 00f0: 6e21 abb3 a303 ff55 0aa8 ad89 b8bf 7723 n!.....U......w#
+
+<emphasis role='bold'>client#</emphasis> keyctl clear @u
+
+<emphasis role='bold'>client#</emphasis> keyctl show
+Session Keyring
+ 17053352 --alswrv 0 0 keyring: _ses
+ 773000099 --alswrv 0 65534 \_ keyring: _uid.0
+</screen>
+ </section>
+ <section remap='h3'>
+ <title>Debugging GSS Keyring</title>
+ <para>Lustre client and server support several debug levels, which
+ can be seen below.</para>
+ <para>Debug levels:</para>
+ <itemizedlist>
+ <listitem>
+ <para>0 - Error</para>
+ </listitem>
+ <listitem>
+ <para>1 - Warn</para>
+ </listitem>
+ <listitem>
+ <para>2 - Info</para>
+ </listitem>
+ <listitem>
+ <para>3 - Debug</para>
+ </listitem>
+ <listitem>
+ <para>4 - Trace</para>
+ </listitem>
+ </itemizedlist>
+ <para>To set the debug level on the client use the Lustre
+ parameter:</para>
+ <para><literal>sptlrpc.gss.lgss_keyring.debug_level</literal></para>
+ <para>For example to set the debug level to trace, enter:</para>
+ <screen>client# lctl set_param sptlrpc.gss.lgss_keyring.debug_level=4</screen>
+ <para>Server-side verbosity is increased by adding additional verbose
+ flags (<literal>-v</literal>) to the command line arguments for the
+ daemon. The following command runs the <literal>lsvcgssd</literal>
+ daemon in the foreground with debug verbosity supporting gssnull and
+ SSK</para>
+ <screen>server# lsvcgssd -f -vvv -z -s</screen>
+ <para><literal>lgss_keyring</literal> is called as part of the
+ <literal>request-key</literal> upcall which has no standard output;
+ therefore logging is done through syslog. The server-side logging with
+ <literal>lsvcgssd</literal> is written to standard output when
+ executing in the foreground and to syslog in daemon mode.</para>
+ </section>
+ <section remap='h3'>
+ <title>Revoking Keys</title>
+ <para>The keys discussed above with <literal>lgss_sk</literal> and the
+ <literal>skpath</literal> mount options are not revoked. They are only
+ used to create valid contexts for client connections. Instead of
+ revoking them they can be invalidated in one of two ways.</para>
+ <itemizedlist>
+ <listitem>
+ <para>Unloading the key from the user keyring on the server will
+ cause new client connections to fail. If no longer necessary it
+ can be deleted.</para>
+ </listitem>
+ <listitem>
+ <para>Changing the nodemap name for the clients on the servers.
+ Since the nodemap is an integral part of the shared key context
+ instantiation, renaming the nodemap a group of NIDs belongs to
+ will prevent any new contexts.</para>
+ </listitem>
+ </itemizedlist>
+ <para>There currently does not exist a mechanism to flush contexts
+ from Lustre. Targets could be unmounted from the servers to purge
+ contexts. Alternatively shorter context expiration could be used
+ when the key is created so that contexts need to be refreshed more
+ frequently than the default. 3600 seconds could be reasonable
+ depending on the use case so that contexts will have to be
+ renegotiated every hour.</para>
+ </section>
+ </section>
+ <section xml:id='ssknodemaprole'>
+ <title>Role of Nodemap in SSK</title>
+ <para>SSK uses Nodemap (See <xref linkend="lustrenodemap"/>)
+ policy group names and their associated NID range(s) as a mechanism to
+ prevent key file forgery, and to control the range of NIDs on which a
+ given key file can be used.</para>
+ <para>Clients assume they are in the nodemap specified in the key file
+ they use. When clients instantiate security contexts an upcall is
+ triggered that specifies information about the context that triggers it.
+ From this context information <literal>request-key</literal> calls
+ <literal>lgss_keyring</literal>, which in turn looks up the key with
+ description lustre:<replaceable>fsname</replaceable> or
+ lustre:<replaceable>target_name</replaceable> for the MGC. Using
+ the key found in the user keyring matching the description, the nodemap
+ name is read from the key, hashed with SHA256, and sent to the server.
+ </para>
+ <para>Servers look up the client’s NID to determine which nodemap the NID
+ is associated with and sends the nodemap name to
+ <literal>lsvcgssd</literal>. The <literal>lsvcgssd</literal> daemon
+ verifies whether the HMAC equals the nodemap value sent by the client.
+ This prevents forgery and invalidates the key when a client’s NID is not
+ associated with the nodemap name defined on the servers.</para>
+ <para>It is not required to activate the Nodemap feature in order for SSK
+ to perform client NID to nodemap name lookups.</para>
+ </section>
+ <section xml:id='sskexamples'>
+ <title>SSK Examples</title>
+ <para>The examples in this section use 1 MGS/MDS (NID 172.16.0.1@tcp),
+ 1 OSS (NID 172.16.0.3@tcp), and 2 clients. The Lustre file system name is
+ <emphasis>testfs</emphasis>.</para>
+ <section remap='h3'>
+ <title>Securing Client to Server Communications</title>
+ <para>This example illustrates how to configure SSK to apply Privacy and
+ Integrity protections to client-to-server PtlRPC traffic on the
+ <literal>tcp</literal> network. Rules that specify a direction,
+ specifically <literal>cli2mdt</literal> and <literal>cli2ost</literal>,
+ are used. This permits server-to-server communications to continue using
+ <literal>null</literal> which is the <emphasis>default</emphasis> flavor
+ for all Lustre connections. This arrangement provides no server-to-server
+ protections, see <xref linkend="ssks2sexample"/>.</para>
+ <orderedlist>
+ <listitem>
+ <para>Create secure directory for storing SSK key files.</para>
+ <screen>mds# mkdir /secure_directory
+mds# chmod 600 /secure_directory
+oss# mkdir /secure_directory
+oss# chmod 600 /secure_directory
+cli1# mkdir /secure_directory
+cli1# chmod 600 /secure_directory
+cli2# mkdir /secure_directory
+cli2# chmod 600 /secure_directory</screen>
+ </listitem>
+ <listitem>
+ <para>Generate a key file for the MDS and OSS servers. Run:</para>
+ <screen>mds# lgss_sk -t server -f testfs -w \
+/secure_directory/testfs.server.key</screen>
+ </listitem>
+ <listitem>
+ <para>Securely copy the /secure_directory/testfs.server.key key
+ file to the OSS.</para>
+ <screen>mds# scp /secure_directory/testfs.server.key \
+oss:/secure_directory/</screen>
+ </listitem>
+ <listitem>
+ <para>Securely copy the
+ <literal>/secure_directory/testfs.server.key</literal> key file to
+ <literal>/secure_directory/testfs.client.key</literal> on
+ <emphasis>client1</emphasis>.</para>
+ <screen>mds# scp /secure_directory/testfs.server.key \
+client1:/secure_directory/testfs.<emphasis role="bold">client</emphasis>.key</screen>
+ </listitem>
+ <listitem>
+ <para>Modify the key file type to <literal>client</literal> on
+ <emphasis>client1</emphasis>. This operation also generates a prime
+ number of <literal>Prime length</literal> to populate the
+ <literal>Prime (p)</literal> attribute. Run:</para>
+ <screen>client1# lgss_sk -t client \
+-m /secure_directory/testfs.client.key</screen>
+ </listitem>
+ <listitem>
+ <para>Create a <literal>/etc/request-key.d/lgssc.conf</literal> file
+ on all nodes that contains this line
+ '<literal>create lgssc * * /usr/sbin/lgss_keyring %o %k %t %d %c %u
+ %g %T %P %S</literal>' without the single quotes. Run:</para>
+ <screen>mds# echo create lgssc \* \* /usr/sbin/lgss_keyring %o %k %t %d %c %u %g %T %P %S > /etc/request-key.d/lgssc.conf
+oss# echo create lgssc \* \* /usr/sbin/lgss_keyring %o %k %t %d %c %u %g %T %P %S > /etc/request-key.d/lgssc.conf
+client1# echo create lgssc \* \* /usr/sbin/lgss_keyring %o %k %t %d %c %u %g %T %P %S > /etc/request-key.d/lgssc.conf
+client2# echo create lgssc \* \* /usr/sbin/lgss_keyring %o %k %t %d %c %u %g %T %P %S > /etc/request-key.d/lgssc.conf</screen>
+ </listitem>
+ <listitem>
+ <para>Configure the <literal>lsvcgss</literal> daemon on the MDS and
+ OSS. Set the <literal>LSVCGSSDARGS</literal> variable in
+ <literal>/etc/sysconfig/lsvcgss</literal> on the MDS to
+ <literal>‘-s -m’</literal>. On the OSS, set the
+ <literal>LSVCGSSDARGS</literal> variable in
+ <literal>/etc/sysconfig/lsvcgss</literal> to
+ <literal>‘-s -o’</literal></para>
+ </listitem>
+ <listitem>
+ <para>Start the <literal>lsvcgssd</literal> daemon on the MDS and
+ OSS. Run:</para>
+ <screen>mds# systemctl start lsvcgss.service
+oss# systemctl start lsvcgss.service</screen>
+ </listitem>
+ <listitem>
+ <para>Mount the MDT and OST with the
+ <literal>-o skpath=/secure_directory</literal> mount option. The
+ <literal>skpath</literal> option loads all SSK key files found in the
+ directory into the kernel keyring.</para>
+ </listitem>
+ <listitem>
+ <para> Set client to MDT and client to OST security flavor to SSK
+ Privacy and Integrity, <literal>skpi</literal>:</para>
+ <screen>mds# lctl conf_param testfs.srpc.flavor.tcp.cli2mdt=skpi
+mds# lctl conf_param testfs.srpc.flavor.tcp.cli2ost=skpi</screen>
+ </listitem>
+ <listitem>
+ <para>Mount the testfs file system on client1 and client2:</para>
+ <screen>client1# mount -t lustre -o skpath=/secure_directory 172.16.0.1@tcp:/testfs /mnt/testfs
+client2# mount -t lustre -o skpath=/secure_directory 172.16.0.1@tcp:/testfs /mnt/testfs
+mount.lustre: mount 172.16.0.1@tcp:/testfs at /mnt/testfs failed: Connection refused</screen>
+ </listitem>
+ <listitem>
+ <para><emphasis>client2</emphasis> failed to authenticate because it
+ does not have a valid key file. Repeat steps 4 and 5, substitute
+ client1 for client2, then mount the testfs file system on
+ client2:</para>
+ <screen>client2# mount -t lustre -o skpath=/secure_directory 172.16.0.1@tcp:/testfs /mnt/testfs</screen>
+ </listitem>
+ <listitem>
+ <para>Verify that the <literal>mdc</literal> and
+ <literal>osc</literal> connections are using the SSK mechanism and
+ that <literal>rpc</literal> and <literal>bulk</literal> security
+ flavors are <literal>skpi</literal>. See
+ <xref linkend="ssksptlrpcctx"/>.</para>
+ <para>Notice the <literal>mgc</literal> connection to the MGS has no
+ secure PtlRPC security context. This is because
+ <literal>skpi</literal> security was only specified for
+ client-to-MDT and client-to-OST connections in step 10. The
+ following example details the steps necessary to secure the
+ connection to the MGS.</para>
+ </listitem>
+ </orderedlist>
+ </section>
+ <section remap='h3'>
+ <title>Securing MGS Communications</title>
+ <para>This example builds on the previous example.</para>
+ <orderedlist>
+ <listitem>
+ <para>Enable <literal>lsvcgss</literal> MGS service support on MGS.
+ Edit <literal>/etc/sysconfig/lsvcgss</literal> on the MGS and add
+ the (<literal>-g</literal>) parameter to the
+ <literal>LSVCGSSDARGS</literal> variable. Restart the
+ <literal>lsvcgss</literal> service.</para>
+ </listitem>
+ <listitem>
+ <para>Add <emphasis>mgs</emphasis> key type and
+ <emphasis>MGS NIDs</emphasis> to
+ <literal>/secure_directory/testfs.server.key</literal>
+ on MDS.</para>
+ <screen>mgs# lgss_sk -t mgs,server -g 172.16.0.1@tcp,172.16.0.2@tcp -m /secure_directory/testfs.server.key</screen>
+ </listitem>
+ <listitem>
+ <para>Load the modified key file on the MGS. Run:</para>
+ <screen>mgs# lgss_sk -l /secure_directory/testfs.server.key</screen>
+ </listitem>
+ <listitem>
+ <para>Add <emphasis>MGS NIDs</emphasis> to
+ <literal>/secure_directory/testfs.client.key</literal>
+ on client, client1.</para>
+ <screen>client1# lgss_sk -g 172.16.0.1@tcp,172.16.0.2@tcp -m /secure_directory/testfs.client.key</screen>
+ </listitem>
+ <listitem>
+ <para>Unmount the testfs file system on client1, then mount with the
+ <literal>mgssec=skpi</literal> mount option:</para>
+ <screen>cli1# mount -t lustre -o mgssec=skpi,skpath=/secure_directory 172.16.0.1@tcp:/testfs /mnt/testfs</screen>
+ </listitem>
+ <listitem>
+ <para>Verify that client1’s MGC connection is using the SSK mechanism
+ and <literal>skpi</literal> security flavor. See
+ <xref linkend="ssksptlrpcctx"/>.</para>
+ </listitem>
+ </orderedlist>
+ </section>
+ <section xml:id='ssks2sexample' remap='h3'>
+ <title>Securing Server to Server Communications</title>
+ <para>This example illustrates how to configure SSK to apply
+ <emphasis>Integrity</emphasis> protection, <literal>ski</literal> flavor,
+ to MDT-to-OST PtlRPC traffic on the <literal>tcp</literal> network.</para>
+ <para>This example builds on the previous example.</para>
+ <orderedlist>
+ <listitem>
+ <para>Create a Nodemap policy group named
+ <literal>LustreServers</literal> on the MGS for the Lustre Servers,
+ enter:</para>
+ <screen>mgs# lctl nodemap_add LustreServers</screen>
+ </listitem>
+ <listitem>
+ <para>Add MDS and OSS NIDs to the LustreServers nodemap, enter:</para>
+ <screen>mgs# lctl nodemap_add_range --name LustreServers --range 172.16.0.[1-3]@tcp</screen>
+ </listitem>
+ <listitem>
+ <para>Create key file of type <literal>mgs,server</literal> for use
+ with nodes in the <emphasis>LustreServers</emphasis> Nodemap range.
+ </para>
+ <screen>mds# lgss_sk -t mgs,server -f testfs -g \
+172.16.0.1@tcp,172.16.0.2@tcp -n LustreServers -w \
+/secure_directory/testfs.LustreServers.key</screen>
+ </listitem>
+ <listitem>
+ <para>Securely copy the
+ <literal>/secure_directory/testfs.LustreServers.key</literal> key
+ file to the OSS.</para>
+ <screen>mds# scp /secure_directory/testfs.LustreServers.key oss:/secure_directory/</screen>
+ </listitem>
+ <listitem>
+ <para>On the MDS and OSS, copy
+ <literal>/secure_directory/testfs.LustreServers.key</literal> to
+ <literal>/secure_directory/testfs.LustreServers.client.key</literal>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>On each server modify the key file type of
+ <literal>/secure_directory/testfs.LustreServers.client.key</literal>
+ to be of type client. This operation also generates a prime number
+ of <emphasis>Prime length</emphasis> to populate the
+ <emphasis>Prime (p)</emphasis> attribute. Run:</para>
+ <screen>mds# lgss_sk -t client -m \
+/secure_directory/testfs.LustreServers.client.key
+oss# lgss_sk -t client -m \
+/secure_directory/testfs.LustreServers.client.key</screen>
+ </listitem>
+ <listitem>
+ <para>Load the
+ <literal>/secure_directory/testfs.LustreServers.key</literal> and
+ <literal>/secure_directory/testfs.LustreServers.client.key</literal>
+ key files into the keyring on the MDS and OSS, enter:</para>
+ <screen>mds# lgss_sk -l /secure_directory/testfs.LustreServers.key
+mds# lgss_sk -l /secure_directory/testfs.LustreServers.client.key
+oss# lgss_sk -l /secure_directory/testfs.LustreServers.key
+oss# lgss_sk -l /secure_directory/testfs.LustreServers.client.key</screen>
+ </listitem>
+ <listitem>
+ <para>Set MDT to OST security flavor to SSK Integrity,
+ <literal>ski</literal>:</para>
+ <screen>mds# lctl conf_param testfs.srpc.flavor.tcp.mdt2ost=ski</screen>
+ </listitem>
+ <listitem>
+ <para>Verify that the <literal>osc</literal> and
+ <literal>osp</literal> connections to the OST have a secure
+ <literal>ski</literal> security context. See
+ <xref linkend="ssksptlrpcctx"/>.</para>
+ </listitem>
+ </orderedlist>
+ </section>
+ </section>
+ <section xml:id='ssksptlrpcctx'>
+ <title>Viewing Secure PtlRPC Contexts</title>
+ <para>From the client (or servers which have mgc, osc, mdc contexts) you
+ can view info regarding all users’ contexts and the flavor in use for an
+ import. For user’s contexts (srpc_context), SSK and gssnull only support
+ a single root UID so there should only be one context. The other file in
+ the import (srpc_info) has additional sptlrpc details. The
+ <literal>rpc</literal> and <literal>bulk</literal> flavors allow you to
+ verify which security flavor is in use.</para>
+ <screen>client1# lctl get_param *.*.srpc_*
+mdc.testfs-MDT0000-mdc-ffff8800da9f0800.srpc_contexts=
+ffff8800da9600c0: uid 0, ref 2, expire 1478531769(+604695), fl uptodate,cached,, seq 7, win 2048, key 27a24430(ref 1), hdl 0xf2020f47cbffa93d:0xc23f4df4bcfb7be7, <emphasis role="bold">mech: sk</emphasis>
+mdc.testfs-MDT0000-mdc-ffff8800da9f0800.srpc_info=
+<emphasis role="bold">rpc flavor: skpi</emphasis>
+<emphasis role="bold">bulk flavor: skpi</emphasis>
+flags: rootonly,udesc,
+id: 3
+refcount: 3
+nctx: 1
+gc internal 3600
+gc next 3505
+mgc.MGC172.16.0.1@tcp.srpc_contexts=
+ffff8800dbb09b40: uid 0, ref 2, expire 1478531769(+604695), fl uptodate,cached,, seq 18, win 2048, key 3e3f709f(ref 1), hdl 0xf2020f47cbffa93b:0xc23f4df4bcfb7be6, <emphasis role="bold">mech: sk</emphasis>
+mgc.MGC172.16.0.1@tcp.srpc_info=
+<emphasis role="bold">rpc flavor: skpi</emphasis>
+<emphasis role="bold">bulk flavor: skpi</emphasis>
+flags: -,
+id: 2
+refcount: 3
+nctx: 1
+gc internal 3600
+gc next 3505
+osc.testfs-OST0000-osc-ffff8800da9f0800.srpc_contexts=
+ffff8800db9e5600: uid 0, ref 2, expire 1478531770(+604696), fl uptodate,cached,, seq 3, win 2048, key 3f7c1d70(ref 1), hdl 0xf93e61c64b6b415d:0xc23f4df4bcfb7bea, <emphasis role="bold">mech: sk</emphasis>
+osc.testfs-OST0000-osc-ffff8800da9f0800.srpc_info=
+<emphasis role="bold">rpc flavor: skpi</emphasis>
+<emphasis role="bold">bulk flavor: skpi</emphasis>
+flags: rootonly,bulk,
+id: 6
+refcount: 3
+nctx: 1
+gc internal 3600
+gc next 3505</screen>
+ </section>
+</chapter>
git://git.whamcloud.com / doc / manual.git / commitdiff