From: Andreas Dilger Date: Tue, 16 Oct 2018 04:45:13 +0000 (-0600) Subject: LUDOC-11 misc: remove some 'l23' conditions add 'l2C' X-Git-Tag: 2.12.0~11 X-Git-Url: https://git.whamcloud.com/?a=commitdiff_plain;h=fcafa5bebef80213b4a1822edd83edc26894eccb;p=doc%2Fmanual.git LUDOC-11 misc: remove some 'l23' conditions add 'l2C' Remove many of the old Lustre 2.3 annotations from the manual, since there are few, if any, users working with such old versions of Lustre, and they can always reference an older version of the manual. Improve content in affected sections of the manual. Add the 'l2C' condition for the Lustre 2.12 release. This is needed to document the new RPC checksum types added to the 2.12 release. Signed-off-by: Andreas Dilger Change-Id: I8d10c6451899a470a12600452bc905ad0b545889 Reviewed-on: https://review.whamcloud.com/33378 Tested-by: Jenkins Reviewed-by: Joseph Gmitter --- diff --git a/BackupAndRestore.xml b/BackupAndRestore.xml index d127971..7b34f55 100644 --- a/BackupAndRestore.xml +++ b/BackupAndRestore.xml @@ -432,9 +432,8 @@ Changelog records consumed: 42 described in this section. File-level restore of an MDT is not possible before Lustre software release 2.3, as the Object Index (OI) file cannot be rebuilt after restore without the OI Scrub functionality. - Since Lustre software release 2.3, - Object Index files are automatically rebuilt at first mount after a - restore is detected (see + Since Lustre 2.3, Object Index files are automatically rebuilt at first + mount after a restore is detected (see LU-957), and file-level backup is supported (see ). @@ -787,17 +786,15 @@ trusted.fid= \ [oss]# lctl get_param -n osd-${fstype}.${fsname}-${target}.oi_scrub - If the file system was used between the time the - backup was made and when it was restored, then the online - LFSCK tool (part of Lustre code after version 2.3) - will automatically be - run to ensure the file system is coherent. If all of the device file - systems were backed up at the same time after the entire Lustre file system - was stopped, this step is unnecessary. In either case, the file system will - be immediately although there may be I/O errors reading - from files that are present on the MDT but not the OSTs, and files that - were created after the MDT backup will not be accessible or visible. See - for details on using LFSCK. + If the file system was used between the time the backup was made and + when it was restored, then the online LFSCK tool will + automatically be run to ensure the filesystem is coherent. If all of the + device filesystems were backed up at the same time after Lustre was + was stopped, this step is unnecessary. In either case, the filesystem + will be immediately although there may be I/O errors reading + from files that are present on the MDT but not the OSTs, and files that + were created after the MDT backup will not be accessible or visible. See + for details on using LFSCK.
diff --git a/ConfigurationFilesModuleParameters.xml b/ConfigurationFilesModuleParameters.xml index 0a7b59f..e44a5ff 100644 --- a/ConfigurationFilesModuleParameters.xml +++ b/ConfigurationFilesModuleParameters.xml @@ -276,7 +276,7 @@ forwarding ("")
<indexterm><primary>configuring</primary><secondary>network</secondary><tertiary>rnet_htable_size</tertiary></indexterm> <literal>rnet_htable_size</literal> - rnet_htable_size is an integer that indicates how many remote networks the internal LNet hash table is configured to handle. rnet_htable_size is used for optimizing the hash table size and does not put a limit on how many remote networks you can have. The default hash table size when this parameter is not specified is: 128. + rnet_htable_size is an integer that indicates how many remote networks the internal LNet hash table is configured to handle. rnet_htable_size is used for optimizing the hash table size and does not put a limit on how many remote networks you can have. The default hash table size when this parameter is not specified is: 128.
diff --git a/LustreProc.xml b/LustreProc.xml index efa9c9c..0148d04 100644 --- a/LustreProc.xml +++ b/LustreProc.xml @@ -1225,6 +1225,18 @@ osc.myth-OST0001-osc-ffff8804296c2800.checksum_type crc32, crc32c + + In Lustre release 2.12 additional checksum types were added to + allow end-to-end checksum integration with T10-PI capable + hardware. The client will compute the appropriate checksum + type, based on the checksum type used by the storage, for the + RPC checksum, which will be verified by the server and passed + on to the storage. The T10-PI checksum types are: + t10ip512, + t10ip4K, + t10crc512, + t10crc4K + osc.osc_instance.max_dirty_mb @@ -1319,10 +1331,11 @@ write RPCs in flight: 0
Tuning File Readahead File readahead is triggered when two or more sequential reads - by an application fail to be satisfied by data in the Linux buffer - cache. The size of the initial readahead is 1 MB. Additional - readaheads grow linearly and increment until the readahead cache on - the client is full at 40 MB. + by an application fail to be satisfied by data in the Linux buffer + cache. The size of the initial readahead is determined by the RPC + size and the file stripe size, but will typically be at least 1 MiB. + Additional readaheads grow linearly and increment until the per-file + or per-system readahead cache limit on the client is reached. Readahead tunables include: diff --git a/LustreTuning.xml b/LustreTuning.xml index e4970f1..20cfee6 100644 --- a/LustreTuning.xml +++ b/LustreTuning.xml @@ -149,8 +149,8 @@ lctl {get,set}_param {service}.thread_{min,max,started} service immediately and disables automatic thread creation behavior. - Lustre software release 2.3 introduced new - parameters to provide more control to administrators. + Parameters are available to provide administrators control + over the number of service threads. @@ -167,16 +167,15 @@ lctl {get,set}_param {service}.thread_{min,max,started}
-
+
<indexterm> <primary>tuning</primary> <secondary>MDS binding</secondary> </indexterm>Binding MDS Service Thread to CPU Partitions - With the introduction of Node Affinity ( - ) in Lustre software release 2.3, MDS threads - can be bound to particular CPU partitions (CPTs) to improve CPU cache - usage and memory locality. Default values for CPT counts and CPU core + With the Node Affinity () feature, + MDS threads can be bound to particular CPU partitions (CPTs) to improve CPU + cache usage and memory locality. Default values for CPT counts and CPU core bindings are selected automatically to provide good overall performance for a given CPU count. However, an administrator can deviate from these setting if they choose. For details on specifying the mapping of CPU cores to @@ -268,17 +267,16 @@ options ksocklnd enable_irq_affinity=0 By default, this parameter is off. As always, you should test the performance to compare the impact of changing this parameter.
-
+
<indexterm> <primary>tuning</primary> <secondary>Network interface binding</secondary> </indexterm>Binding Network Interface Against CPU Partitions - Lustre software release 2.3 and beyond provide enhanced network - interface control. The enhancement means that an administrator can bind - an interface to one or more CPU partitions. Bindings are specified as - options to the LNet modules. For more information on specifying module - options, see + Lustre allows enhanced network interface control. This means that + an administrator can bind an interface to one or more CPU partitions. + Bindings are specified as options to the LNet modules. For more + information on specifying module options, see For example, o2ib0(ib0)[0,1] will ensure that all messages for @@ -324,9 +322,9 @@ ksocklnd credits=256 ko2iblnd credits=256 - - In Lustre software release 2.3 and beyond, LNet may revalidate - the NI credits, so the administrator's request may not persist. + + LNet may revalidate the NI credits, so the administrator's + request may not persist.
@@ -375,10 +373,9 @@ ko2iblnd credits=256 lnet large_router_buffers=8192 - - In Lustre software release 2.3 and beyond, LNet may revalidate - the router buffer setting, so the administrator's request may not - persist. + + LNet may revalidate the router buffer setting, so the + administrator's request may not persist.
@@ -525,16 +522,16 @@ lnet large_router_buffers=8192 be MAX.
-
+
<indexterm> <primary>tuning</primary> <secondary>libcfs</secondary> </indexterm>libcfs Tuning - Lustre software release 2.3 introduced binding service threads via - CPU Partition Tables (CPTs). This allows the system administrator to - fine-tune on which CPU cores the Lustre service threads are run, for both - OSS and MDS services, as well as on the client. + Lustre allows binding service threads via CPU Partition Tables + (CPTs). This allows the system administrator to fine-tune on which CPU + cores the Lustre service threads are run, for both OSS and MDS services, + as well as on the client. CPTs are useful to reserve some cores on the OSS or MDS nodes for system functions such as system monitoring, HA heartbeat, or similar diff --git a/SystemConfigurationUtilities.xml b/SystemConfigurationUtilities.xml index 1ee5736..44a6426 100644 --- a/SystemConfigurationUtilities.xml +++ b/SystemConfigurationUtilities.xml @@ -817,12 +817,25 @@ root@oss1# ll_decode_filter_fid #12345[4,5,8]
-
+
<indexterm><primary>ll_recover_lost_found_objs</primary></indexterm> ll_recover_lost_found_objs - The ll_recover_lost_found_objs utility helps recover Lustre OST objects (file data) from a lost and found directory and return them to their correct locations. - - Running the ll_recover_lost_found_objs tool is not strictly necessary to bring an OST back online, it just avoids losing access to objects that were moved to the lost and found directory due to directory corruption. + The ll_recover_lost_found_objs utility was + used to help recover Lustre OST objects (file data) from the + lost+found directory of an OST and return them to + their correct locations based on information stored in the + trusted.fid extended attribute stored on every + OST object containing data. + This utility is not needed with Lustre 2.6 + and later, and is removed in Lustre 2.8 since LFSCK + online scanning will automatically move objects from + lost+found to the proper place in the OST. + + + The ll_recover_lost_found_objs tool is not + strictly necessary to bring an OST back online, it just avoids losing + access to objects that were moved to the lost+found directory due to + directory corruption on the OST.
Synopsis @@ -832,9 +845,6 @@ ll_recover_lost_found_objs Description The first time Lustre modifies an object, it saves the MDS inode number and the objid as an extended attribute on the object, so in case of directory corruption of the OST, it is possible to recover the objects. Running e2fsck fixes the corrupted OST directory, but it puts all of the objects into a lost and found directory, where they are inaccessible to Lustre. Use the ll_recover_lost_found_objs utility to recover all (or at least most) objects from a lost and found directory and return them to the O/0/d* directories. To use ll_recover_lost_found_objs, mount the file system locally (using the -t ldiskfs, or -t zfs command), run the utility and then unmount it again. The OST must not be mounted by Lustre when ll_recover_lost_found_objs is run. - This utility is not needed for 2.6 and later, - since the LFSCK online scanning will move objects - from lost+found to the proper place in the OST.
Options @@ -2110,15 +2120,16 @@ mount.lustre user_fid2path - Enable FID to path translation by regular - users. Note: This option allows a potential security hole - because it allows regular users direct access to a file by its - File ID, bypassing POSIX path-based permission checks which - could otherwise prevent the user from accessing a file in a - directory that they do not have access to. Regular permission - checks are still performed on the file itself, so the user - cannot access a file to which they have no access rights. - + Enable FID-to-path translation by regular users. + + This option allows a potential security hole because + it allows regular users direct access to a file by its Lustre + File ID. This bypasses POSIX path-based permission checks, + and could allow the user to access a file in a directory that + they do not have access to. Regular POSIX file mode and ACL + permission checks are still performed on the file itself, so + users cannot access a file to which they have no permission. + @@ -2126,7 +2137,7 @@ mount.lustre nouser_fid2path - Disable FID to path translation by + Disable FID to path translation by regular users. Root and processes with CAP_DAC_READ_SEARCH can still perform FID to path translation. diff --git a/TroubleShootingRecovery.xml b/TroubleShootingRecovery.xml index 3015893..fc30da0 100644 --- a/TroubleShootingRecovery.xml +++ b/TroubleShootingRecovery.xml @@ -57,14 +57,9 @@ Dec 29 14:11:32 mookie kernel: Remounting filesystem read-only In the vast majority of cases, the Lustre software can cope with any inconsistencies found on the disk and between other devices in the file system. - - The legacy offline-LFSCK tool included with e2fsprogs is rarely - required for Lustre file system operation. offline-LFSCK is not to be - confused with LFSCK tool, which is part of Lustre and provides online - consistency checking. - For problem analysis, it is strongly recommended that - e2fsck be run under a logger, like script, to record all + e2fsck be run under a logger, like + script, to record all of the output and changes that are made to the file system in case this information is needed later. If time permits, it is also a good idea to first run @@ -105,7 +100,7 @@ root# e2fsck -fp /dev/sda # fix errors with prudent answers (usually corruption of Lustre file system Recovering from Corruption in the Lustre File System In cases where an ldiskfs MDT or OST becomes corrupt, you need to run - e2fsck to correct the local filesystem consistency, then use + e2fsck to ensure local filesystem consistency, then use LFSCK to run a distributed check on the file system to resolve any inconsistencies between the MDTs and OSTs, or among MDTs. @@ -186,7 +181,7 @@ root# e2fsck -fp /dev/sda # fix errors with prudent answers (usually (Version-based Recovery).
-
+
<indexterm> <primary>recovery</primary> @@ -196,34 +191,34 @@ root# e2fsck -fp /dev/sda # fix errors with prudent answers (usually <literal> <primary>recovery</primary> <secondary>LFSCK</secondary> </indexterm>Checking the file system with LFSCK - LFSCK is an administrative tool introduced in Lustre - software release 2.3 for checking and repair of the attributes specific to a - mounted Lustre file system. It is similar in concept to an offline fsck repair - tool for a local filesystem, but LFSCK is implemented to run as part of the - Lustre file system while the file system is mounted and in use. This allows - consistency of checking and repair by the Lustre software without unnecessary - downtime, and can be run on the largest Lustre file systems with negligible - disruption to normal operations. - Since Lustre software release 2.3, LFSCK can verify - and repair the Object Index (OI) table that is used internally to map - Lustre File Identifiers (FIDs) to MDT internal ldiskfs inode numbers, in - an internal table called the OI Table. An OI Scrub traverses the OI table - and makes corrections where necessary. An OI Scrub is required after - restoring from a file-level MDT backup ( - ), or in case the OI Table is - otherwise corrupted. Later phases of LFSCK will add further checks to the - Lustre distributed file system state. + LFSCK is an administrative tool for checking and repair of the + attributes specific to a mounted Lustre file system. It is similar + in concept to an offline fsck repair tool for a local filesystem, + but LFSCK is implemented to run as part of the Lustre file system + while the file system is mounted and in use. This allows consistency + checking and repair of Lustre-specific metadata without unnecessary + downtime, and can be run on the largest Lustre file systems with + minimal impact to normal operations. + LFSCK can verify + and repair the Object Index (OI) table that is used internally to map + Lustre File Identifiers (FIDs) to MDT internal ldiskfs inode numbers, in + an internal table called the OI Table. An OI Scrub traverses the OI table + and makes corrections where necessary. An OI Scrub is required after + restoring from a file-level MDT backup ( + ), or in case the OI Table is + otherwise corrupted. Later phases of LFSCK will add further checks to the + Lustre distributed file system state. In Lustre software release 2.4, LFSCK namespace - scanning can verify and repair the directory FID-in-dirent and LinkEA - consistency. + scanning can verify and repair the directory FID-in-dirent and LinkEA + consistency. In Lustre software release 2.6, LFSCK layout scanning - can verify and repair MDT-OST file layout inconsistencies. File layout - inconsistencies between MDT-objects and OST-objects that are checked and - corrected include dangling reference, unreferenced OST-objects, mismatched - references and multiple references. + can verify and repair MDT-OST file layout inconsistencies. File layout + inconsistencies between MDT-objects and OST-objects that are checked and + corrected include dangling reference, unreferenced OST-objects, mismatched + references and multiple references. In Lustre software release 2.7, LFSCK layout scanning - is enhanced to support verify and repair inconsistencies between multiple - MDTs. + is enhanced to support verify and repair inconsistencies between multiple + MDTs. Control and monitoring of LFSCK is through LFSCK and the /proc file system interfaces. LFSCK supports three types of interface: switch interface, status interface, and adjustment interface. diff --git a/UpgradingLustre.xml b/UpgradingLustre.xml index d009f88..b2dec8d 100644 --- a/UpgradingLustre.xml +++ b/UpgradingLustre.xml @@ -146,7 +146,7 @@ xml:id="upgradinglustre"> ea_inode option reseults in ea_inode in the file system feature list. - + To generate a list of all files with more than 160 stripes use lfs find with the --stripe-count option: diff --git a/UserUtilities.xml b/UserUtilities.xml index 7314279..a3eb0c3 100644 --- a/UserUtilities.xml +++ b/UserUtilities.xml @@ -1631,7 +1631,7 @@ File size of /mnt/lustre/foo is 1468297786 (1433888 blocks of 1024 bytes) - Enables/disables FID to path translation by + Enables/disables FID to path translation by regular users diff --git a/style/customstyle_common.xsl b/style/customstyle_common.xsl index 6c54395..f62c74f 100644 --- a/style/customstyle_common.xsl +++ b/style/customstyle_common.xsl @@ -86,9 +86,15 @@ + + + + + + - + @@ -182,6 +188,9 @@ L 2.11 + + L 2.12 + L ?.? diff --git a/style/customstyle_fo.xsl b/style/customstyle_fo.xsl index 7c4149f..ffa8b2f 100644 --- a/style/customstyle_fo.xsl +++ b/style/customstyle_fo.xsl @@ -155,6 +155,7 @@ Introduced in Lustre 2.9 Introduced in Lustre 2.10 Introduced in Lustre 2.11 + Introduced in Lustre 2.11 Documentation Error: unrecognised condition attribute @@ -194,6 +195,7 @@ L 2.9 L 2.10 L 2.11 + L 2.11