From: Andreas Dilger Date: Fri, 29 Sep 2023 00:25:30 +0000 (-0600) Subject: LUDOC-11 dne: fix examples for default dir layout X-Git-Url: https://git.whamcloud.com/?a=commitdiff_plain;h=refs%2Fchanges%2F52%2F52552%2F2;p=doc%2Fmanual.git LUDOC-11 dne: fix examples for default dir layout Improve wording around default directory layouts, and discourage the use of default striped directories. Fix the examples given for enabling default directory layouts during upgrade to actually include the "-D" option. Signed-off-by: Andreas Dilger Change-Id: I13dee2b7fc4d80db4ac9e33a193cc3a4773ebbe5 Reviewed-on: https://review.whamcloud.com/c/doc/manual/+/52552 Tested-by: jenkins Reviewed-by: Lai Siyao --- diff --git a/LustreOperations.xml b/LustreOperations.xml index 5639dab..44d737f 100644 --- a/LustreOperations.xml +++ b/LustreOperations.xml @@ -440,17 +440,14 @@ client# mount -t lustre mgsnode@tcp0:/bar /mnt/bar files and sub-directories, to be stored on specific MDTs. To create a sub-directory on a given MDT use the command: - -client# lfs mkdir –i -mdt_index -/mount_point/remote_dir + +client# lfs mkdir -i mdt_index /mount_point/remote_dir This command will allocate the sub-directory - remote_dir onto the MDT of index - mdt_index. For more information on adding additional MDTs - and - mdt_index see - . + remote_dir onto the MDT with index + mdt_index. For more information on adding additional + MDTs and mdt_index see . + An administrator can allocate remote sub-directories to separate MDTs. Creating remote sub-directories in parent directories not hosted on @@ -500,39 +497,41 @@ client# lfs mkdir –i striping metadata Creating a directory striped across multiple MDTs - The Lustre 2.8 DNE feature enables individual files in a given - directory to store their metadata on separate MDTs (a striped - directory) once additional MDTs have been added to the + The Lustre 2.8 DNE feature enables files in a single large + directory to be distributed across multiple MDTs (a striped + directory), if there are mutliple MDTs added to the filesystem, see . - The result of this is that metadata requests for - files in a striped directory are serviced by multiple MDTs and metadata + The result is that metadata requests for files in a single large + striped directory are serviced by multiple MDTs and metadata service load is distributed over all the MDTs that service a given directory. By distributing metadata service load over multiple MDTs, - performance can be improved beyond the limit of single MDT - performance. Prior to the development of this feature all files in a - directory must record their metadata on a single MDT. + performance of very large directories can be improved beyond the limit + of one MDT. Normally, all files in a directory must be created + on a single MDT. This command to stripe a directory over mdt_count MDTs is: - - -client# lfs mkdir -c -mdt_count -/mount_point/new_directory + +client# lfs mkdir -c mdt_count /mount_point/new_directory + The striped directory feature is most useful for distributing - single large directories (50k entries or more) across multiple MDTs, - since it incurs more overhead than non-striped directories. + a single large directory (50k entries or more) across multiple MDTs. + This should be used with discretion since creating and removing striped + directories incurs more overhead than non-striped directories.
Directory creation by space/inode usage If the starting MDT is not specified when creating a new directory, this directory and its stripes will be distributed on MDTs by space usage. - For example the following will create a directory and its stripes on MDTs - with balanced space usage: - lfs mkdir -c 2 <dir1> + For example the following will create a new directory on an MDT + preferring one that has less space usage: + lfs mkdir -c 1 -i -1 dir1 Alternatively, if a default directory stripe is set on a directory, - the subsequent syscall mkdir under - <dir1> will have the same effect: - lfs setdirstripe -D -c 2 <dir1> + the subsequent use of mkdir for subdirectories in + dir1 will have the same effect: + +client# lfs setdirstripe -D -c 1 -i -1 dir1 + + The policy is: If free inodes/blocks on all MDT are almost the same, diff --git a/SettingUpLustreSystem.xml b/SettingUpLustreSystem.xml index ce887ef..94eb9a4 100644 --- a/SettingUpLustreSystem.xml +++ b/SettingUpLustreSystem.xml @@ -125,11 +125,11 @@ at creation time using the lfs mkdir -c stripe_count command, where stripe_count is often the - number of MDTs in the filesystem. Striped directories should typically - not be used for all directories in the filesystem, since this incurs - extra overhead compared to non-striped directories, but is useful for - larger directories (over 50k entries) where many output files are being - created at one time. + number of MDTs in the filesystem. Striped directories should + not be used for all directories in the filesystem, since this + incurs extra overhead compared to unstriped directories. This is indended + for specific applications where many output files are being created in + one large directory (over 50k entries).
@@ -782,11 +782,14 @@ over multiple MDTs with the lfs mkdir -c command, which increases the single directory limit by a factor of the number of directory stripes used. + In the 2.12 release, the + large_dir feature of ldiskfs was added to + allow the use of directories over 10M entries, but not enabled + by default. + Starting in the 2.14 release, the - large_dir feature of ldiskfs is enabled by - default to allow directories with more than 10M entries. In - the 2.12 release, the large_dir feature was - present but not enabled by default. + large_dir feature is enabled by default. + diff --git a/UpgradingLustre.xml b/UpgradingLustre.xml index 16aaeb4..5ba6137 100644 --- a/UpgradingLustre.xml +++ b/UpgradingLustre.xml @@ -317,11 +317,19 @@ client# lfs mkdir -c 2 /testfs/new_striped_dir In Lustre 2.13 and later, it is possible to set - the default striping on existing directories - so that new remote subdirectories are created on less-full MDTs: + the default directory layout on existing + directories so new remote subdirectories are created on less-full + MDTs: -client# lfs setdirstripe -c 1 -i -1 /testfs/some_dir +client# lfs setdirstripe -D -c 1 -i -1 /testfs/some_dir + See for details. + + In Lustre 2.15 and later, if no default + directory layout is set on the root directory, the MDS will + automatically set the default directory + layout the root directory to distribute the top-level directories + round-robin across all MDTs, see .