From: Patrick Farrell Date: Tue, 25 Jun 2019 21:22:46 +0000 (-0400) Subject: LUDOC-443 stripe: Add overstriping to manual X-Git-Tag: 2.13.0~17 X-Git-Url: https://git.whamcloud.com/?a=commitdiff_plain;h=c13474d938685818c03e937c36d186bff5cfab48;p=doc%2Fmanual.git LUDOC-443 stripe: Add overstriping to manual Add overstriping to Lustre manual. Change-Id: If092cf0276e74400527b740dbb4e5b92a9ab489d Signed-off-by: Patrick Farrell Reviewed-on: https://review.whamcloud.com/35324 Reviewed-by: Joseph Gmitter Tested-by: jenkins --- diff --git a/ManagingFileSystemIO.xml b/ManagingFileSystemIO.xml index 1ff1dc8..7d0f9be 100644 --- a/ManagingFileSystemIO.xml +++ b/ManagingFileSystemIO.xml @@ -382,7 +382,8 @@ client# lfs setstripe --pool|-p pool_name To set striping patterns, run: client# lfs setstripe [--size|-s stripe_size] [--offset|-o start_ost] - [--count|-c stripe_count] [--pool|-p pool_name] + [--stripe-count|-c stripe_count] [--overstripe-count|-C stripe_count] + [--pool|-p pool_name] dir|filename diff --git a/ManagingStripingFreeSpace.xml b/ManagingStripingFreeSpace.xml index 8114b22..bf703bc 100644 --- a/ManagingStripingFreeSpace.xml +++ b/ManagingStripingFreeSpace.xml @@ -89,8 +89,7 @@ In cases like these, a file can be striped over as many OSSs as it takes to achieve the required peak aggregate bandwidth for that file. Striping across a larger number of OSSs should only be used when the file size is very large and/or is accessed by many nodes - at a time. Currently, Lustre files can be striped across up to 2000 OSTs, the maximum - stripe count for an ldiskfs file system. + at a time. Currently, Lustre files can be striped across up to 2000 OSTs Improving performance when OSS bandwidth is exceeded. @@ -100,6 +99,18 @@ the I/O rate of the clients/jobs divided by the performance per OSS. + Matching stripes to I/O + pattern.When writing to a single file from multiple nodes, + having more than one client writing to a stripe can lead to issues + with lock exchange, where clients contend over writing to that stripe, + even if their I/Os do not overlap. This can be avoided if I/O can be + stripe aligned so that each stripe is accessed by only one client. + Since Lustre 2.13, the 'overstriping' feature is available, allowing more + than stripe per OST. This is particularly helpful for the case where + thread count exceeds OST count, making it possible to match stripe count + to thread count even in this case. + + Providing space for very large files. Striping is useful when a single OST does not have enough free space to hold the entire file. @@ -174,7 +185,7 @@ Setting the File Layout/Striping Configuration (lfs setstripe) Use the lfs setstripe command to create new files with a specific file layout (stripe pattern) configuration. - lfs setstripe [--size|-s stripe_size] [--count|-c stripe_count] \ + lfs setstripe [--size|-s stripe_size] [--count|-c stripe_count] [--overstripe-count|-C stripe_count] \ [--index|-i start_ost] [--pool|-p pool_name] filename|dirname stripe_size @@ -185,10 +196,15 @@ stripe_size of 0 causes the default stripe size to be used. Otherwise, the stripe_size value must be a multiple of 64 KB. - stripe_count + stripe_count (--stripe-count, --overstripe-count) - - The stripe_count indicates how many OSTs to use. The default stripe_count value is 1. Setting stripe_count to 0 causes the default stripe count to be used. Setting stripe_count to -1 means stripe over all available OSTs (full OSTs are skipped). + + The stripe_count indicates how many stripes to use. + The default stripe_count value is 1. Setting + stripe_count to 0 causes the default stripe count to be + used. Setting stripe_count to -1 means stripe over all + available OSTs (full OSTs are skipped). When --overstripe-count is used, + per OST if necessary. start_ost diff --git a/UserUtilities.xml b/UserUtilities.xml index a3eb0c3..4fce6bb 100644 --- a/UserUtilities.xml +++ b/UserUtilities.xml @@ -38,7 +38,8 @@ lfs getstripe [--obd|-O ost_name] [--quiet|-q] [--ver [--mdt-index|-M] [--recursive|-r] [--raw|-R] [--layout|-L] dirname|filename ... -lfs setstripe [--size|-s stripe_size] [--count|-c stripe_count] +lfs setstripe [--size|-s stripe_size] [--stripe-count|-c stripe_count] + [--overstripe-count|-C stripe_count] [--stripe-index|-i start_ost_index] [--ost-list|-o ost_indicies] [--pool|-p pool] @@ -536,7 +537,7 @@ lfs help - --count + --stripe-count @@ -647,7 +648,7 @@ lfs help - --count stripe_cnt + --stripe-count stripe_cnt @@ -664,6 +665,24 @@ lfs help + --overstripe-count stripe_cnt + + + + The same as --stripe-count, but allows overstriping, + which will place more than one stripe per OST if + stripe_cnt is greater than the number of + OSTs. Overstriping is useful for matching the number of stripes + to the number of processes, or with very fast OSTs, where one + stripe per OST is not enough to get full performance. + + + + +   + + + --size stripe_size The default stripe-size is 0. The default start-ost is