From 24ff0d8d1bc65f91ef667c3ce0fd59673eecbf44 Mon Sep 17 00:00:00 2001 From: Joseph Gmitter Date: Wed, 27 Mar 2019 10:08:07 -0400 Subject: [PATCH] LUDOC-435 quotas: Update project quotas requirements This patch adds specific notes in the project quota documentation to explicitly state that a patched server is required and that patchless servers do not support project quotas. The patch also removes the quota enabling instructions for Lustre versions older than 2.4. Signed-off-by: Joseph Gmitter Change-Id: Iaaa602012b244c225605ffe458e63634a7f9e40a Reviewed-on: https://review.whamcloud.com/34516 Tested-by: Jenkins Reviewed-by: Wang Shilong Reviewed-by: Andreas Dilger --- ConfiguringQuotas.xml | 167 +++++++++++++++++++++++--------------------------- 1 file changed, 75 insertions(+), 92 deletions(-) diff --git a/ConfiguringQuotas.xml b/ConfiguringQuotas.xml index 58fc0e3..83bb6b0 100644 --- a/ConfiguringQuotas.xml +++ b/ConfiguringQuotas.xml @@ -91,92 +91,71 @@ quite file system. lctl get_param version to identify you are currently using. -
- Enabling Disk Quotas (Lustre Software Prior to Release 2.4) - - - For Lustre software releases older than release 2.4, - lfs quotacheck must be first run from a client node to - create quota files on the Lustre targets (i.e. the MDT and OSTs). - lfs quotacheck requires the file system to be quiescent - (i.e. no modifying operations like write, truncate, create or delete - should run concurrently). Failure to follow this caution may result in - inaccurate user/group disk usage. Operations that do not change Lustre - files (such as read or mount) are okay to run. - lfs quotacheck performs a scan on all the Lustre - targets to calculates the block/inode usage for each user/group. If the - Lustre file system has many files, - quotacheck may take a long time to complete. Several - options can be passed to - lfs quotacheck: - -# lfs quotacheck -ug /mnt/testfs - - - - - u-- checks the user disk quota information - - - - g-- checks the group disk quota information - - - By default, quota is turned on after - quotacheck completes. However, this setting isn't - persistent and quota will have to be enabled again (via - lfs quotaon) if one of the Lustre targets is - restarted. - lfs quotaoff is used to turn off quota. - To enable quota permanently with a Lustre software release older - than release 2.4, the - quota_type parameter must be used. This requires - setting - mdd.quota_type and - ost.quota_type, respectively, on the MDT and OSTs. - quota_type can be set to the string - u (user), - g (group) or - ug for both users and groups. This parameter can be - specified at - mkfs time ( - mkfs.lustre --param mdd.quota_type=ug) or with - tunefs.lustre. As an example: - -tunefs.lustre --param ost.quota_type=ug $ost_dev - - When using - mkfs.lustre --param mdd.quota_type=ug or - tunefs.lustre --param ost.quota_type=ug, be sure to - run the command on all OSTs and the MDT. Otherwise, abnormal results may - occur. - - - In Lustre software releases before 2.4, when new OSTs are - added to the file system, quotas are not automatically propagated to - the new OSTs. As a workaround, clear and then reset quotas for each - user or group using the - lfs setquota command. In the example below, quotas - are cleared and reset for user - bob on file system - testfs: - -$ lfs setquota -u bob -b 0 -B 0 -i 0 -I 0 /mnt/testfs -$ lfs setquota -u bob -b 307200 -B 309200 -i 10000 -I 11000 /mnt/testfs - - -
-
+
Enabling Disk Quotas (Lustre Software Release 2.4 and later) - Quota setup is orchestrated by the MGS and all setup - commands in this section must be run on the MGS and project quotas need - lustre Relase 2.10 and later. Once setup, verification of the - quota state must be performed on the MDT. Although quota enforcement is - managed by the Lustre software, each OSD implementation relies on the - back-end file system to maintain per-user/group/project block and inode - usage. Hence, differences exist when setting up quotas with ldiskfs or - ZFS back-ends: + + Quota setup is orchestrated by the MGS and all setup + commands in this section must be run directly on the MGS. + Support for project quotas specifically requires Lustre Release 2.10 or + later. A patched server may be required, depending + on the kernel version and backend filesystem type: + + + + + + + + + Configuration + + + + + Patched Server Required? + + + + + + + + ldiskfs with kernel version < 4.5 + + Yes + + + + ldiskfs with kernel version >= 4.5 + + No + + + + zfs version >=0.8 with kernel + version < 4.5 + + Yes + + + + zfs version >=0.8 with kernel + version > 4.5 + + No + + + + + *Note: Project quotas are not supported on zfs versions earlier + than 0.8. + + Once setup, verification of the quota state must be performed on the + MDT. Although quota enforcement is managed by the Lustre software, each + OSD implementation relies on the back-end file system to maintain + per-user/group/project block and inode usage. Hence, differences exist + when setting up quotas with ldiskfs or ZFS back-ends: For ldiskfs backends, @@ -184,17 +163,18 @@ $ lfs setquota -u bob -b 307200 -B 309200 -i 10000 -I 11000 /mnt/testfs enables the QUOTA feature flag in the superblock which turns quota accounting on at mount time automatically. e2fsck was also modified to fix the quota files when the QUOTA feature flag is present. The - project quota feature is disabled by default, and + project quota feature is disabled by default, and tune2fs needs to be run to enable every target manually. For ZFS backend, the project quota feature is not - supported yet. Accounting ZAPs are created and maintained - by the ZFS file system itself. While ZFS tracks per-user and group - block usage, it does not handle inode accounting for ZFS versions - prior to zfs-0.7.0. The ZFS OSD implements its own support for inode - tracking. Two options are available: + supported on zfs versions less than 0.8.0. Accounting ZAPs + are created and maintained by the ZFS file system itself. While ZFS + tracks per-user and group block usage, it does not handle inode + accounting for ZFS versions prior to zfs-0.7.0. The ZFS OSD previously + implemented its own support for inode tracking. Two options are + available: The ZFS OSD can estimate the number of inodes in-use based @@ -314,7 +294,7 @@ lctl conf_param fsname.quota.ost|mdt $ lctl conf_param testfs3.quota.mdt=none -
+
<indexterm> <primary>Quotas</primary> @@ -634,7 +614,10 @@ $ cp: writing `/mnt/testfs/foo`: Disk quota exceeded. Lustre 2.10, <emphasis role="bold">all Lustre servers and clients must be upgraded to Lustre release 2.10 or later for project quota to work correctly</emphasis>. Otherwise, project quota will be inaccessible on - clients and not be accounted for on OSTs.</para> + clients and not be accounted for on OSTs. Furthermore, the + <emphasis role="bold">servers may be required to use a patched kernel, + </emphasis> for more information see + <xref linkend="enabling_disk_quota_after24"/>.</para> </section> <section xml:id="granted_cache_and_quota_limits"> <title> -- 1.8.3.1