<literal>grpquota</literal> options to mount. Space accounting is
enabled by default and quota enforcement can be enabled/disabled on
a per-filesystem basis with <literal>lctl conf_param</literal>.</para>
+ <para condition="l28">It is worth noting that the
+ <literal>lfs quotaon</literal>, <literal>lfs quotaoff</literal>,
+ <literal>lfs quotacheck</literal> and <literal>quota_type</literal>
+ sub-commands are deprecated as of Lustre 2.4.0, and removed completely
+ in Lustre 2.8.0.</para>
</listitem>
</itemizedlist>
<caution>
to fix the quota files when the QUOTA feature flag is present. The
project quota feature is disabled by default, and
<literal>tune2fs</literal> needs to be run to enable every target
- manually.</para>
+ manually. If user, group, and project quota usage is inconsistent,
+ run <literal>e2fsck -f</literal> on all unmounted MDTs and OSTs.
+ </para>
</listitem>
<listitem>
<para>For ZFS backend, <emphasis>the project quota feature is not
</listitem>
</itemizedlist>
<note>
+ <para>To (re-)enable space usage quota on ldiskfs filesystems, run
+ <literal>tune2fs -O quota</literal> against all targets. This command
+ sets the QUOTA feature flag in the superblock and runs e2fsck internally.
+ As a result, the target must be offline to build the per-UID/GID disk
+ usage database.</para>
<para condition="l2A">Lustre filesystems formatted with a Lustre release
prior to 2.10 can be still safely upgraded to release 2.10, but will not
- have project quota usage reporting functional until
+ have project quota usage reporting functional until 2.15.0 or
<literal>tune2fs -O project</literal> is run against all ldiskfs backend
targets. This command sets the PROJECT feature flag in the superblock and
runs e2fsck (as a result, the target must be offline). See
to be installed on the server nodes when using the ldiskfs backend
(e2fsprogs is not needed with ZFS backend). In general, we recommend
to use the latest e2fsprogs version available on
- <link xl:href="http://downloads.whamcloud.com/e2fsprogs/">
- http://downloads.whamcloud.com/public/e2fsprogs/</link>.</para>
+ <link xl:href="https://downloads.whamcloud.com/public/e2fsprogs/">
+ https://downloads.whamcloud.com/public/e2fsprogs/</link>.</para>
<para>The ldiskfs OSD relies on the standard Linux quota to maintain
accounting information on disk. As a consequence, the Linux kernel
running on the Lustre servers using ldiskfs backend must have
<para>To limit quota usage for a specific project ID on a specific
directory ("<literal>/mnt/testfs/dir</literal>" in this example), run:</para>
<screen>
-$ chattr +P /mnt/testfs/dir
-$ chattr -p 1 /mnt/testfs/dir
+$ lfs project -s -p 1 -r /mnt/testfs/dir
$ lfs setquota -p 1 -b 307200 -B 309200 -i 10000 -I 11000 /mnt/testfs
</screen>
+ <para> Recursively list all descendants'(of the directory) project attribute on
+ directory ("<literal>/mnt/testfs/dir</literal>" in this example), run:</para>
+ <screen>
+$ lfs project -r /mnt/testfs/dir
+</screen>
<para>Please note that if it is desired to have
<literal>lfs quota -p</literal> show the space/inode usage under the
directory properly (much faster than <literal>du</literal>), then the
<screen>
# lfs setquota -P -b 0 -B 0 -i 0 -I 0 /mnt/testfs
</screen>
+ <para>To set user 'bob' to use the default user quota:</para>
+ <screen>
+# lfs setquota -u bob -d /mnt/testfs
+ </screen>
+ <para>To set group 'bob' to use the default group quota:</para>
+ <screen>
+# lfs setquota -g bob -d /mnt/testfs
+ </screen>
+ <para>To set project 1000 to use the default project quota:</para>
+ <screen>
+# lfs setquota -p 1000 -d /mnt/testfs
+ </screen>
<note>
<para>
If quota limits are set for some user, group or project, it will use
those specific quota limits instead of the default quota. Quota limits for
any user, group or project will use the default quota by setting its quota
- limits to <literal>0</literal>.
+ limits with option '-d'.
</para>
</note>
</section>
<literal>lctl get_param *.*.filestotal</literal>). For more information on
using the
<literal>lfs df -i</literal> command and the command output, see
- <xref linkend="dbdoclet.checking_free_space" />.</para>
+ <xref linkend="file_striping.checking_free_space" />.</para>
<para>Unfortunately, the
<literal>statfs</literal> interface does not report the free inode count
directly, but instead reports the total inode and used inode counts. The
<emphasis role="bold">servers may be required to use a patched kernel,
</emphasis> for more information see
<xref linkend="enabling_disk_quotas"/>.</para>
+ <para condition="l2E"><literal>df</literal> and <literal>lfs df</literal>
+ will return the amount of space available to that project rather than the
+ total filesystem space, if the project quota limit is smaller.
+ <emphasis role="bold"> Only client need be upgraded to Lustre
+ release 2.14 or later to apply this new behavior</emphasis>.</para>
</section>
<section xml:id="granted_cache_and_quota_limits">
<title>
4293, respectively. The unit is microseconds (μs).</para>
</section>
</section>
+ <section xml:id="quota_pools" condition='l2E'>
+ <title>
+ <indexterm>
+ <primary>Quotas</primary>
+ <secondary>pools</secondary>
+ </indexterm>Pool Quotas</title>
+ <para>
+ OST Pool Quotas feature gives an ability to limit user's (group's/project's)
+ disk usage at OST pool level. Each OST Pool Quota (PQ) maps directly to the
+ OST pool of the same name. Thus PQ could be tuned with standard <literal>
+ lctl pool_new/add/remove/erase</literal> commands. All PQ are subset of a
+ global pool that includes all OSTs and MDTs (DOM case).
+ It may be initially confusing to be prevented from using "all of" one quota
+ due to a different quota setting. In Lustre, a quota is a limit, not a right
+ to use an amount. You don't always get to use your quota - an OST may be out
+ of space, or some other quota is limiting. For example, if there is an inode
+ quota and a space quota, and you hit your inode limit while you still have
+ plenty of space, you can't use the space. For another example, quotas may
+ easily be over-allocated: everyone gets 10PB of quota, in a 15PB system.
+ That does not give them the right to use 10PB, it means they cannot use more
+ than 10PB. They may very well get ENOSPC long before that - but they will not
+ get EDQUOT. This behavior already exists in Lustre today, but pool quotas
+ increase the number of limits in play: user, group or project global space quota
+ and now all of those limits can also be defined for each pool. In all cases,
+ the net effect is that the actual amount of space you can use is limited to the
+ smallest (min) quota out of everything that is applicable.
+ See more details in
+ <link xl:href="http://wiki.lustre.org/OST_Pool_Quotas_HLD">
+ OST Pool Quotas HLD</link>
+ </para>
+ <section remap="h3">
+ <title>DOM and MDT pools</title>
+ <para>
+ From Quota Master point of view, "data" MDTs are regular members together
+ with OSTs. However Pool Quotas support only OSTs as there is currently
+ no mechanism to group MDTs in pools.
+ </para>
+ </section>
+ <section remap="h3">
+ <title>Lfs quota/setquota options to setup quota pools</title>
+ <para>
+ The same long option <literal>--pool</literal> is used to setup and report
+ Pool Quotas with <literal>lfs setquota</literal> and <literal>lfs setquota</literal>.
+ </para>
+ <para>
+ <literal>lfs setquota --pool <replaceable>pool_name</replaceable></literal>
+ is used to set the block and soft usage limit for the user, group, or
+ project for the specified pool name.
+ </para>
+ <para>
+ <literal>lfs quota --pool <replaceable>pool_name</replaceable></literal>
+ shows the user, group, or project usage for the specified pool name.
+ </para>
+ </section>
+ <section remap="h3">
+ <title>Quota pools interoperability</title>
+ <para>
+ Both client and server should have at least Lustre 2.14 to support Pool Quotas.
+ </para>
+ <note>
+ <para>Pool Quotas may be able to work with older clients if server
+ supports Pool Quotas. Pool quotas cannot be viewed or modified by
+ older clients. Since the quota enforcement is done on the servers, only
+ a single client is needed to configure the quotas. This could be done by
+ mounting a client directly on the MDS if needed.
+ </para>
+ </note>
+ </section>
+ <section remap="h3">
+ <title>Pool Quotas Hard Limit setup example</title>
+ <para>
+ Let's imagine you need to setup quota usage for already existed OST pool
+ <literal>flash_pool</literal>:
+ </para>
+ <screen>
+# it is a limit for global pool. PQ don't work properly without that
+lfs setquota -u <replaceable>ivan</replaceable> -B<replaceable>100T /mnt/testfs</replaceable>
+# set 1TiB block hard limit for ivan in a flash_pool
+lfs setquota -u <replaceable>ivan</replaceable> --pool <replaceable>flash_pool</replaceable> -B<replaceable>1T /mnt/testfs</replaceable>
+ </screen>
+ <para>
+ <note>
+ <para>System-side hard limit is required before setting Quota Pool limit.
+ If you do not need to limit user at all OSTs and MDTs at system,
+ only per pool, it is recommended to set some unrealistic big hard limit.
+ Without a global limit in place the Quota Pool limit will not be enforced.
+ No matter hard or soft global limit - at least one of them should be set.
+ </para>
+ </note>
+ </para>
+ </section>
+ <section remap="h3">
+ <title>Pool Quotas Soft Limit setup example</title>
+ <screen>
+# notify OSTs to enforce quota for ivan
+lfs setquota -u <replaceable>ivan</replaceable> -B<replaceable>10T /mnt/testfs</replaceable>
+# soft limit 10MiB for ivan in a pool flash_pool
+lfs setquota -u <replaceable>ivan</replaceable> --pool <replaceable>flash_pool</replaceable> -b<replaceable>1T /mnt/testfs</replaceable>
+# set block grace 600 s for all users at flash_pool
+lfs setquota -t -u --block-grace <replaceable>600</replaceable> --pool <replaceable>flash_pool /mnt/testfs</replaceable>
+ </screen>
+ </section>
+ </section>
</chapter>
-<!--vim:expandtab:shiftwidth=2:tabstop=8:-->
+<!--
+ vim:expandtab:shiftwidth=2:tabstop=8:
+ -->
git://git.whamcloud.com / doc / manual.git / blobdiff