<?xml version='1.0' encoding='utf-8'?>
<chapter xmlns="http://docbook.org/ns/docbook"
-xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"
-xml:id="configuringquotas">
+ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US"
+ xml:id="configuringquotas">
<title xml:id="configuringquotas.title">Configuring and Managing
Quotas</title>
<section xml:id="quota_configuring">
<literal>lctl</literal> commands (post-mount).</para>
</listitem>
<listitem>
-
<para>The quota feature in Lustre software is distributed
+ <para>The quota feature in Lustre software is distributed
throughout the system (as the Lustre file system is a distributed file
system). Because of this, quota setup and behavior on Lustre is
- different from local disk quotas in the following ways:</para>
+ somewhat different from local disk quotas in the following ways:</para>
<itemizedlist>
<listitem>
<para>No single point of administration: some commands must be
resolution.</para>
</listitem>
<listitem>
- <para>Accuracy: quota information is distributed throughout
-the file system and can only be accurately calculated with a completely
-quite file system.</para>
+ <para>Accuracy: quota information is distributed throughout the file
+ system and can only be accurately calculated with a quiescent file
+ system in order to minimize performance overhead during normal use.
+ </para>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para>Client does not set the
<literal>usrquota</literal> or
- <literal>grpquota</literal> options to mount. As of Lustre software
- release 2.4, space accounting is always enabled by default and quota
- enforcement can be enabled/disabled on a per-file system basis with
- <literal>lctl conf_param</literal>. It is worth noting that both
- <literal>lfs quotaon</literal> and
- <literal>quota_type</literal> are deprecated as of Lustre software
- release 2.4.0.</para>
+ <literal>grpquota</literal> options to mount. Space accounting is
+ enabled by default and quota enforcement can be enabled/disabled on
+ a per-filesystem basis with <literal>lctl conf_param</literal>.</para>
+ <para condition="l28">It is worth noting that the
+ <literal>lfs quotaon</literal>, <literal>lfs quotaoff</literal>,
+ <literal>lfs quotacheck</literal> and <literal>quota_type</literal>
+ sub-commands are deprecated as of Lustre 2.4.0, and removed completely
+ in Lustre 2.8.0.</para>
</listitem>
</itemizedlist>
<caution>
responsible for management and enforcement. The back-end file
system is responsible for resource usage and accounting. Because of
this, it is necessary to begin enabling quotas by enabling quotas on the
- back-end disk system. Because quota setup is dependent on the Lustre
- software version in use, you may first need to run
- <literal>lctl get_param version</literal> to identify
- <xref linkend="whichversion"/> you are currently using.
+ back-end disk system.
</para>
- <section>
- <title>Enabling Disk Quotas (Lustre Software Prior to Release 2.4)
- </title>
- <para>
- For Lustre software releases older than release 2.4,
- <literal>lfs quotacheck</literal> must be first run from a client node to
- create quota files on the Lustre targets (i.e. the MDT and OSTs).
- <literal>lfs quotacheck</literal> requires the file system to be quiescent
- (i.e. no modifying operations like write, truncate, create or delete
- should run concurrently). Failure to follow this caution may result in
- inaccurate user/group disk usage. Operations that do not change Lustre
- files (such as read or mount) are okay to run.
- <literal>lfs quotacheck</literal> performs a scan on all the Lustre
- targets to calculates the block/inode usage for each user/group. If the
- Lustre file system has many files,
- <literal>quotacheck</literal> may take a long time to complete. Several
- options can be passed to
- <literal>lfs quotacheck</literal>:</para>
- <screen>
-# lfs quotacheck -ug /mnt/testfs
-</screen>
- <itemizedlist>
- <listitem>
- <para>
- <literal>u</literal>-- checks the user disk quota information</para>
- </listitem>
- <listitem>
- <para>
- <literal>g</literal>-- checks the group disk quota information</para>
- </listitem>
- </itemizedlist>
- <para>By default, quota is turned on after
- <literal>quotacheck</literal> completes. However, this setting isn't
- persistent and quota will have to be enabled again (via
- <literal>lfs quotaon</literal>) if one of the Lustre targets is
- restarted.
- <literal>lfs quotaoff</literal> is used to turn off quota.</para>
- <para>To enable quota permanently with a Lustre software release older
- than release 2.4, the
- <literal>quota_type</literal> parameter must be used. This requires
- setting
- <literal>mdd.quota_type</literal> and
- <literal>ost.quota_type</literal>, respectively, on the MDT and OSTs.
- <literal>quota_type</literal> can be set to the string
- <literal>u</literal> (user),
- <literal>g</literal> (group) or
- <literal>ug</literal> for both users and groups. This parameter can be
- specified at
- <literal>mkfs</literal> time (
- <literal>mkfs.lustre --param mdd.quota_type=ug</literal>) or with
- <literal>tunefs.lustre</literal>. As an example:</para>
- <screen>
-tunefs.lustre --param ost.quota_type=ug $ost_dev
-</screen>
- <para>When using
- <literal>mkfs.lustre --param mdd.quota_type=ug</literal> or
- <literal>tunefs.lustre --param ost.quota_type=ug</literal>, be sure to
- run the command on all OSTs and the MDT. Otherwise, abnormal results may
- occur.</para>
- <warning>
- <para>
- In Lustre software releases before 2.4, when new OSTs are
- added to the file system, quotas are not automatically propagated to
- the new OSTs. As a workaround, clear and then reset quotas for each
- user or group using the
- <literal>lfs setquota</literal> command. In the example below, quotas
- are cleared and reset for user
- <literal>bob</literal> on file system
- <literal>testfs</literal>:
- <screen>
-$ lfs setquota -u bob -b 0 -B 0 -i 0 -I 0 /mnt/testfs
-$ lfs setquota -u bob -b 307200 -B 309200 -i 10000 -I 11000 /mnt/testfs
-</screen></para>
- </warning>
- </section>
- <section remap="h3" condition="l24">
- <title>Enabling Disk Quotas (Lustre Software Release 2.4 and
- later)</title>
- <para>Quota setup is orchestrated by the MGS and <emphasis>all setup
- commands in this section must be run on the MGS and project quotas need
- lustre Relase 2.10 and later</emphasis>. Once setup, verification of the
- quota state must be performed on the MDT. Although quota enforcement is
- managed by the Lustre software, each OSD implementation relies on the
- back-end file system to maintain per-user/group/project block and inode
- usage. Hence, differences exist when setting up quotas with ldiskfs or
- ZFS back-ends:</para>
+ <caution>
+ <para>Quota setup is orchestrated by the MGS and <emphasis>all setup
+ commands in this section must be run directly on the MGS</emphasis>.
+ Support for project quotas specifically requires Lustre Release 2.10 or
+ later. A <emphasis>patched server</emphasis> may be required, depending
+ on the kernel version and backend filesystem type:</para>
+ <informaltable frame="all">
+ <tgroup cols="2">
+ <colspec colname="c1" colwidth="50*" />
+ <colspec colname="c2" colwidth="50*" align="center" />
+ <thead>
+ <row>
+ <entry>
+ <para>
+ <emphasis role="bold">Configuration</emphasis>
+ </para>
+ </entry>
+ <entry>
+ <para>
+ <emphasis role="bold">Patched Server Required?</emphasis>
+ </para>
+ </entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><para>
+ <emphasis>ldiskfs with kernel version < 4.5</emphasis>
+ </para></entry>
+ <entry><para>Yes</para></entry>
+ </row>
+ <row>
+ <entry><para>
+ <emphasis>ldiskfs with kernel version >= 4.5</emphasis>
+ </para></entry>
+ <entry><para>No</para></entry>
+ </row>
+ <row>
+ <entry><para>
+ <emphasis>zfs version >=0.8 with kernel
+ version < 4.5</emphasis>
+ </para></entry>
+ <entry><para>Yes</para></entry>
+ </row>
+ <row>
+ <entry><para>
+ <emphasis>zfs version >=0.8 with kernel
+ version > 4.5</emphasis>
+ </para></entry>
+ <entry><para>No</para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </informaltable>
+ <para>*Note: Project quotas are not supported on zfs versions earlier
+ than 0.8.</para>
+ </caution>
+ <para>Once setup, verification of the quota state must be performed on the
+ MDT. Although quota enforcement is managed by the Lustre software, each
+ OSD implementation relies on the back-end file system to maintain
+ per-user/group/project block and inode usage. Hence, differences exist
+ when setting up quotas with ldiskfs or ZFS back-ends:</para>
<itemizedlist>
<listitem>
<para>For ldiskfs backends,
enables the QUOTA feature flag in the superblock which turns quota
accounting on at mount time automatically. e2fsck was also modified
to fix the quota files when the QUOTA feature flag is present. The
- project quota feature is disabled by default, and
+ project quota feature is disabled by default, and
<literal>tune2fs</literal> needs to be run to enable every target
- manually.</para>
+ manually. If user, group, and project quota usage is inconsistent,
+ run <literal>e2fsck -f</literal> on all unmounted MDTs and OSTs.
+ </para>
</listitem>
<listitem>
<para>For ZFS backend, <emphasis>the project quota feature is not
- supported yet.</emphasis> Accounting ZAPs are created and maintained
- by the ZFS file system itself. While ZFS tracks per-user and group
- block usage, it does not handle inode accounting for ZFS versions
- prior to zfs-0.7.0. The ZFS OSD implements its own support for inode
- tracking. Two options are available:</para>
+ supported on zfs versions less than 0.8.0.</emphasis> Accounting ZAPs
+ are created and maintained by the ZFS file system itself. While ZFS
+ tracks per-user and group block usage, it does not handle inode
+ accounting for ZFS versions prior to zfs-0.7.0. The ZFS OSD previously
+ implemented its own support for inode tracking. Two options are
+ available:</para>
<orderedlist>
<listitem>
<para>The ZFS OSD can estimate the number of inodes in-use based
</listitem>
</itemizedlist>
<note>
- <para>Lustre file systems formatted with a Lustre release prior to 2.4.0
- can be still safely upgraded to release 2.4.0, but will not have
- functional space usage report until
- <literal>tunefs.lustre --quota</literal> is run against all targets. This
- command sets the QUOTA feature flag in the superblock and runs e2fsck (as
- a result, the target must be offline) to build the per-UID/GID disk usage
- database.</para>
+ <para>To (re-)enable space usage quota on ldiskfs filesystems, run
+ <literal>tune2fs -O quota</literal> against all targets. This command
+ sets the QUOTA feature flag in the superblock and runs e2fsck internally.
+ As a result, the target must be offline to build the per-UID/GID disk
+ usage database.</para>
<para condition="l2A">Lustre filesystems formatted with a Lustre release
prior to 2.10 can be still safely upgraded to release 2.10, but will not
- have project quota usage reporting functional until
+ have project quota usage reporting functional until 2.15.0 or
<literal>tune2fs -O project</literal> is run against all ldiskfs backend
targets. This command sets the PROJECT feature flag in the superblock and
runs e2fsck (as a result, the target must be offline). See
considerations.</para>
</note>
<caution>
- <para>Lustre software release 2.4 and later requires a version of
- e2fsprogs that supports quota (i.e. newer or equal to 1.42.13.wc5,
- 1.42.13.wc6 or newer is needed for project quota support) to be
- installed on the server nodes using ldiskfs backend (e2fsprogs is not
- needed with ZFS backend). In general, we recommend to use the latest
- e2fsprogs version available on
- <link xl:href="http://downloads.whamcloud.com/e2fsprogs/">
- http://downloads.whamcloud.com/public/e2fsprogs/</link>.</para>
+ <para>Lustre requires a version of e2fsprogs that supports quota
+ to be installed on the server nodes when using the ldiskfs backend
+ (e2fsprogs is not needed with ZFS backend). In general, we recommend
+ to use the latest e2fsprogs version available on
+ <link xl:href="https://downloads.whamcloud.com/public/e2fsprogs/">
+ https://downloads.whamcloud.com/public/e2fsprogs/</link>.</para>
<para>The ldiskfs OSD relies on the standard Linux quota to maintain
accounting information on disk. As a consequence, the Linux kernel
running on the Lustre servers using ldiskfs backend must have
<literal>CONFIG_QUOTACTL</literal> and
<literal>CONFIG_QFMT_V2</literal> enabled.</para>
</caution>
- <para>As of Lustre software release 2.4.0, quota enforcement is thus
- turned on/off independently of space accounting which is always enabled.
- <literal>lfs quota
- <replaceable>on|off</replaceable></literal> as well as the per-target
- <literal>quota_type</literal> parameter are deprecated in favor of a
- single per-file system quota parameter controlling inode/block quota
- enforcement. Like all permanent parameters, this quota parameter can be
- set via
- <literal>lctl conf_param</literal> on the MGS via the following
- syntax:</para>
+ <para>Quota enforcement is turned on/off independently of space
+ accounting which is always enabled. There is a single per-file
+ system quota parameter controlling inode/block quota enforcement.
+ Like all permanent parameters, this quota parameter can be set via
+ <literal>lctl conf_param</literal> on the MGS via the command:</para>
<screen>
lctl conf_param <replaceable>fsname</replaceable>.quota.<replaceable>ost|mdt</replaceable>=<replaceable>u|g|p|ugp|none</replaceable>
</screen>
<para>To turn on user, group, and project quotas for block only on
file system
<literal>testfs1</literal>, <emphasis>on the MGS</emphasis> run:</para>
- <screen>$ lctl conf_param testfs1.quota.ost=ugp
-</screen>
+ <screen>mgs# lctl conf_param testfs1.quota.ost=ugp </screen>
<para>To turn on group quotas for inodes on file system
<literal>testfs2</literal>, on the MGS run:</para>
- <screen>$ lctl conf_param testfs2.quota.mdt=g
-</screen>
+ <screen>mgs# lctl conf_param testfs2.quota.mdt=g </screen>
<para>To turn off user, group, and project quotas for both inode and block
on file system
<literal>testfs3</literal>, on the MGS run:</para>
- <screen>$ lctl conf_param testfs3.quota.ost=none
-</screen>
- <screen>$ lctl conf_param testfs3.quota.mdt=none
-</screen>
- <section>
- <title>
- <indexterm>
- <primary>Quotas</primary>
- <secondary>verifying</secondary>
- </indexterm>Quota Verification</title>
- <para>Once the quota parameters have been configured, all targets
+ <screen>mgs# lctl conf_param testfs3.quota.ost=none</screen>
+ <screen>mgs# lctl conf_param testfs3.quota.mdt=none</screen>
+ <section xml:id="quota_verification">
+ <title>
+ <indexterm>
+ <primary>Quotas</primary>
+ <secondary>verifying</secondary>
+ </indexterm>Quota Verification</title>
+ <para>Once the quota parameters have been configured, all targets
which are part of the file system will be automatically notified of the
new quota settings and enable/disable quota enforcement as needed. The
per-target enforcement status can still be verified by running the
group uptodate: glb[1],slv[1],reint[0]
</screen>
</section>
- </section>
</section>
<section xml:id="quota_administration">
<title>
<primary>Quotas</primary>
<secondary>creating</secondary>
</indexterm>Quota Administration</title>
-
<para>Once the file system is up and running, quota limits on blocks
+ <para>Once the file system is up and running, quota limits on blocks
and inodes can be set for user, group, and project. This is <emphasis>
controlled entirely from a client</emphasis> via three quota
parameters:</para>
<para>To limit quota usage for a specific project ID on a specific
directory ("<literal>/mnt/testfs/dir</literal>" in this example), run:</para>
<screen>
-$ chattr +P /mnt/testfs/dir
-$ chattr -p 1 /mnt/testfs/dir
+$ lfs project -s -p 1 -r /mnt/testfs/dir
$ lfs setquota -p 1 -b 307200 -B 309200 -i 10000 -I 11000 /mnt/testfs
</screen>
+ <para> Recursively list all descendants'(of the directory) project attribute on
+ directory ("<literal>/mnt/testfs/dir</literal>" in this example), run:</para>
+ <screen>
+$ lfs project -r /mnt/testfs/dir
+</screen>
<para>Please note that if it is desired to have
<literal>lfs quota -p</literal> show the space/inode usage under the
directory properly (much faster than <literal>du</literal>), then the
<screen>
lctl get_param osd-*.*.quota_slave.limit*
</screen></para>
- <note>
- <para>Prior to 2.4, global quota limits used to be stored in
- administrative quota files using the on-disk format of the linux quota
- file. When upgrading MDT0000 to 2.4, those administrative quota files are
- converted into IAM indexes automatically, conserving existing quota
- limits previously set by the administrator.</para>
- </note>
+ </section>
+ <section condition='l2C' xml:id="default_quota">
+ <title>
+ <indexterm>
+ <primary>Quotas</primary>
+ <secondary>default</secondary>
+ </indexterm>Default Quota</title>
+ <para>The default quota is used to enforce the quota limits for any user,
+ group, or project that do not have quotas set by administrator.</para>
+ <para>The default quota can be disabled by setting limits to
+ <literal>0</literal>.</para>
+ <section xml:id="defalut_quota_usage">
+ <title>
+ <indexterm>
+ <primary>Quotas</primary>
+ <secondary>usage</secondary>
+ </indexterm>Usage</title>
+ <screen>
+lfs quota [-U|--default-usr|-G|--default-grp|-P|--default-prj] <replaceable>/mount_point</replaceable>
+lfs setquota {-U|--default-usr|-G|--default-grp|-P|--default-prj} [-b <replaceable>block-softlimit</replaceable>] \
+ [-B <replaceable>block_hardlimit</replaceable>] [-i <replaceable>inode_softlimit</replaceable>] [-I <replaceable>inode_hardlimit</replaceable>] <replaceable>/mount_point</replaceable>
+lfs setquota {-u|-g|-p} <replaceable>username|groupname</replaceable> -d <replaceable>/mount_point</replaceable>
+ </screen>
+ <para>To set the default user quota:</para>
+ <screen>
+# lfs setquota -U -b 10G -B 11G -i 100K -I 105K /mnt/testfs
+ </screen>
+ <para>To set the default group quota:</para>
+ <screen>
+# lfs setquota -G -b 10G -B 11G -i 100K -I 105K /mnt/testfs
+ </screen>
+ <para>To set the default project quota:</para>
+ <screen>
+# lfs setquota -P -b 10G -B 11G -i 100K -I 105K /mnt/testfs
+ </screen>
+ <para>To disable the default user quota:</para>
+ <screen>
+# lfs setquota -U -b 0 -B 0 -i 0 -I 0 /mnt/testfs
+ </screen>
+ <para>To disable the default group quota:</para>
+ <screen>
+# lfs setquota -G -b 0 -B 0 -i 0 -I 0 /mnt/testfs
+ </screen>
+ <para>To disable the default project quota:</para>
+ <screen>
+# lfs setquota -P -b 0 -B 0 -i 0 -I 0 /mnt/testfs
+ </screen>
+ <para>To set user 'bob' to use the default user quota:</para>
+ <screen>
+# lfs setquota -u bob -d /mnt/testfs
+ </screen>
+ <para>To set group 'bob' to use the default group quota:</para>
+ <screen>
+# lfs setquota -g bob -d /mnt/testfs
+ </screen>
+ <para>To set project 1000 to use the default project quota:</para>
+ <screen>
+# lfs setquota -p 1000 -d /mnt/testfs
+ </screen>
+ <note>
+ <para>
+ If quota limits are set for some user, group or project, it will use
+ those specific quota limits instead of the default quota. Quota limits for
+ any user, group or project will use the default quota by setting its quota
+ limits with option '-d'.
+ </para>
+ </note>
+ </section>
</section>
<section xml:id="quota_allocation">
<title>
(aka QSD) which connects to the QMT to allocate/release quota space. The
QSD is setup directly from the OSD layer.</para>
<para>To reduce quota requests, quota space is initially allocated to QSDs
- in very large chunks. How much unused quota space can be hold by a target
+ in very large chunks. How much unused quota space can be held by a target
is controlled by the qunit size. When quota space for a given ID is close
to exhaustion on the QMT, the qunit size is reduced and QSDs are notified
of the new qunit size value via a glimpse callback. Slaves are then
<literal>lctl get_param *.*.filestotal</literal>). For more information on
using the
<literal>lfs df -i</literal> command and the command output, see
- <xref linkend="dbdoclet.checking_free_space" />.</para>
+ <xref linkend="file_striping.checking_free_space" />.</para>
<para>Unfortunately, the
<literal>statfs</literal> interface does not report the free inode count
directly, but instead reports the total inode and used inode counts. The
<primary>Quotas</primary>
<secondary>Interoperability</secondary>
</indexterm>Quotas and Version Interoperability</title>
- <para>The new quota protocol introduced in Lustre software release 2.4.0
- <emphasis role="bold">is not compatible</emphasis> with previous
- versions. As a consequence,
- <emphasis role="bold">all Lustre servers must be upgraded to release 2.4.0
- for quota to be functional</emphasis>. Quota limits set on the Lustre file
- system prior to the upgrade will be automatically migrated to the new quota
- index format. As for accounting information with ldiskfs backend, they will
- be regenerated by running
- <literal>tunefs.lustre --quota</literal> against all targets. It is worth
- noting that running
- <literal>tunefs.lustre --quota</literal> is
- <emphasis role="bold">mandatory</emphasis> for all targets formatted with a
- Lustre software release older than release 2.4.0, otherwise quota
- enforcement as well as accounting won't be functional.</para>
- <para>Besides, the quota protocol in release 2.4 takes for granted that the
- Lustre client supports the
- <literal>OBD_CONNECT_EINPROGRESS</literal> connect flag. Clients supporting
- this flag will retry indefinitely when the server returns
- <literal>EINPROGRESS</literal> in a reply. Here is the list of Lustre client
- version which are compatible with release 2.4:</para>
- <itemizedlist>
- <listitem>
- <para>Release 2.3-based clients and later</para>
- </listitem>
- <listitem>
- <para>Release 1.8 clients newer or equal to release 1.8.9-wc1</para>
- </listitem>
- <listitem>
- <para>Release 2.1 clients newer or equal to release 2.1.4</para>
- </listitem>
- </itemizedlist>
<para condition="l2A">To use the project quota functionality introduced in
Lustre 2.10, <emphasis role="bold">all Lustre servers and clients must be
upgraded to Lustre release 2.10 or later for project quota to work
correctly</emphasis>. Otherwise, project quota will be inaccessible on
- clients and not be accounted for on OSTs.</para>
+ clients and not be accounted for on OSTs. Furthermore, the
+ <emphasis role="bold">servers may be required to use a patched kernel,
+ </emphasis> for more information see
+ <xref linkend="enabling_disk_quotas"/>.</para>
+ <para condition="l2E"><literal>df</literal> and <literal>lfs df</literal>
+ will return the amount of space available to that project rather than the
+ total filesystem space, if the project quota limit is smaller.
+ <emphasis role="bold"> Only client need be upgraded to Lustre
+ release 2.14 or later to apply this new behavior</emphasis>.</para>
</section>
<section xml:id="granted_cache_and_quota_limits">
<title>
4293, respectively. The unit is microseconds (μs).</para>
</section>
</section>
+ <section xml:id="quota_pools" condition='l2E'>
+ <title>
+ <indexterm>
+ <primary>Quotas</primary>
+ <secondary>pools</secondary>
+ </indexterm>Pool Quotas</title>
+ <para>
+ OST Pool Quotas feature gives an ability to limit user's (group's/project's)
+ disk usage at OST pool level. Each OST Pool Quota (PQ) maps directly to the
+ OST pool of the same name. Thus PQ could be tuned with standard <literal>
+ lctl pool_new/add/remove/erase</literal> commands. All PQ are subset of a
+ global pool that includes all OSTs and MDTs (DOM case).
+ It may be initially confusing to be prevented from using "all of" one quota
+ due to a different quota setting. In Lustre, a quota is a limit, not a right
+ to use an amount. You don't always get to use your quota - an OST may be out
+ of space, or some other quota is limiting. For example, if there is an inode
+ quota and a space quota, and you hit your inode limit while you still have
+ plenty of space, you can't use the space. For another example, quotas may
+ easily be over-allocated: everyone gets 10PB of quota, in a 15PB system.
+ That does not give them the right to use 10PB, it means they cannot use more
+ than 10PB. They may very well get ENOSPC long before that - but they will not
+ get EDQUOT. This behavior already exists in Lustre today, but pool quotas
+ increase the number of limits in play: user, group or project global space quota
+ and now all of those limits can also be defined for each pool. In all cases,
+ the net effect is that the actual amount of space you can use is limited to the
+ smallest (min) quota out of everything that is applicable.
+ See more details in
+ <link xl:href="http://wiki.lustre.org/OST_Pool_Quotas_HLD">
+ OST Pool Quotas HLD</link>
+ </para>
+ <section remap="h3">
+ <title>DOM and MDT pools</title>
+ <para>
+ From Quota Master point of view, "data" MDTs are regular members together
+ with OSTs. However Pool Quotas support only OSTs as there is currently
+ no mechanism to group MDTs in pools.
+ </para>
+ </section>
+ <section remap="h3">
+ <title>Lfs quota/setquota options to setup quota pools</title>
+ <para>
+ The same long option <literal>--pool</literal> is used to setup and report
+ Pool Quotas with <literal>lfs setquota</literal> and <literal>lfs setquota</literal>.
+ </para>
+ <para>
+ <literal>lfs setquota --pool <replaceable>pool_name</replaceable></literal>
+ is used to set the block and soft usage limit for the user, group, or
+ project for the specified pool name.
+ </para>
+ <para>
+ <literal>lfs quota --pool <replaceable>pool_name</replaceable></literal>
+ shows the user, group, or project usage for the specified pool name.
+ </para>
+ </section>
+ <section remap="h3">
+ <title>Quota pools interoperability</title>
+ <para>
+ Both client and server should have at least Lustre 2.14 to support Pool Quotas.
+ </para>
+ <note>
+ <para>Pool Quotas may be able to work with older clients if server
+ supports Pool Quotas. Pool quotas cannot be viewed or modified by
+ older clients. Since the quota enforcement is done on the servers, only
+ a single client is needed to configure the quotas. This could be done by
+ mounting a client directly on the MDS if needed.
+ </para>
+ </note>
+ </section>
+ <section remap="h3">
+ <title>Pool Quotas Hard Limit setup example</title>
+ <para>
+ Let's imagine you need to setup quota usage for already existed OST pool
+ <literal>flash_pool</literal>:
+ </para>
+ <screen>
+# it is a limit for global pool. PQ don't work properly without that
+lfs setquota -u <replaceable>ivan</replaceable> -B<replaceable>100T /mnt/testfs</replaceable>
+# set 1TiB block hard limit for ivan in a flash_pool
+lfs setquota -u <replaceable>ivan</replaceable> --pool <replaceable>flash_pool</replaceable> -B<replaceable>1T /mnt/testfs</replaceable>
+ </screen>
+ <para>
+ <note>
+ <para>System-side hard limit is required before setting Quota Pool limit.
+ If you do not need to limit user at all OSTs and MDTs at system,
+ only per pool, it is recommended to set some unrealistic big hard limit.
+ Without a global limit in place the Quota Pool limit will not be enforced.
+ No matter hard or soft global limit - at least one of them should be set.
+ </para>
+ </note>
+ </para>
+ </section>
+ <section remap="h3">
+ <title>Pool Quotas Soft Limit setup example</title>
+ <screen>
+# notify OSTs to enforce quota for ivan
+lfs setquota -u <replaceable>ivan</replaceable> -B<replaceable>10T /mnt/testfs</replaceable>
+# soft limit 10MiB for ivan in a pool flash_pool
+lfs setquota -u <replaceable>ivan</replaceable> --pool <replaceable>flash_pool</replaceable> -b<replaceable>1T /mnt/testfs</replaceable>
+# set block grace 600 s for all users at flash_pool
+lfs setquota -t -u --block-grace <replaceable>600</replaceable> --pool <replaceable>flash_pool /mnt/testfs</replaceable>
+ </screen>
+ </section>
+ </section>
</chapter>
+<!--
+ vim:expandtab:shiftwidth=2:tabstop=8:
+ -->
git://git.whamcloud.com / doc / manual.git / blobdiff