From 31a91486b7da74e93fc17dcc8f6eb1df72544ac0 Mon Sep 17 00:00:00 2001 From: Sergey Cheremencev Date: Wed, 29 Apr 2020 15:32:57 +0300 Subject: [PATCH] LUDOC-467 quota: add ost pool quotas Add documentation for ost pool quotas. Change-Id: I6b2facc8eb4f10b4c3d232d1bad5b369b67f096e Signed-off-by: Sergey Cheremencev Reviewed-on: https://review.whamcloud.com/38414 Tested-by: jenkins Reviewed-by: Andreas Dilger --- ConfiguringQuotas.xml | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) diff --git a/ConfiguringQuotas.xml b/ConfiguringQuotas.xml index 0a7b88f..46c744c 100644 --- a/ConfiguringQuotas.xml +++ b/ConfiguringQuotas.xml @@ -912,6 +912,109 @@ adjust_qunit 1 samples [us] 70 70 70 4293, respectively. The unit is microseconds (μs). +
+ + <indexterm> + <primary>Quotas</primary> + <secondary>pools</secondary> + </indexterm>Pool Quotas + + OST Pool Quotas feature gives an ability to limit user's (group's/project's) + disk usage at OST pool level. Each OST Pool Quota (PQ) maps directly to the + OST pool of the same name. Thus PQ could be tuned with standard + lctl pool_new/add/remove/erase commands. All PQ are subset of a + global pool that includes all OSTs and MDTs (DOM case). + It may be initially confusing to be prevented from using "all of" one quota + due to a different quota setting. In Lustre, a quota is a limit, not a right + to use an amount. You don't always get to use your quota - an OST may be out + of space, or some other quota is limiting. For example, if there is an inode + quota and a space quota, and you hit your inode limit while you still have + plenty of space, you can't use the space. For another example, quotas may + easily be over-allocated: everyone gets 10PB of quota, in a 15PB system. + That does not give them the right to use 10PB, it means they cannot use more + than 10PB. They may very well get ENOSPC long before that - but they will not + get EDQUOT. This behavior already exists in Lustre today, but pool quotas + increase the number of limits in play: user, group or project global space quota + and now all of those limits can also be defined for each pool. In all cases, + the net effect is that the actual amount of space you can use is limited to the + smallest (min) quota out of everything that is applicable. + See more details in + + OST Pool Quotas HLD + +
+ DOM and MDT pools + + From Quota Master point of view, "data" MDTs are regular members together + with OSTs. However Pool Quotas support only OSTs as there is currently + no mechanism to group MDTs in pools. + +
+
+ Lfs quota/setquota options to setup quota pools + + The same long option --pool is used to setup and report + Pool Quotas with lfs setquota and lfs setquota. + + + lfs setquota --pool pool_name + is used to set the block and soft usage limit for the user, group, or + project for the specified pool name. + + + lfs quota --pool pool_name + shows the user, group, or project usage for the specified pool name. + +
+
+ Quota pools interoperability + + Both client and server should have at least Lustre 2.14 to support Pool Quotas. + + + Pool Quotas may be able to work with older clients if server + supports Pool Quotas. Pool quotas cannot be viewed or modified by + older clients. Since the quota enforcement is done on the servers, only + a single client is needed to configure the quotas. This could be done by + mounting a client directly on the MDS if needed. + + +
+
+ Pool Quotas Hard Limit setup example + + Let's imagine you need to setup quota usage for already existed OST pool + flash_pool: + + +# it is a limit for global pool. PQ don't work properly without that +lfs setquota -u ivan -B100T /mnt/testfs +# set 1TiB block hard limit for ivan in a flash_pool +lfs setquota -u ivan --pool flash_pool -B1T /mnt/testfs + + + + System-side hard limit is required before setting Quota Pool limit. + If you do not need to limit user at all OSTs and MDTs at system, + only per pool, it is recommended to set some unrealistic big hard limit. + Without a global limit in place the Quota Pool limit will not be enforced. + No matter hard or soft global limit - at least one of them should be set. + + + +
+
+ Pool Quotas Soft Limit setup example + +# notify OSTs to enforce quota for ivan +lfs setquota -u ivan -B10T /mnt/testfs +# soft limit 10MiB for ivan in a pool flash_pool +lfs setquota -u ivan --pool flash_pool -b1T /mnt/testfs +# set block grace 600 s for all users at flash_pool +lfs setquota -t -u --block-grace 600 --pool flash_pool /mnt/testfs + +
+