LU-15873 quota: don't release for RO device There is no need to release quota space for readonly device. And further problem is there is inconsistency between Lustre osd and ldiskfs, ldiskfs won't load quota inode on Readonly mount, however Lustre osd is not aware of this and load accounting objects even in RO. this might potentially cause problems when Lustre want to access quota. Change-Id: I7db5fe3f3bed3103ed62f6beba1d6f47fce12a21 Signed-off-by: Wang Shilong <wshilong@ddn.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Li Xi <lixi@ddn.com> Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48096 Reviewed-by: Oleg Drokin <green@whamcloud.com> Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com>
LU-16415 quota: enforce project quota for root Patch adds an option to enforce project quotas for root. It is disabled by default, to enable set osd-ldiskfs.*.quota_slave.root_prj_enable to 1 at each target where you need this option. Patch also adds sanity-quota_1j to test new feature. Signed-off-by: Sergey Cheremencev <scherementsev@ddn.com> Change-Id: I978dc8442235149794f85110309f63bc560defdc Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49460 Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com> Reviewed-by: Sebastien Buisson <sbuisson@ddn.com> Reviewed-by: Oleg Drokin <green@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Shuichi Ihara <sihara@ddn.com> Tested-by: Maloo <maloo@whamcloud.com>
LU-13135 quota: improve checks in OSDs to ignore quota for root-owned files. sanity/60a: zfs before 80s, after 66s ldiskfs before 65s, after 38s ave.write declaration in sanity/60a: zfs before 3.21 usec, after 1.16 usec ldiskfs before 4.06 usec, after 0.66 usec Change-Id: Ib9ba50d260eac408f1f5e43c4d722ff5024135cf Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com> Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/37232 Reviewed-by: Oleg Drokin <green@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Wang Shilong <wangshilong1991@gmail.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com>
LU-13335 ldiskfs: add projid to debug logs There is virtually no tracking of projid changes in osd-ldiskfs, which makes it very difficult to debug operations therein. Add some minimal debugging on the client and servers to log the projid when it is changed, along with the affected FID. Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Change-Id: Ibcf3f09ee243ebe052c8f9119383897072ce7057 Reviewed-on: https://review.whamcloud.com/46369 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com> Reviewed-by: Li Dongyang <dongyangli@ddn.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-11303 quota: enforce block quota for chgrp In patch https://review.whamcloud.com/30146 "LU-5152 quota: enforce block quota for chgrp", problems were introduced due to synchronous requests from the MDS to the OSS to change the quota assignment of files during chgrp operations. However, in some cases, the OSTs are themselves out of grant and may send a quota request to the MDS, which may result in a deadlock. Another issue is the slow performance caused by the synchronous operation between MDT and OSTs. This patch drops the synchronous RPC requirement of the original patch #30146 to avoid this problem. Previously, problems in quota tracking related to chgrp were introduced due to synchronous RPCs from the MDS to the OSS when changing the group ownership of objects for quota tracking since Fixes: 8a71fd5061b ("LU-5152 quota: enforce block quota for chgrp") Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com> Change-Id: I40556b9e8a0628eb18aa806d2f6b3dfb9b53e874 Reviewed-on: https://review.whamcloud.com/33996 Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Wang Shilong <wangshilong1991@gmail.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-13539 all: Cleanup LASSERTF uses missing newlines LASSERTF() usage that does not include a terminating newline present an unclear syslog entry from admins. This adds the terminating newline for a few cases. Test-Parameters: trivial HPE-bug-id: LUS-8853 Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: I3f088601c2ad6d59e2868e86bfbd5315e36ece98 Reviewed-on: https://review.whamcloud.com/38540 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Arshad Hussain <arshad.super@gmail.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-13043 quota: remove annoying message in osd_declare_inode_qid() The admin shouldn't be getting console error messages when a user goes over quota(this would be happening continuously at some sites). In some call paths, the "*flags" parameter may be NULL, don't try to access it in that case. As a general cleanup, move the QUOTA_FL_* flags over to a named enum "enum osd_quota_local_flags" so that it is easier to see what this field actually holds, rather than a totally generic "int *flags" argument that has to be hunted through the code. Fixes: d30f9e6b6c5d ("LU-11425 quota: support quota for DoM") Change-Id: Id5686ecdb8a943e48a2888067e321f83b8569188 Signed-off-by: Wang Shilong <wshilong@ddn.com> Reviewed-on: https://review.whamcloud.com/36906 Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com> Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
LU-9679 general: avoid bare return; at end of void function Having: return; } at the end of a void function is unnecessary noise. Where it is the *only* statement in the function, it can be useful, so that remain unchanged. The rest have been removed. Test-Parameters: trivial Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: If02f6f5b91d4134cf95a68ebccc83df28c360fb2 Reviewed-on: https://review.whamcloud.com/36654 Tested-by: jenkins <devops@whamcloud.com> Reviewed-by: Shaun Tancheff <stancheff@cray.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com> Reviewed-by: Ben Evans <bevans@cray.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-10048 ofd: take local locks within transaction The patch (with companion patch LU-10048 osd: async truncate) addresses long outstanding technical debt resulting in different locking order on MDT and OST. With OUT introduction this mismatch lead to deadlocks as OUT is used by the both sides and can't easily support two different locking models simultanously. The unified locking rules are: - open transaction (dt_trans_start()), - then take a lock (dt_{read|write}_lock()). Change-Id: I1eaeb58ae3d61869914293a9fea2d0a1faefe76b Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com> Reviewed-on: https://review.whamcloud.com/31293 Reviewed-by: Mike Pershin <mpershin@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-12477 kernel: remove < 2.6.39 kernel support We no longer support kernels 2.6.38 or older so we can remove all the various autoconf test and simplify the lustre code. Only leave LC_EXPORT_TRUNCATE_COMPLETE_PAGE since 4.X kernel test are linked to this test. Change-Id: I0532507dd3310b7ef4f4e4201dfa8b47c2991163 Signed-off-by: James Simmons <uja.ornl@yahoo.com> Reviewed-on: https://review.whamcloud.com/35551 Reviewed-by: Shaun Tancheff <stancheff@cray.com> Tested-by: jenkins <devops@whamcloud.com> Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com> Reviewed-by: Yang Sheng <ys@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-11425 quota: support quota for DoM Support block quota enforcement on MDT to accommodate the data writing introduced by DoM. This patch adds a new qsd_instance on the OSD device at MDT to implement the quota enforcement on block data, for there is only one type of quota (meta data or block data) to be handled in one qsd_instance in the current quota design of Lustre, it's much more changes to adapt the qsd_instance to support more quota type than to use a separated qsd_instance to enfore meta data on MDT. Change-Id: I7c57f383d338cf0070e05ecc318d42586c39371e Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com> Reviewed-on: https://review.whamcloud.com/33257 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Wang Shilong <wshilong@ddn.com> Reviewed-by: Mike Pershin <mpershin@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-11412 kernel: kernel update [SLES12 SP3 4.4.155-94.50] Update SLES12 SP3 kernel to 4.4.155-94.50. Test-Parameters: envdefinitions=CONF_SANITY_EXCEPT=103 \ mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs \ clientdistro=sles12sp3 ossdistro=sles12sp3 mdsdistro=sles12sp3 \ testlist=sanity,conf-sanity,sanity-sec Change-Id: I47b72420664fe614a2da7c864c401a9729d96c55 Signed-off-by: Jian Yu <yujian@whamcloud.com> Reviewed-on: https://review.whamcloud.com/33236 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Yang Sheng <ys@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-11094 osd-ldiskfs: Fix style issues for osd_quota.c This patch fixes issues reported by checkpatch for file lustre/osd-ldiskfs/osd_quota.c Change-Id: I1a01c3e6327ec56a1ffcf85c5d06934a5f8e8c54 Test-Parameters: trivial Signed-off-by: Arshad Hussain <arshad.super@gmail.com> Reviewed-on: https://review.whamcloud.com/32724 Tested-by: Jenkins Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: James Simmons <uja.ornl@yahoo.com>
LU-11017 quota: ignore quota for CAP_SYS_RESOURCE properly Currently, lustre quota will ignore this type of quota if quota id is 0 or we force to ignore. For write, we have passed CAP_SYS_RESOURCE properly, but For metadata operations this is not done. Test-Parameters: testlist=sanity-quota Change-Id: Ibcdc0e53ad125042d4889ac51a9a9ead4066c0c8 Signed-off-by: Wang Shilong <wshilong@ddn.com> Reviewed-on: https://review.whamcloud.com/32378 Tested-by: Jenkins Reviewed-by: Fan Yong <fan.yong@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Yingjin Qian <qian@ddn.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-10565 osd: unify interface for vfs Some vfs changes were applied to other part but OSD. So unify them with OSD layer. Signed-off-by: Yang Sheng <yang.sheng@intel.com> Change-Id: Ia3e907964d6321571f52e4c24a46a8ab64e4d056 Reviewed-on: https://review.whamcloud.com/31646 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Bob Glossman <bob.glossman@intel.com> Reviewed-by: Wang Shilong <wshilong@ddn.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-8999 quota: fix issue of multiple call of seq start Multiple call of lprocfs_quota_seq_start could change the block orders in the lower level of the quota tree, which will cause quota entries to be skipped. This patch also fix a problem in walk_tree_dqentry, which some entries could be skipped for the "index" can be added even if a valid quota entry has been found. Change-Id: I44936c70d4060bd83db22aba0e3f665981cfa50a Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com> Reviewed-on: https://review.whamcloud.com/31721 Reviewed-by: Fan Yong <fan.yong@intel.com> Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-10308 misc: update Intel copyright messages for 2017 Update copyright messages for files updated in 2016, excluding trivial patches. Add trivial patches to updatecw.sh script exclude list. Revert some changes that were incorrectly attributed to the 2016 (d10200a80770f0029d1d665af954187b9ad883df) and 2015 (0754bc8f2623bea184111af216f7567608db35b6) copyright update patches themselves, since they were not in the exclude list when the subsequent script was run. Test-Parameters: trivial Signed-off-by: Andreas Dilger <andreas.dilger@intel.com> Change-Id: I82f21c30c4dac75792bb49fc139bee2ca51f5545 Reviewed-on: https://review.whamcloud.com/30341 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Jian Yu <jian.yu@intel.com> Reviewed-by: James Nunez <james.a.nunez@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-7991 quota: project quota against ZFS backend This patch enables the project quota feature against ZFS backend. The project quota patch for ZFS itself will be landed via https://github.com/zfsonlinux/zfs/pull/6290. Such patch may be released via zfs-0.7.x (x >= 5). This patch also cleanup the quota logic for ZFS backend and sanity-quota.sh Signed-off-by: Fan Yong <fan.yong@intel.com> Change-Id: Ic488efe19a6e34a3615811f79fb9ea7954ad9263 Reviewed-on: https://review.whamcloud.com/27093 Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Wang Shilong <wshilong@ddn.com> Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4017 quota: cleanup to improve quota codes Add man page updates for project quota, also cleanup to address some style and minor problems Change-Id: I3ee3e866dd0300a1b07e0f5319dfd695c0bafba0 Signed-off-by: Wang Shilong <wshilong@ddn.com> Reviewed-on: https://review.whamcloud.com/26577 Tested-by: Jenkins Reviewed-by: Niu Yawei <yawei.niu@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Li Xi <lixi@ddn.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4017 quota: add setting/getting project id function Extend Attr RPC to support project ID attribute, new ioctl is introduced to get/set project id, it is kept same ioctl number as VFS, you could use: lsattr -p <file> chattr -p <file> Redefine LL_MAXQUOTAS as 3, change @qsd_acct_failed into qqi, otherwise osd-zfs could not use USR/GRP quota normally. Remove osd_quota_migration(), it only output some messages in case upgrading from 1.8 which give us extra efforts for project quota, remove it indead. also fix a typo bug in osp_attr_set(). Change-Id: I4b537be9efa29fda730592b6d5e41c6b60fb6c0d Signed-off-by: Wang Shilong <wshilong@ddn.com> Reviewed-on: https://review.whamcloud.com/26202 Reviewed-by: Oleg Drokin <oleg.drokin@intel.com> Tested-by: Oleg Drokin <oleg.drokin@intel.com>