git://git.whamcloud.com - fs/lustre-release.git/log

LU-10220 mdd: fix buf alloc in mdd_changelog_data_store_by_fid

Fix allocation of mti_big_buf by call to lu_buf_check_and_alloc()
in mdd_changelog_data_store_by_fid().
reclen must take the header size of struct llog_changelog_rec into
account.

Maybe no memory corruptions were seen before because the buffer size
allocated in a previous call to mdd_declare_changelog_store() was
covering the need. But audit will add more information in changelog
records, provoking memory corruptions without this fix.

Lustre-change: https://review.whamcloud.com/30014
Lustre-commit: f173f93032c62bd1f95330f331d3d93ce5a31598

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Id0a06c412b54c0ae12c15d53f3e166e3e5d9ed68
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30356
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-8497 osp: handle remote -ENOMEM from osp_send_update_req()

In osp_send_update_req() detect an unsent request by checking
rq_queued_time == 0 rather than rq_set == NULL, which is always true
after returning from ptlrpc_queue_wait().

Lustre-change: https://review.whamcloud.com/30083
Lustre-commit: cef8983c8b5bf51b58df23a779769cc4b8ca8db5

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ief959b71600157a9c3521775cc06994326e50c51
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30354
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9941 lov: lsm_is_composite isn't right

LOVEA magic containing LOV_MAGIC_MAGIC will also be regarded as
a composite magic.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I3ef37ee80364b2a8f27831e3c53fb88b464f2039
Reviewed-on: https://review.whamcloud.com/28845
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit 385518f67eb917335de0c1d727941094f4ab5874)
Reviewed-on: https://review.whamcloud.com/30342
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>

New tag 2.10.2-RC1.

First release candidate for 2.10.2 release.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I207f36c4e6815dfb06ba7b9622e939ade167eb89

LU-10234 tests: use wait correctly in sanity-hsm test_3[4-6]

Test 34, 35 and 36 did not properly check the return code of wait
and potentially missed regressions.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I21e8b16b55b946a8e5837549c3a3698379132e48
Reviewed-on: https://review.whamcloud.com/30102
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dominique Martinet <dominique.martinet@cea.fr>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit eec4b8985f5eaacd13f0e0db7328dcdd610655bc)
Reviewed-on: https://review.whamcloud.com/30208
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10086 libcfs: use dynamic minors for /dev/{lnet,obd}

Request dynamic minor allocation when registering /dev/lnet and
/dev/obd. Remove the obsolete create-device-if-not-found code from
register_ioc_dev().

Lustre-change: https://review.whamcloud.com/29741
Lustre-commit: e446c1667dc9136e3a1cb3603c3851975b68e0ac

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I59c70912b4729f58a76dc6107b3e1d7379c6d7a3
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29945
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-10241 kernel: kernel update RHEL6.9 [2.6.32-696.16.1.el6]

Update RHEL6.9 kernel to 2.6.32-696.16.1.el6

Test-Parameters: clientdistro=el6.9 mdsdistro=el6.9 \
ossdistro=el6.9 mdtfilesystemtype=ldiskfs \
ostfilesystemtype=ldiskfs testgroup=review-ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ib2215b56e647fb8e19bc8b7e344a94564c7a8c7f
Reviewed-on: https://review.whamcloud.com/30131
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10166 mdc: invalid free in changelog reader

Use kthread_stop() to instruct the producer thread to exit
when the device is closed, and only then: release the CRS
structure.

The previous implementation left small time windows open,
during which the producer threads could free a structure
before the consumer thread was completely done with it.

Change-Id: Id2038aa9b7fcfd2c3347f628e749f9d2c265ac6e
Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-on: https://review.whamcloud.com/29818
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit 89e52326b5bd3d2716ce6ec5d9f9d787947d91a1)
Reviewed-on: https://review.whamcloud.com/30207

LU-10040 nodemap: add nodemap idmap correctly

There are 3 situations when adding an idmap {id_client:id_fs} to
a nodemap tree:
- both id_client and id_fs are matched, that means this idmap
   already exists, so return -EEXIT;
- neither id_client nor id_fs is matched, that means this is a
   new idmap, so insert it;
- only "id_client" or "id_fs" is matched, since idmap uses
   "id_client" as index key, we need to delete that old idmap and
   its index by its id_client, and then insert it again. In the
   original implementation, idmap_insert() calls rb_replace_node()
   to replace the old idmap without re-sorting its both id_client
   and id_fs, so that this new added idmap can't be found by
   idmap_search() due to its wrong left and right nodes, and can't
   be deleted either.

Also, this patch improves the following code:
- nm_idmap_lock: use type rw_semaphore instead of rwlock, because
   nodemap_idx_idmap_del() may sleep on FS operations while this
   lock is being held.
- Add "update_idmaps" to sanity-sec.sh test_15 to verify this fix,
   and improve other part of this test case as well to make it run
   correctly.

Test-Parameters: testlist=sanity-sec
Signed-off-by: Stephan Thiell <sthiell@stanford.edu>
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Icf777f14c2e1dd56fa5cd0eb56666240e206d199
Reviewed-on: https://review.whamcloud.com/29364
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit 253ccbd55ffe7fcdc405c9fcc4f72a47578920fe)
Reviewed-on: https://review.whamcloud.com/30206

LU-10041 tests: sanity/51 to cleanup properly

sanity/51 should cleanup properly in case of failure

Test-Parameters: trivial testlist=sanity ostfilesystemtype=zfs mdtfilesystemtype=zfs
Test-Parameters: trivial testlist=sanity mdscount=2 mdtcount=4

Lustre-change: https://review.whamcloud.com/29246
Lustre-commit: 0d5ceb891e904cc9bab3a6b5ce0944b8bab06975

Change-Id: Ib1c0e1a116a6802f2bf25d665be5c076ae07c8b8
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30023
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10204 tests: Fix line continuation

There is a simple missing in one of the error paths in
ladvise_willread_performance which results in an unknown command
error. Fix it by adding the missing backslash.

Test-Parameters: trivial
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Change-Id: Ifff854c6cf5c1add71fd1571cb4de01d56204fde
Reviewed-on: https://review.whamcloud.com/29982
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-10007 pacemaker: Use lctl and load lustre

When scripts are started, load lustre module.
Use lctl instead of directly accessing health_check file.

Test-Parameters: trivial
Lustre-change: https://review.whamcloud.com/29144
Lustre-commit: 6e299c2a3bd7566a322cf9bc9494f6850d2c1b7e

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I4a81248939464e498006dc2c4072d44685add018
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29946
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-10015 o2iblnd: fix race at kiblnd_connect_peer

cmid will be destroyed at OFED if kiblnd_cm_callback return error.
if error happen before the end of kiblnd_connect_peer, it will touch
destroyed cmid and fail as
(o2iblnd_cb.c:1315:kiblnd_connect_peer())
ASSERTION( cmid->device != ((void *)0) ) failed:

Lustre-change: https://review.whamcloud.com/29134
Lustre-commit: 576551cb97c6a5efb27d8afc59ad60550dde51ce

Seagate-bug-id: MRP-4592
Signed-off-by: Alexander Boyko <alexander.boyko@seagate.com>
Change-Id: I83eb5bceeb567acef0316498b936d25d6c6ccd95
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29881
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9951 lustre_compat: add wrapper function for posix_acl_update_mode

posix_acl_update_mode is introduced in kernel 4.9, add the precheck
of it, if not exists, use inline wrapper function instead.

Lustre-change: https://review.whamcloud.com/28871
Lutre-commit: ec6bdd55bb2128dcf9779017ef9623461afe05f8

Change-Id: I8a1476d611c387a88efef5d5b8707edf5feacca8
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29880
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-10047 tests: stop skipping test_102 subtests

Remove obsolete version check on tar.
All tar versions in supported distros are now capable of --xattrs.
But not all tar version have --xattrs-include, so must test for and
adapt for that.

Lustre-commit: 036641a1e1f6d5d010873267b669528890f19fe4
Lustre-change: https://review.whamcloud.com/29288

Test-Parameters: trivial envdefinitions=ONLY=102 testlist=sanity
Test-Parameters: trivial clientdistro=el6.9 envdefinitions=ONLY=102 testlist=sanity
Test-Parameters: trivial clientdistro=sles12sp2 envdefinitions=ONLY=102 testlist=sanity

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I5e76bd1a762c4e01cf8a3a33789ca3a30c15abb0
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-on: https://review.whamcloud.com/29823
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9663 tests: replace “ls -s” with “stat -c '%s'” to get file size

This patch improves acl/2561{,_zfs}.test to use “stat -c '%s'” to
get file size instead of using “ls -s”, which prints the allocated
size of the file in blocks.

Test-Parameters: trivial \
envdefinitions=ONLY=103 \
mdtfilesystemtype=zfs ostfilesystemtype=zfs \
testlist=sanity

Lustre-change: https://review.whamcloud.com/27961
Lustre-commit: 45b8c6cf6916cf012ad0512f9cf249c152b45236

Change-Id: I8bfea5197102baeece244e20731c90172c5cc2f1
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29816
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9752 man: Reference zgenhostid instead of genhostid

In ZFS 0.7.0, they added zgenhostid(8) to be used in place of
Redhat's genhostid, so that there would be a platform agnostic
way to generate /etc/hostid.

Test-Parameters: trivial
Lustre-change: https://review.whamcloud.com/29327
Lustre-commit: a1eb6de081473545fbd5c1fe33e209fe391bf708

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I691266d04f91d5fa7c50b72948c801afa69d647d
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29805
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9578 llite: use security context if it's enabled in the kernel

if it's disabled, then Lustre stop to work properly (can not create
files, etc)

Lustre-change: https://review.whamcloud.com/27364
Lustre-commmit: 61f26ea47f96632a90702f2b404e845a01498612

Change-Id: I1e431ec95a2b0613b43893567eb6d1a64ec832de
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29737
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10141 llapi: integer overflow in llapi_changelog_start

Use the appropriate type to store and check the return value from lseek.
This prevents from misinterpreting high offsets as errors.

Lustre-change: https://review.whamcloud.com/29674
Lustre-commit: aaa808e7a40198581956a1d648f413cc30e830ed

Change-Id: I15e92be3454af20ee6611c2a7ddfc1b597d639c2
Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Thomas LEIBOVICI <thomas.leibovici@cea.fr>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29735
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-10150 build: Build with ZFS 0.7.3

Update ZFS and SPL version to 0.7.3
Also Fixup ChangeLog

Changelog: https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.7.3

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I88f0f2601262c0c535431d256a159d9e9d6ebd75
Reviewed-on: https://review.whamcloud.com/29713
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9840 lod: add ldo_dir_stripe_loaded

Add ldo_dir_stripe_loaded flag to avoid loading
stripes mulitple times especcially for non-stripe
directory.

Lustre-change: https://review.whamcloud.com/28962
Lustre-commit: 744fe412fa101609a38f7ccc77efc4f1c540e008

Change-Id: Ia9360aac9e24706e401184c75fae4ec7f8ec46d9
Signed-off-by: Di Wang <di.wang@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-on: https://review.whamcloud.com/29143
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10168 kernel: kernel update [SLES12 SP2 4.4.90-92.45]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
mdsdistro=sles12sp2 ossdistro=sles12sp2 \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: If24eddeeadc825b1b3073439304c0958c0ae1b74
Reviewed-on: https://review.whamcloud.com/29840
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10132 llite: handle xattr cache refill race

In ll_xattr_cache_refill() if the xattr cache was invalid (and no
request was sent) then return -EAGAIN so that ll_getxattr_common()
caller will fetch the xattr from the MDT.

Lustre-change: https://review.whamcloud.com/#/c/29654/
Lustre-commit: 9cc8c3a20c547ec75325dde3dd17f4b1dcc66348

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ia9ec7424e8786d92bdecf4897fafcf71d5061fb1
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29795
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-10164 kernel: kernel update [SLES12 SP3 4.4.92-6.18]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp3 testgroup=review-ldiskfs \
mdsdistro=sles12sp3 ossdistro=sles12sp3 \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I4b046f0ac64b75545cd0e2a3a44e9117f4d79ef4
Reviewed-on: https://review.whamcloud.com/29825
Tested-by: Jenkins
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10142 kernel: kernel update RHEL7.4 [3.10.0-693.5.2.el7]

update RHEL 7.4 kernel to 3.10.0-693.5.2.el7

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I6f22d9d96b86d7bd860a400cd803fad26a6b5d1a
Reviewed-on: https://review.whamcloud.com/29728
Tested-by: Jenkins
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10131 llite: Update i_nlink on unlink

Currently, the client inode link count is not updated on
last unlink.  This is fine because the dentries are all
gone and the inode is eligible for reclaim, but it's still
incorrect.  This causes two problems:

1. Inode is not immediately reclaimed
2. i_nlink count is > 0 for a fully unlinked file, which
confuses wrapfs

On last unlink, the MDT sends back attributes.  Use the
nlink count from these to update the client inode.

Remove null check inherited from ll_get_child_fid, because
the inode should never be null on an unlink.

Re-enabled test 76, which passes with this patch.
Removed slab allocator tuning from test_76, because slab is
no longer the default Linux allocator.

Lustre-change: https://review.whamcloud.com/29651
Lustre-commit: 947b22b08d3725852a811a9e330562dd986d7336

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: Ib253b5cf3d35188554cf8fc33a8a3d4b8bb237e8
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29785
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10041 osd: osd-zfs to choose dnode size

depending on dnodesize property it can be:
legacy (512 bytes), auto (512 bytes to 16K) or absolute
size (512, 1024, 2048, 4096, 8192, 16384).

Lustre-change: https://review.whamcloud.com/29242
Lustre-commit: d993de7b40cd9625b48e5361ae29f77bfceb207a

Change-Id: Iea35d8ae850523440272467320410850821f484c
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29770
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9782 osd-ldiskfs: avoid extra search

Extent tree grow greatly durin random IO test with small block size.
osd_is_mapped responsible to large cpu consumption in this case.

|          |
|          |--94.49%-- ldiskfs_es_find_delayed_extent_range
|          |          ldiskfs_fiemap
|          |          osd_is_mapped
|          |          osd_declare_write_commit
|          |
|          |--5.49%-- ldiskfs_fiemap
|          |          osd_is_mapped
|          |          osd_declare_write_commit
|
|--21.80%-- ldiskfs_es_find_delayed_extent_range
|          |
|          |--100.00%-- ldiskfs_fiemap
|          |          osd_is_mapped

let's cache a osd_is_mapped result to avoid extra search in extent
tree,

Lustre-change: https://review.whamcloud.com/28145
Lustre-commit: 1eb0573fdeb7ef8ed667974fd30d4d110b8166c9

Seagate-bug-id: MRP-4474
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@seagate.com>
Change-Id: I63d480bfc7c6b7599b80ceeec9447b227a1610c8
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29739
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-7990 llite: increase whole-file readahead to RPC size

Increase the default whole-file readahead limit to match the current
RPC size. That ensures that files smaller than the RPC size will be
read in a single round-trip instead of sending multiple smaller RPCs.

Lustre-change: https://review.whamcloud.com/26955
Lustre-commit: 627d0133d9d7bef908544537a8a2b2093719fc72

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I3bdb1c7f92c546d58951a9e6b783af23c83ebbe5
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29738
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-4705 mdc: improve mdc_enqueue() error message

Include the parent/child FIDs and name in the mdc_enqueue()
debug message.

Test-Parameters: trivial

Lustre-change: https://review.whamcloud.com/28978
Lustre-commit: 471c5303eb29d5ea1ba5a683173bda63095dae78

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I7b84921a52a4650be70fe87eea691ba2217bb3a6
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29736
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9814 ldiskfs: restore simple_strtol in prealloc

Since kstrtol needs a null-terminor string so we
back to use simple_strtol in prealloc patches.

Lustre-change: https://review.whamcloud.com/28553
Lustre-commit: 685ef61f0fb9849e3f7a32c4ebf2980d3300afb0

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I331308d830fbeef9c00156bb8c14b43651d66420
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29733
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10029 osd-ldiskfs: make project inherit attr removeable

Inherit attribute should be clearable now

Lustre-change: https://review.whamcloud.com/29189
Lustre-commit: 9d068048603fefbab1eea8d8bb5a246ce4375dca

Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I878fde0dc134c9820436ee80979d87e6dacfb70d
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29688
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9140 nrs: measure the runtime of dd directly

This patch changes the way to measure the runtime of "dd". Instead
of parsing the output of "dd", we use date command to calculate
the runtime of dd directly, avoiding the parsing failure caused
by changed output format of "dd".

Lustre-change: https://review.whamcloud.com/27878
Lustre-commit: 66abf6ffe9f032aaa7c271ee2bd1f9b569058bd5

Change-Id: Ibd2e3963f791404ee927981238227012cf4dbf2c
Test-Parameters: trivial testlist=sanityn
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29686
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9260 test: Use the correct mount device when test against lustre

The changes pass the MGSNID:/FSNAME into test, instead of
using the default loop device when testing against lustre.
The corresponding changes to the Posix test suites are also needed
to make the testing pass. Related changes apply to toolkit.

Test-Parameters: trivial testlist=posix

Lustre-change: https://review.whamcloud.com/28661
Lustre-commit: 7b59ed3ab3c9bfae95b9904982869d31a7e65770

Change-Id: I32fc5a401fdc53ed133a78dc4c84b4a7e2a5ad19
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29628
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9888 tests: Update disk2_7-zfs.tar.bz2 for quota

Add a blimit file with the larger 40960 limit in it, as this seems to
be how the image was created, but the default (without this file) is
20K.
Also add an ilimit file with 4 (default for new images). Old default
value is 2.

This patch rolls in LU-9826 fix for larger ZFS
Lustre-change: https://review.whamcloud.com/28464
Lustre-commit: 6ab9866730c1bd2344869502586fc322f6ed7b71

This patch on master
Lustre-change: https://review.whamcloud.com/28820
Lustre-commit: f37b25fbb4acf3f96ebc61bb4d9137a8f0401e35

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I33921ac58a5252f3259145d5e00faedcd21559f9
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29068
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-7988 hsm: update many cookie status at once

Instead of calling mdt_agent_record_update, which calls
cdt_llog_process, once for every HAL, build a list of the cookies to
update with their status and call mdt_agent_record_update just once
per seconds at most.

Update mdt_agent_record_update to take a status for every cookie.

Test-Parameters: trivial testlist=sanity-hsm

Lustre-change: https://review.whamcloud.com/19584
Lustre-commit: f3a415289b560b5f422efe2bd08b3b7cff113cf0

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: Ie4afd667727e07570ed6a2d51e8dfaea8302b97b
Signed-off-by: Ben Evans <bevans@cray.com>
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28973
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-3719 ldiskfs: adjust s_mb_prealloc_table_size correctly

When mb prealloc table item value is not valid, the
s_mb_prealloc_table_size should be adjust accordingly.
Port patch to rhel7 & sles12.

Lustre-change: https://review.whamcloud.com/27748
Lustre-commit: b9ecf2671cba08530cc49d47713a360f2c6c25ca

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alexander Boyko <alexander_boyko@xyratex.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I5aa3a32a6a3aedc70160409c6443746fd2ccbbc9
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28228
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9462 doc: update lfs setstripe man page and usage

Update the lfs-setstripe.1 man page formatting and content.
Update the "lfs setstripe" usage message to be in "common use" order.

Test-Parameters: trivial
Reviewed-on: https://review.whamcloud.com/27066
Lustre-change-by: Andreas Dilger <andreas.dilger@intel.com>
Lustre-commit: aa83ef5a9ec470d2f6af07d47b3dd763f4a0e7ce

Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ia761c7562fa398a8c1bb4354d09757cd5f3ebbe5
Reviewed-by: Jian Yu <jian.yu@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29627
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9158 test: Use project ID for project quota for quota_scan

Use project ID instead of quota user for project quota in
function quota_scan. This is not a fix for the rebalancing error,
only correct the test script issue.

Lustre-change: https://review.whamcloud.com/28957
Lustre-commit: 3a2f24fefbc8c4b6e98dfd6ba9957326ea0e7a97

Change-Id: I9165ada17de0f32ac38f720a5e9e9be46363b41f
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29626
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9741 test: Correct check of stripe count for directories

With the progressive file layout feature, the 'lfs getstripe -d'
output will report stripe count for each component. Since
sanity test 27w checks that there is only one "stripe_count"
line in the 'lfs getstripe -d' output, this needs to be changed
to check that there is at least one "stripe_count" reported.

Test-Parameters: trivial
Lustre-change: https://review.whamcloud.com/27980
Lustre-commit: 1cc354d559d1df90a2eaa886eb0e3956bf385a9e

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I51f141b2542e65d1bc296cb2cb14c12f22afdcec
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29625
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9672 gss: fix expiration time of sunrpc cache

Expiration time of sunrpc cache is misinterpreted. Downcal
and response from user space must provide an epoch time,
not a duration.
And on kernel side, expiry must always be expressed in seconds
from boot, as set when retrieved from get_expiry().

Lustre-change: https://review.whamcloud.com/27667
Lustre-commit: f35425801545e0b47279597983cfbd02e837c45f

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I35c58a040a62410374dee0be3ae5bed7956cd985
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29624
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9967 tests: run-llog to cleanup properly

move ignore_errors so that setup's error is ignored, the script
continues with detach command needed to release the module.

Lustre-change: https://review.whamcloud.com/29057
Lustre-commit: e3921c4727c1d2e9866e0013fd37a7a4d4616621

Change-Id: I899de7a321fa927ca9884a6e4f9e8785bbc46f7c
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29366
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9887 lfsck: calculate LFSCK speed properly

Originally, we used do_div(a,b) to calculate the LFSCK average
speed, and got the result from the parameter @a. But later, we
replaced do_div(a,b) with div_u64(a,b). The latter one doesn't
stores the quotient in the parameter @a, instead, the quotient
is returned via the function return value. The patch fixes the
LFSCK logic to obtain the LFSCK average speed from div_u64(a,b)
return value.

Lustre-change: https://review.whamcloud.com/28617
Lustre-commit: cf800c062c8c6424c442509139297095f8a708db

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I442fb8f7e6c51a4853ea37694e3c221f97e26b19
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29294
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9963 test: add parallel-scale tests to ALWAYS_EXCEPT

add the following parallel-scale tests

parallel_grouplock :
test_parallel_grouplock: test failed to respond and timed out
Associated Jira ticket LU-9429 is in open state.
to ALWAYS_EXCEPT list.

Test-Parameters: trivial testlist=parallel-scale

Lustre-change: https://review.whamcloud.com/28914
Lustre-commit: 38c78e21ca7abe4bd6e6500d88a692e7a56a7712

Change-Id: I25709af1ab49a30498a89e5369521582c5ab6cf8
Signed-off-by: dilip krishnagiri <dilipx.krishnagiri@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: James Casper <jamesx.casper@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29146
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9802 pfl: swapping lcm_entry_count correctly

It's a u16 integer so it should use le16_to_cpu() instead of
le32_to_cpu().

Test-Parameters: trivial

Lustre-change: https://review.whamcloud.com/28256
Lustre-commit: 21ea1f7629535090d64ee3355c491a9e35256f23

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I43c31a76d78aa294a3e3296a1bb69f4d6fb1423d
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29141
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10023 kernel: kernel update [SLES12 SP3 4.4.82-6.9]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp3 testgroup=review-ldiskfs mdsdistro=sles12sp3 ossdistro=sles12sp3 mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs
Change-Id: Ie65afb3d7356c4679f4f37f4af324e955261b5af
Signed-off-by: Bob Glossman <bob.glossman@intel.com>

LU-9956 kernel: kernel upgrade [SLES12 SP3 4.4.82-6.3]

Minor linux version upgrade, but SP2 and SP3 use linux 4.4 versions.
Some new kernel patches, a few revised ldiskfs patches.
All new target and config files.
Some autoconf changes to adapt to new or altered kernel APIs.

Test-Parameters: clientdistro=sles12sp3 testgroup=review-ldiskfs mdsdistro=sles12sp3 ossdistro=sles12sp3 mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I99e2b6848197ea19402fa415fdb562d03e87d947

LU-10119 scripts: Correct shebang/hashpling format

Shebang/hashpling should not have a space between the number sign and
exclamation mark.

Test-Parameters: trivial
Lustre-change: https://review.whamcloud.com/29605
Lustre-commit: dd23aa4a64c776784cee05f69560c00b36440dc7

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I4a31d7dd7579bfaf231284c678e4100acea77d9b
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29689
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9416 hsm: add kkuc before sending registration RPCs

This avoids a situation where the registration completes and the CDT
sends HSM actions just before the kkuc registration happens. In this
case the client drops the actions because there are no CT pipes in the
kkuc list.

Lustre-change: https://review.whamcloud.com/28751
Lustre-commit: bdb04079570dd2de3b9310221e0206d64efb9ecc

Change-Id: Icbd6575f04c0ca7e8f731ee8481ec72a9ff4f2e1
Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29687
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9574 llite: pipeline readahead better with large I/O

Fixed a bug where next readahead is not set correctly when
appplication issues large I/O;
Extend the readahead window length to at least cover the size of
current I/O.

Lustre-change: https://review.whamcloud.com/#/c/27388/
Lustre-commit: e94dcdcf70526177633b71cfa1c6cb68c8f28ef0

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I43c5e4f25ea30d4a36263db2588bde0401122990
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29016
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10000 kernel: kernel update [SLES12 SP2 4.4.74-92.38]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
mdsdistro=sles12sp2 ossdistro=sles12sp2 \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I825369d9cafd4f75dc193f614d5adfe87769fcd1
Reviewed-on: https://review.whamcloud.com/29069
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9558 tests: add completion.h header to kinode.c

When building lustre on a ARM fedora box running a 4.11 kernel
kinode.c failed to build due to the completion.h header missing.
A simple including of this header fixes everything.

Test-Parameters: trivial

Lustre-change: https://review.whamcloud.com/28998
Lustre-commit: 982d660edbcacd528579f25ed2a3ae076a0134f3

Change-Id: I2a193346b214616a6f5dee5049466fb55567e396
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Frank Zago <fzago@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/29567
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-7802 ldlm: No -EINVAL for canceled != unused

If any locks are removed from or added to the lru, the
check of "number unused vs number cancelled" may be wrong.
This is fine - do not return an error or print debug in
this case.

Lustre-change: https://review.whamcloud.com/28560
Lustre-commit: a5081b7362e44b8d38aee1112f9a7d3aae1642c0

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I2ee9b8d86fbd6c9bd2c29e3472e3d410ee303374
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28975
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-8276 ldlm: Make lru clear always discard read lock pages

A significant amount of time is sometimes spent during
lru clearing (IE, echo 'clear' > lru_size) checking
pages to see if they are covered by another read lock.
Since all unused read locks will be destroyed by this
operation, the pages will be freed momentarily anyway,
and this is a waste of time.

This patch sets the LDLM_FL_DISCARD_DATA flag on all the PR
locks which are slated for cancellation by
ldlm_prepare_lru_list when it is called from
ldlm_ns_drop_cache.

The case where another lock covers those pages (and is in
use and so does not get cancelled by lru clear) is safe for
a few reasons:

1. When discarding pages, we wait (discard_cb->cl_page_own)
until they are in the cached state before invalidating.
So if they are actively in use, we'll wait until that use
is done.

2. Removal of pages under a read lock is something that can
happen due to memory pressure, since these are VFS cache
pages. If a client reads something which is then removed
from the cache and goes to read it again, this will simply
generate a new read request.

This has a performance cost for that reader, but if anyone
is clearing the ldlm lru while actively doing I/O in that
namespace, then they cannot expect good performance.

In the case of many read locks on a single resource, this
improves cleanup time dramatically. In internal testing at
Cray with ~80,000 read locks on a single file, this improves
cleanup time from ~60 seconds to ~0.5 seconds. This also
slightly improves cleanup speed in the case of 1 or a few
read locks on a file.

Lustre-change: https://review.whamcloud.com/20785
Lustre-commit: 6a369b59f3729513dd8e81c4964dc6183287b601

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I0c076b31ea474bb5f012373ed2033de3e447b62d
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29264
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-8578 ldlm: ELC shouldn't wait on lock flush

commit ad87adbff5148a74ac616dbc9bfa02a4e3814034
LU-5727 ldlm: revert the changes for lock canceling policy

but it removes the fix for LU-4300 when lru_resize is disabled.

Introduce ldlm_cancel_aged_no_wait_policy to be used by ELC.

Lustre-change: https://review.whamcloud.com/22286
Lustre-commit: 7ca60f338936465c534c2ac9a0ea467d276a7485

Change-Id: Ic74a6bce4a261dc96c8c0f6d8f7e20315def8c0e
Seagate-bug-id: MRP-3662
Signed-off-by: Andriy Skulysh <andriy.skulysh@seagate.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@seagate.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29263
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9469 ldiskfs: add additional attach_jinode call

In some execution paths jinode data structures are not
getting initialized. Add an extra init call to fix that.

Test-Parameters: clientdistro=sles12sp2 testlist=conf-sanity \
mdsdistro=sles12sp2 ossdistro=sles12sp2 testgroup=review-ldiskfs \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Lustre-change: https://review.whamcloud.com/28665
Lustre-commit: 84f690eee28f3b9123b89249c568dd6255a4a348

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I087eb06b9c5122be1cfd0aabbc04ea1db7ec765a
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Hongchao Zhang <hongchao.zhang@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28968
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9908 tests: force umount client in test 70e, 41b, and 105

In test_70e, import state may not update while
mds stopping. Since statfs will be invoked in sles12
before umounting, so umount with force flag to avoid
waitting a long time.
Add -f to stopall call in test-framework for the same reason,
to do umount with force flag.

Lustre-change: https://review.whamcloud.com/28767
Lustre-commit: 20787a89ad7dc99290c4d9cff8247b0a532e92b9

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I3d1e73b3501e98008ef18c05f7b5498d12cb46fb
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29108
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9929 nodemap: add default ACL unmapping handling

This patch adds default ACL unmapping code to mdt_getxattr
functions so that clients can get a correctly unmapped id.
Also, test_23b is added to sanity-sec.sh to verify this fix.

Lustre-change: https://review.whamcloud.com/29010
Lustre-commit: 62fee20556a4c90361bd28edb903dc77c9540133

Change-Id: I6562372c58ca9772f16f7d6b0b98b45ada87971a
Test-Parameters: testlist=sanity-sec
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29369
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10088 kernel: kernel update RHEL6.9 [2.6.32-696.13.2.el6]

Update RHEL6.9 kernel to 2.6.32-696.13.2.el6

Test-Parameters: clientdistro=el6.9 mdsdistro=el6.9 \
ossdistro=el6.9 mdtfilesystemtype=ldiskfs \
ostfilesystemtype=ldiskfs testgroup=review-ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Iedd8e4fc71fabfd7f779804ce546b5e3173d23d6
Reviewed-on: https://review.whamcloud.com/29362
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9887 tests: ignore error sanity-lfsck test 9a,b

sanity-lfsck tests 9a and 9b are failing consistently on
checking that speed limiting LFSCK takes less time than the
user defined maximum speed. We should ignore these errors
for now and print the layout or namespace to help understand
this issue.

Test-Parameters: trivial testgroup=review-zfs-part-1
Test-Parameters: testgroup=review-dne-part-2

Lustre-change: https://review.whamcloud.com/28588
Lustre-commit: 6754b09335508ca4d977d10d1d05b5befd1a8aad

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I64cac59edd456e6fd519961a4055130c8dbc8a4a
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29293
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-8541 ldlm: don't use jiffies as sysfs parameter

The ldlm sysfs file handles lru_max_age in jiffies which is wrong
since jiffies are not consistent across machine since HZ is
configurable at compile time. Talking to most users they thought
lru_max_age was in seconds which is incorrect. The best way to
fix this is to move lru_max_age to millisecs since most systems
lustre deals with sets HZ to 1000. To make it clear it is in
milliseconds print out lru_max_age with "ms". Since users tend
to think in seconds allow passing in seconds besides milliseconds
and internally converting them to nanaseconds. Since we have to
support milliseconds move to ktime_t since we can't use time64_t.
Unfortunately, this makes a relatively large patch, but I could
not find a way to split it up some more without breaking atomicity
of the change.

Lustre-change: https://review.whamcloud.com/28370
Lustre-commit: 800ffd4711863e1067a693f1283bccf4edddf2a2

Change-Id: I0b1814fd9d903767f62fe141d2c95845b75fb95a
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29265
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9983 llog: skip non-exist log cancellation

Let's skip the record cancellation if the llog does
not exist at all, which usually happens when remote
log creation failed.

Signed-off-by: Di Wang <di.wang@intel.com>
Change-Id: I8b27978b725acf8dbffd1a743873a3854ee15efb
Reviewed-on: https://review.whamcloud.com/29238
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9980 tests: save specific facet in save_lustre_params()

In save_lustre_params(), while there are multiple server facets
having the same host, and the parameter has wildcard, duplicate
parameters with wrong facets will be saved.

This patch fixes the above issue by greping service name to save
the parameter with specific facet.

Test-Parameters: clientcount=4 osscount=2 mdscount=2 mdtcount=1 \
austeroptions=-R failover=true iscsi=1 testlist=replay-vbr

Lustre-change: https://review.whamcloud.com/28963
Lustre-commit: 7921d0d8db942057eb10042291d6d90448e5c197

Change-Id: Icba3fc532f4c67f02272c39e8e64d49325dad0e7
Signed-off-by: Elena Gryaznova <elena.gryaznova@seagate.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29147
Tested-by: Jenkins
Reviewed-by: Elena Gryaznova <c17455@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9930 llite: only clear lli_sai if the setter

Previous to this patch, start_statahead_thread() was
unconditionnally clearing lli->lli_sai upon error, leading
to crash upon racy scenario where it has just been set in/by
another thread context.
Now, only clear lli_sai if current thread has set it.

Lustre-change: https://review.whamcloud.com/28794
Lustre-commit: f4a0ebe28e88cbdd79061fad275cb9494fdba225

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I555febfad3494c9dd90eeb72d6dd9157428179ea
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29145
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9682 nodemap: delete nids range from nodemap correctly

In function nodemap_del_range(), we should check if the current
nodemap has the specified range before delete it from global
range tree.
Also, test_10b is added to sanity-sec.sh to verify this patch.

Lustre-change: https://review.whamcloud.com/28922
Lustre-commit: 62ca9b823524ca44d3344b2555754890abc55a3b

Change-Id: Ibab79056509d14d52f99b1ebe3319c301dbe45d9
Test-Parameters: testlist=sanity-sec
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29138
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9996 build: include MOFED lib

Test-Parameters: trivial

Lustre-change: https://review.whamcloud.com/#/c/29020/
Lustre-commit: 81ebf59c99b1e58d05d6d9af1ce645b09ecf3792

Change-Id: Ia1b5a97f59d5055311786934c9165a81f2af7cae
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29118
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9558 llite: user enhanced getattr functionality in newer kernels

Starting with the linux kernel 4.11 a new system call was added
to enhance the file info available. This new functionality has
changed the getattr functions in the linux kernel. This patch
only provides the most basic support for now.

Linux-commit: a528d35e8bfcc521d7cb70aaf03e1bd296c8493f

Lustre-commit: 57730260e846cd5011469fe2edc667ef73ad3bad
Lustre-change: https://review.whamcloud.com/27715

Change-Id: Iee1b02bb94dab05a3197866af9eedd6f35cac7c5
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-on: https://review.whamcloud.com/29006
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9870 build: handle SNMP missing on build box

Currently the lustre spec file doesn't handle the case when SNMP
is missing. So even if the user does configure --disable-snmp our
rpm build process will ignore this and fail to build rpms. Pass
to the rpm build process the missing SNMP case.

Lustre-commit: 574451a4402df65d26d7af96d94a7bf0875c86af
Lustre-change: https://review.whamcloud.com/28494

Test-Parameters: trivial

Change-Id: Ia6dcfd31b50f4f67afe7a4545fe417c32df6e559
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/28987
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9733 obd: fix fsname parsing

Lustre support Lustre fsname started with numbers.
So here we should try fsname firstly, if it failed,
fall back to number parsing.

Lustre-change: https://review.whamcloud.com/27916
Lustre-commit: cae71501680330a38c9bb16c25bf9c85dc0ec4e0

Change-Id: Idd98b9fdc8375729faa61f51251caa101035dd93
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28814
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9417 mdc: excessive memory consumption by the xattr cache

The refill operation of the xattr cache does not know the
reply size in advance, so it makes a guess based on
the maxeasize value returned by the MDS.

In practice, it allocates 16 KiB for the common case and
4 MiB for the large xattr case. However, a typical reply
is just a few hundred bytes. Even worse, RHEL6 has a very
bad vmalloc()/vfree() design with global locks.

If we follow the conservative approach, we can prepare a
single memory page for the reply. It is large enough for
any reasonable xattr set and, at the same time, it does
not require multiple page memory reclaim, which can be
costly.

If, for a specific file, the reply is larger than a single
page, the client is prepared to handle that and will fall back
to non-cached xattr code. Indeed, if this happens often and
xattrs are often used to store large values, it makes sense to
disable the xattr cache at all since it wasn't designed for
such [mis]use.

Lustre-change: https://review.whamcloud.com/26887
Lustre-commit: 94f080b395d6517e8da06a945bb10b0bfe69ad78

Change-Id: I98d97ffea5c83bccbc9f254717af8d2c0ac4d77f
Signed-off-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28223
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

New release 2.10.1.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Id69eb38fd2bb3987005fb0f4ec0a6da460dd1a19

New tag 2.10.1-RC2.

Second release candidate for 2.10.1 release.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I849387cd0dac7a3897fd83ae250062bbcea5fe32

LU-10037 kernel: kernel update RHEL6.9 [2.6.32-696.10.3.el6]

Update RHEL6.9 kernel to 2.6.32-696.10.3.el6

Test-Parameters: clientdistro=el6.9 mdsdistro=el6.9 \
ossdistro=el6.9 mdtfilesystemtype=ldiskfs \
ostfilesystemtype=ldiskfs testgroup=review-ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ic693e29056e099638ee49403b114f6872f7bc73f
Reviewed-on: https://review.whamcloud.com/29248
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9983 osp: align the OSP request size by 4k

Let's align the OSP request size by 4k, otherwise OUT
might get incorrect out requests.

Signed-off-by: Di Wang <di.wang@intel.com>
Change-Id: Id5c16b031dcb16d764c4e4f325f51b9ecf454533
Reviewed-on: https://review.whamcloud.com/29208
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

New tag 2.10.1-RC1.

First release candidate for 2.10.1 release.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ie2149519d4e272ca1a170f33871b28243ca897da

LU-9978 kernel: kernel update RHEL7.4 [3.10.0-693.2.2.el7]

update RHEL 7.4 kernel to 3.10.0-693.2.2.el7

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I2fe4e038f185f8d4acec74d9cc398da987bcad6b
Reviewed-on: https://review.whamcloud.com/29002
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9816 kernel: kernel upgrade RHEL7.4 [3.10.0-693.el7]

With this mod we switch our supported el7 version to RHEL 7.4

Lustre-change: https://review.whamcloud.com/#/c/28301/
Lustre-commit: b2a39555d1bd4a75017ee6ca2a9a6920a30bafa8

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ib0bcb3547fcf220e1b4665229930c6ff28c6906c
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28532
Tested-by: Jenkins
Tested-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9767 utils: validate filesystem name for mkfs.lustre

In an earlier LU-6401 patch various user land functions
used to validate poolnames and file system names were
removed. The checks instead were enforced on the kernel
side to ensure any possible user land software directly
interfacing to the kernel wouldn't be able to break
things badly. For the case of formating the backend
file system no kernel interaction doesn't happen until
it tries to mount the MDT/OST/MGT which is very late
in the process. So for this case lets add back the
file system name verification to mkfs.lustre to warn
users long before they try to mount anything.

Secondly we remove the verify_poolname() in lfs.c since
it really doesn't do much.

Test-Parameters: trivial

Lustre-commit: 28216a73f4c0b19ef45e3ec2d6949801f6b9f828
Lustre-change: https://review.whamcloud.com/28070

Change-Id: If094644e56a70b6dd8e6b0378adc8736911aeef1
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/28070
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/28996
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9979 kernel: kernel update RHEL6.9 [2.6.32-696.10.2.el6]

Update RHEL6.9 kernel to 2.6.32-696.10.2.el6

Test-Parameters: clientdistro=el6.9 mdsdistro=el6.9 \
ossdistro=el6.9 mdtfilesystemtype=ldiskfs \
ostfilesystemtype=ldiskfs testgroup=review-ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ie86a84fda1ec391c4e0b9ab18a82d4a5b0bd25d1
Reviewed-on: https://review.whamcloud.com/28969
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Jenkins
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9960 osd-zfs: don't auto-upgrade quota

To preserve the ability to down-grade from 0.7.x to 0.6.x,
don't auto-upgrade quotas.
Print warning if quotas haven't been upgraded when mouting with 0.7.0.
Do check based on zpool feature in sanity-quota instead of just
version.

Lustre-change: https://review.whamcloud.com/#/c/28924/
Lustre-commit: 0bbef0afc16081e1af3529642436864954a73e3c

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I2b0dcba3a230c9b2dec3d07d1b4ca6f1a1717d47
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28954
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9907 build: add patchless server for lbuild

Adding lbuild support for building patchless server
Cleanup unused TARGET_ARCHS and BUILD_ARCHS

Test-Parameters: trivial

Lustre-change: https://review.whamcloud.com/#/c/28672/
Lustre-commit: 7018f18957817afd5dcd3b6f173e7634b93101e9

Change-Id: I946352fa243c86d5729779406264e6ee37856145
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28952
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9347 ioctl: Add BLKSSZGET ioctl support

Add BLKSSZGET ioctl and return PAGE_SIZE for the minimun
alignment from ll_file_ioctl() for this call.

Lustre-change: https://review.whamcloud.com/28578
Lustre-commit: dcc32cd7d0d89a49f0c73ecf99130a2678442e55

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Id8a77e77cd7e1807aa90474ca6d3d1fea4d7c269
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28861
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-8958 llite: remove llite_loop left overs

With the removal of llite_loop several pieces of code are still
present in the llite layer that were only used by the lloop device.
We can remove these no longer used pieces.

Lustre-change: https://review.whamcloud.com/26795
Lustre-commit: 2e875d5fc5b73f735bd42f5da54c23e4c2d35d5c

Change-Id: I67f2761ae29b8ba7bb3bc9bcc3e3f8ece73a3ea3
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28860
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9842 osd: return ENODATA for XATTR_NAME_FID on MDT

The XATTR_NAME_FID xattr is OST side EA, if someone calls
getxattr() for XATTR_NAME_FID on MDT, then return -ENODATA.

Lustre-change: https://review.whamcloud.com/28434
Lustre-commit: eb34cf7695766b39e15704861d6ac3d636042196

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I18b1466cf62d10fa28f7ed9731490e963b6274f4
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28761
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9841 lov: do not split IO for single striped file

stripe size for single striped file is not reliable, it shouldn't
be used to split I/O.

Lustre-change: https://review.whamcloud.com/28451
Lustre-commit: 078a099d26ef7f5d26131c0e18615855a39f341d

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I47c31d59b46b07d4a6760b8985e1c19da4765a5c
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28760
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9828 ptlrpc: Do not assert when bd_nob_transferred != 0

There is a case in the routine ptlrpc_register_bulk() where we were
asserting if bd_nob_transferred != 0 when not resending. There is
evidence that network errors can create a situation where
this does happen. So we should not be asserting!

This patch changes that assert to an error return code of -EIO.

Lustre-change: https://review.whamcloud.com/28491
Lustre-commit: e6490ea6cf0b793c0b47f17ac5a5fa3a2a136e0d

Signed-off-by: Doug Oucharek <doug.s.oucharek@intel.com>
Change-Id: I6a73ca1b04a86f187744d3b8b5d46df71d95e416
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28759
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9558 llite: handle struct vm_operations changes

For the linux 4.11 kernel passing in struct vm_area_struct
to struct vm_operations members has been removed since
struct vm_area_struct has been merged into struct vm_fault.
Handle these changes in the llite layer.

Linux-commit: 11bac80004499ea59f361ef2a5516c84b6eab675

Test-Parameters: trivial

Lustre-commit: a1fc8dffef216b71cb4a29a5a8faa2aa7919d2ae
Lustre-change: https://review.whamcloud.com/27651

Change-Id: I3f8767f02695515d83c59a61f6e9921b3d823109
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/28573
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9104 obd: Ignore unknown config param while mounting

class_process_proc_param() returns positive value when it encounters
unknown parameters in order to have the below levels process them.
At the very bottom layer the positive value returned by
class_process_proc_param() needs to be dropped.osd_process_config()
missed that, which resulted in target mount failure.

Make sure that osd_process_config() does not return positive value
returned by class_process_proc_param().

Test case has been added to check processing of unknown config
parameters.

Lustre-change: https://review.whamcloud.com/25368
Lustre-commit: 1385dbf8b9caedca7bb32f35db1529e4d5c52d4f

Test-Parameters: testlist=conf-sanity
Signed-off-by: Parinay Kondekar <parinay.kondekar@seagate.com>
Signed-off-by: Rahul Deshmukh <rahul.deshmukh@seagate.com>
Seagate-bug-id: MRP-4162
Change-Id: I068b8f2aee4cee69629efc83745d7cb88aea268c
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28232
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9654 mdt: fix problem of RAoLU HSM policy

mdt_attr_get_complex() clears all known attributes even they are
already valid. So in mdt_handle_last_unlink(), the valid attributes
of HSM or nlink should be checked before calling that function.

Lustre-change: https://review.whamcloud.com/27564
Lustre-commit: 2e4e60234028743404a302893fb1aa9f8d6a95e7

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I9ba561cadcc40baf5e28172cfda699cdecce7ea8
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28220
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9781 llog: Improve catalog full warning

When warning that a catalog file is full, provide the name
of the catalog file. If the name of catalog file isn't
defined, print its FID.

Lustre-change: https://review.whamcloud.com/28093
Lustre-commit: 1c4900dcd367d2a0b2e6f26796328f5aa12508db

Signed-off-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Change-Id: I559e43d08febfd8a1512ceb58fd3030b06372e9f
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28815
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9410 ldiskfs: no check mb bitmap if flex_bg enabled

When initializes (reformat) the filesystem, the number of
free blocks in the group descriptor is calculated via the
ext2fs_reserve_super_and_bgd() (e2fsprogs). As commented
in such function: "This is not necessarily the case when
the flex_bg feature is enabled, so callers should take care!".

So it is normal that we may find the block group descriptor
that has LDISKFS_BG_BLOCK_UNINIT flag but with 0 free blocks.
The ldiskfs_mb_check_ondisk_bitmap() should NOT report error
for such block group, instead, skip the check directly.

Lustre-change: https://review.whamcloud.com/28566
Lustre-commit: 5506c15a65b3eebb9f15000105e6eb7c02742a10

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iba0fb2bf0632a6e54222472bc724a8ea0478e9ae
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28765
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9890 osd-zfs: dmu_objset_own/disown changes

ZFS 0.8.0 will introduce ZFS encryption. The interfaces
to 'dmu_objset_own' and 'dmu_objset_disown' have changed.
Add configure checks to determine which versions of these
functions are available and call them appropriately.

Test-Parameters: trivial ostfilesystemtype=zfs mdtfilesystemtype=zfs testlist=sanity

Lustre-change: https://review.whamcloud.com/28593
Lustre-commit: 0fedb017c12629d145fa0577451d43adc757eb36

Signed-off-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Change-Id: Ide1a712858770e373404445b06596130a574d85b
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28764
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9869 lnet: fix incorrect arguments order calling lstcon_session_new

The arguments args->lstio_ses_force and args->lstio_ses_timeout are
in the incorrect order. Fix this by swapping them around.

Detected by CoverityScan, CID#1226833 ("Arguments in wrong order")

Test-Parameters: trivial testlist=lnet-selftest

Lustre-change: https://review.whamcloud.com/28487
Lustre-commit: 0a80067c60043da428c473c331ed7b602c6e960b

Change-Id: If11c574655425db5bbf21ba2264be8d83a7e8bf8
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28763
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9848 llog: check padding size for update reclen

Update log only checks padding size for split case, which should also
be done if it's less than chunk size.

Lustre-change: https://review.whamcloud.com/28554
Lustre-commit: 6705d6cc40a83b0e94668d651c444c855482bd01

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Ie7819f67dd9bcbfb060713bb208c9777420c5178
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28762
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9499 lfsck: set target bitmap properly

If the notify from the peer server has LF_INCOMPLETE flags,
then record it in the target bitmap unconditionally to avoid
missing to update the bitmap for some corner cases.

This patch also addes more debug information when the LFSCK
updates the bitmap and handle double_scan_result.

Lustre-change: https://review.whamcloud.com/27632
Lustre-commit: 775780cb90ba6069aefc7063adfe6862b26ce935

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I3a6195136d608aa47e59e61f95c92978503e3a4b
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28518
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-8760 lib: avoid unexpected out of order execution

There is race condtion in __l_wait_event() because of the
out-of-order execution between changing thread state and
checking condition. It may block the thread (to be waken)
for ever. Consider the following real execution order:

1. Thread1 checks condition on CPU1, gets false.
2. Thread2 sets condition on CPU2.
3. Thread2 calls wake_up() on CPU2 to wake the threads with
   state TASK_INTERRUPTIBLE | TASK_UNINTERRUPTIBLE. But the
   Thread1'sstate is TASK_RUNNING at that time.
4. Thread1 sets its state as TASK_INTERRUPTIBLE on CPU1,
   then schedule.

If the '__timeout' variable is zero, the Thread1 will have
no chance to check the condition again.

Generally, the interval between out-of-ordered step1 and step4
is very tiny, as to above step2 and step3 cannot happen. On some
degree, it can explain why we seldom hit related trouble. But
such race really exists, especially consider that the step1 and
step4 can be interruptible.

The patch adds barrier between changing thread's state and
checking condition to avoid out-of-order execution.

Lustre-change: https://review.whamcloud.com/23564
Lustre-commit: c2b6030e9217e54e7153c0a33cce0c2ea4afa54c

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I32caee6b332f037d864419ea8728112da563cce0
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28322
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9856 mdd: handle NULL buffer in mdd_xattr_list()

The upper layer may call mdd_xattr_list() with a NULL buffer to get
the length of the xattr name list. Handle this case safely by skipping
the removal of the link xattr for unlinked objects.

Lustre-change: https://review.whamcloud.com/28469
Lustre-commit: 33a4b5ef00e88b33136d09d2f4029223a3c4d681

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Iae87fba20325b228ef75ee762acfa49353932b1b
Reviewed-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28766
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9772 utils: Enable new ZFS MMP on mkfs

ZFS 0.7.0 come with new multi-modifier protection, this patch
enables it by default, on mkfs.

This also ensures canmount is off for pools that were not just
created.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: If9b87e9786e0eaefe5ac9a536edcdca3d1012585
Reviewed-on: https://review.whamcloud.com/28051
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit 25e1cea871abd3c08dffb06ea62046ad84a822c1)
Reviewed-on: https://review.whamcloud.com/28652