git://git.whamcloud.com - fs/lustre-release.git/log

LU-10399 test: use /dev/urandom in sanity-hsm test_1b()

In sanity-hsm test_1b() use /dev/urandom instead of /dev/random so we
won't have to wait for the second law of thermodynamics to become
true.

Test-Parameters: trivial testlist=sanity-hsm
Lustre-change: https://review.whamcloud.com/30556
Lustre-commit: 54cf0962e286b8f1bcfca490cf828fd126f1fa60

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ib64691fd6ee1a172c631ffa55956e1d69f24d349
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30722
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-10371 ptlrpc: check for posix_acl a_entries

Since commit 2211d5ba5c6c4e972ba6dbc912b2897425ea6621
posix_acl_xattr_entry a_entries[0] was removed.
Make sure the LASSERTF test works with Kernels
after this commit.

Test-Parameters: trivial
Lustre-change: https://review.whamcloud.com/30495
Lustre-commit: 429ad0e18eccd7261011ee2c7c193b822747e732

Signed-off-by: Thomas Stibor <t.stibor@gsi.de>
Change-Id: If2404c89775a5f9077c7d9379d73c8187b796a3a
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30691
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-618 llite: IO accounting of page read

When CONFIG_TASK_IO_ACCOUNTING is used with Lustre, writes are
accounted for but not read.

The accounting is normally done in the kernel for page writeback
and readahead functionlity, Therefore, as Lustre implements its
own readahead, it must also maintain its own accounting on read
(but not for write)

Lustre-change: https://review.whamcloud.com/1636
Lustre-commit: 48de9b195d71cb57ee50636d75c59d78ee068cee

Change-Id: I19f330be65324a8da002f9d61cb9262345ecb012
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-on: https://review.whamcloud.com/30570
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10377 build: Update ZFS Version to 0.7.5

Update SPL and ZFS version that is built against.

https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.7.5

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: If010d3a7e78b66a2acbd70242fe517218a438c02
Reviewed-on: https://review.whamcloud.com/30523
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10464 kernel: kernel update [SLES12 SP2 4.4.103-92.56]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
mdsdistro=sles12sp2 ossdistro=sles12sp2 \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I0eb5dc783bcbcd4ee8643431345daefb9cb89eb1
Reviewed-on: https://review.whamcloud.com/30750
Tested-by: Jenkins
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10458 kernel: kernel update [SLES12 SP3 4.4.103-6.38]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp3 testgroup=review-ldiskfs \
mdsdistro=sles12sp3 ossdistro=sles12sp3 \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ib7a308dbce58d94c5f5775cd54f33563cf067e75
Reviewed-on: https://review.whamcloud.com/30743
Tested-by: Jenkins
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10456 kernel: kernel update RHEL6.9 [2.6.32-696.18.7.el6]

Update RHEL6.9 kernel to 2.6.32-696.18.7.el6

Test-Parameters: clientdistro=el6.9 mdsdistro=el6.9 \
ossdistro=el6.9 mdtfilesystemtype=ldiskfs \
ostfilesystemtype=ldiskfs testgroup=review-ldiskfs

Change-Id: Ib0b7d5ff7f50f96c8bbc064541ae5c8bf2a4367f
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-on: https://review.whamcloud.com/30737
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Jenkins
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10455 kernel: kernel update RHEL7.4 [3.10.0-693.11.6.el7]

update RHEL 7.4 kernel to 3.10.0-693.11.6.el7

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Id3428aa00e4b1501b642587db7911b6adafd51ef
Reviewed-on: https://review.whamcloud.com/30735
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10427 kernel: kernel update [SLES12 SP2 4.4.103-92.53]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
mdsdistro=sles12sp2 ossdistro=sles12sp2 \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I065ab670ad58fda19610e14440e1c304d0eae74f
Reviewed-on: https://review.whamcloud.com/30652
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10425 kernel: kernel update [SLES12 SP3 4.4.103-6.33]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp3 testgroup=review-ldiskfs \
mdsdistro=sles12sp3 ossdistro=sles12sp3 \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ibdaf6edeb34388d3d9bf39dc7f84639d5fb9992f
Reviewed-on: https://review.whamcloud.com/30648
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Cliff White <cliff.white@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-8999 test: check chown result

For test_38 in sanity-quota, the chown operation should be
checked whether it succeeds or not. the patch also collects
the output of quota users if the test failed.

Lustre-change: https://review.whamcloud.com/30243
Lustre-commit: 817b367707d0d4f5c406b4f18cffceea2aae8406

Change-Id: I6c04f9519f3f097af7126064380faf1bdc4fff6a
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30631
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10232 lov: call cl_object_attr_get under cl_attr lock

cl_object_attr_get() must be called under cl_object_attr_lock
get. There is place in lov_getstripe where it is called
without that lock.

Lustre-change: https://review.whamcloud.com/30052
Lustre-commit: 80515fa15ee76fb0174fd3be80c4a113a8d3c875

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Ia0a2322ba4ff0ff4affb081375cb108fbf2988c4
Reviewed-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30586
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10346 utils: use the correct lib path in mount_utils.c

The backfs_load_module() function of mount_utils.c used an outdated
to link itself against the osd library. This patch fixes it.

Test-Parameters: trivial
Lustre-change: https://review.whamcloud.com/30431
Lustre-commit: 74eb79b0fe4815daa0bebd3293c71f052c5b19d4

Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I90a523b45b5772c05f7520e810eec7eac14b75be
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30584
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9535 tests: check server version in sanityn 77j

The QoS feature in the TBF policy was added to Lustre with
tag 2.9.53 and a patch with commit hash d2c403363f6. Thus,
sanityn test 77j should only be run for servers with version
2.9.53 and later.

Test-Parameters: trivial mdsjob=lustre-b2_9 ossjob=lustre-b2_9 serverbuildno=2 testlist=sanityn

Lustre-change: https://review.whamcloud.com/30385
Lustre-commit: 8552476c5495a94d22501d002a35830181b5ea7d

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ia2a04b5ae688809d4a84a2dc8459598f02932119
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30583
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10267 llog: fix EOF handling in llog_client_next_block()

In llog_client_next_block() update *cur_idx and *cur_offset in the
special case that the handler has returned -EIO after reaching the end
of the log without finding the desired record. This fixes client side
EOF detection in llog_process_thread().

Lustre-change: https://review.whamcloud.com/30313
Lustre-commit: a485c51379a0218aaf01f7c0baf8e4cc993d8824

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: If30ebec065ddd38fb7b06c3e16f96bb3cd76fa1b
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30582
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-10268 lfsck: postpone lfsck start until initialized

Sometimes, the LFSCK start request may comes (from remote server)
before local target initialized. If we start the LFSCK right away
on current server, the LFSCK engine may access NULL pointer, such
as lookup FID with NULL 'ss_server_fld'.

To avoid such trouble, start LFSCK logic will return -EINPROGRESS
to the request sponsor. It is the sponsor duty to retry the start
request some time later.

This is a port to b2_10 of
Lustre-change: https://review.whamcloud.com/#/c/30259/
Lustre-commit: 6ec4b7d3fb7351f699569e1a8f5ad2cfa7c78df9

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: If7bc44e025b5f3c4f977b3a35e3784ada548a2df
Reviewed-on: https://review.whamcloud.com/30421
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10134 lfsck: not add requests if engine out of work

There is race condition between LFSCK assistant engine and
LFSCK request generators: before the LFSCK assistant engine
exit, it will mark itself as 'stopping', then cleanup the
in-queue requests, and then mark itself as 'stopped'. It is
expected that the 'stopping' status will prevent generators
adding more LFSCK requests. But current implementation only
checks 'stopped' or not. So if the LFSCK engine thread exit
before the whole system scanned that may because of some
failures or on demand, more LFSCK requests may be added in
the queue after the cleanup.

The patch fixes the wrong logic by checking 'running' or not,
and stop adding more LFSCK requests if not 'running'.

This is a port to b2_10 of
Lustre-change: https://review.whamcloud.com/#/c/30165/
Lustre-commit: 159bde49676b9a49eb3e4cb57c8c3a58a49ba0af

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ic2b5ca3f5e80b5be5a5c60aa24f0b54682b717d9
Reviewed-on: https://review.whamcloud.com/30375
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-5020 llite: don't reconnect MGC if connecting

1. In mgc_set_info_async(KEY=KEY_INIT_RECOV_BACKUP "init_recov_bk"),
   the MGC import should be reconnected only if its state is
   LUSTRE_IMP_DISCON

2. in mgc_target_register, if the target will regenerate the config,
   we should use some longer delay limit to wait the MGC to connect
   to MGS for the target (server) will fail to exit if the request
   expired due to delay limit.

3. In case of parallel mount, the async cleanup of OSS will affects
   the following mount for the OSS can't be setup again, then there
   should be some barrier to sync with the OSS cleanup.

Lustre-change: https://review.whamcloud.com/10229
Lustre-commit: 1a7ff02c1fbb8e85ac2e8fa458ba3fb810a76ea4

Change-Id: I805b84cf12100ec2cc68f95bb65a9c396e0fbc1b
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30581
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-4134 obdclass: fix double free in failure path

We should just decref for obd if register is failed.
Since class_export_put will invoke class_free_dev,
then we may face a double free.

Lustre-change: https://review.whamcloud.com/29967
Lustre-commit: 77d2e604b6bc3c319adefca069b5182c325a43b0

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Ia8d1e487c69c4de1c7c247158cc8615aa6b6093a
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30487
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-4134 obdclass: obd_device improvement

The patch removes self exports from obd's reference counting which
allows to avoid freeing of self exports by zombie thread.
A pair of functions class_register_device()/class_unregister_device()
is to make sure that an obd can not be referenced again once its
refcount reached 0. For target_handle_connect() take a reference for
obd_device during finding it by name.

Fix grant mismatch message
"tot_granted 4194304 != fo_tot_granted".

Lustre-change: https://review.whamcloud.com/8045
Lustre-commit: 45900a7777ac02130d8bf65724c4b6cffca9d546

Signed-off-by: Vladimir Saveliev <vladimir.saveliev@seagate.com>
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@seagate.com>
Signed-off-by: Alexander Boyko <alexander.boyko@seagate.com>
Seagate-bug-id: MRP-2139 MRP-3267
Change-Id: I9cc6860431c6bb7db6983e0d15a5d3d2b564265e
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29740
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10301 kernel: kernel update RHEL7.4 [3.10.0-693.11.1.el7]

update RHEL 7.4 kernel to 3.10.0-693.11.1.el7

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Iea7a4604c333c26fdc1b2439e6a00f27472d2410
Reviewed-on: https://review.whamcloud.com/30441
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9558 llite: port lustre to unified handling of bdi

For the linux 4.12 kernel the bdi handling was unified for all
file systems. This was done by allocating struct backing_dev_info
separately instead of embedding it inside superblock. For older
kernels we move all the bdi handling lustre does to the function
super_setup_bdi_name() which is what exist in the latest kernels.

Linux-commit: 9594caf216dc0fe3e318b34af0127276db661241

Lustre-change: https://review.whamcloud.com/28511
Lustre-commit: 1defa62a19e71c021ec2cf9fc9e0481e9ab962e0

Change-Id: I5af60ea3661e3d3a97973fd99a79c28dcd1ce1cc
Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/30575
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-7340 mdd: changelogs garbage collection

When changelogs are almost full (few number of
free entries in catalog), try to recover some space
by unregistering users that are idle since too long,
based on new Changelog User record field (in fact,
using previous cur_padding unused field) to keep
track of last user's changelog cancel request time,
or based on gap between user index and current
ChangeLog record, for older registered user.
sanity/test_160[f,g] have been added to verify feature.

Lustre-change: https://review.whamcloud.com/27103
Lustre-commit: 3442db6faf685fbdbd092bdfdc8d273e4990a141

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I2100b101772e6d027675e5efa5606d4be24342a0
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30573
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9558 kernel: replace CURRENT_TIME macro

CURRENT_TIME macro is not y2038 safe on 32 bit systems.

The patch replaces all the uses of CURRENT_TIME by current_time() for
filesystem times, and ktime_get_* functions for others.

struct timespec is also not y2038 safe. Retain timespec for timestamp
representation here as lustre uses it internally everywhere. These
references will be changed to use struct timespec64 in a separate patch.

This is also in preparation for the patch that transitions vfs
timestamps to use 64 bit time and hence make them y2038 safe.
current_time() is also planned to be transitioned to y2038 safe behavior
along with this change.

CURRENT_TIME macro will be deleted before merging the aforementioned
change.

Linux-commit: 47f38c539e9a42344ff5a664942075bd4df93876

Lustre-change: https://review.whamcloud.com/28305
Lustre-commit: 213db47f6497c0ea49ffef19de0d4673c932eefd

Change-Id: Iec67df8f48066801c227aba2bc9221d444dada65
Link: http://lkml.kernel.org/r/1491613030-11599-10-git-send-email-deepa.kernel@gmail.com
Signed-off-by: Deepa Dinamani <deepa.kernel@gmail.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30572
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-7991 quota: project quota against ZFS backend

This patch enables the project quota feature against ZFS
backend. The project quota patch for ZFS itself will be
landed via https://github.com/zfsonlinux/zfs/pull/6290.
Such patch may be released via zfs-0.7.x (x >= 5).

This patch also cleanup the quota logic for ZFS backend
and sanity-quota.sh

Lustre-change: https://review.whamcloud.com/27093
Lustre-commit: a046e879fcadd601c9a19fd906f82ecbd2d4efd5

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ic488efe19a6e34a3615811f79fb9ea7954ad9263
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/30540
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-7988 hsm: wake up cdt when requests are empty

The coordinator only runs once per second, we need a mechanism
to send more work when everything is done (cdt_request_count
goes to zero)

Without this, there is a hard limit of max_requests per sec
requests that can be processed, causing performance issues
with small files.

Lustre-change: https://review.whamcloud.com/29742
Lustre-commit: 7251fea8dc3c4d29e30c5a3f763c4c33d35f90a7

Signed-off-by: Ben Evans <bevans@cray.com>
Change-Id: I563666a1a3e53f0ec5908de593de71ff4d925467
Reviewed-by: Frank Zago <fzago@cray.com>
Reviewed-by: Sergey Cheremencev <cherementsev@gmail.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30538
Tested-by: Jenkins
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10256 hsm: enable setting archive_id in hsm_set

Setting HSM flags in lfs_hsm_change_flags(...,LFS_HSM_SET)
does not allow to specify the archive_id, that is,
in llapi_hsm_state_set(path, mask, 0, 0 /* archive_id */)
archive_id = 0 is always set, which means no identifier change.
For having full flexibility (e.g. for debugging), introduce
the additional option --archive-id in hsm_set. If the option
is not provided, the default behavior (archive_id = 0) is used
and no archive identifier change is done. In addition a
test case is provided to check the functionality and shell
function get_hsm_archive_id() is modified in favor of more
robust grep for contents after pattern approach.

Lustre-change: https://review.whamcloud.com/30150
Lustre-commit: d1855f8e22a929066a69470c7e3d082c70478575

Test-Parameters: trivial testlist=sanity-hsm

Signed-off-by: Thomas Stibor <t.stibor@gsi.de>
Change-Id: I2145a18ecf32479527bb045140e5e881e58dd115
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30500
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-6210 utils: Use C99 struct initializers in lfs_hsm_state()

This patch makes no functional changes. Struct initializers that
use C89 or GCC-only syntax are updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

This patch updates lfs_hsm_state() to use the C99 syntax.

Test-Parameters: trivial
Lustre-change: https://review.whamcloud.com/27525
Lustre-commit: 73c2c103f4f43f9bb37119a2e90d6c0fc1870711

Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I32f427396e99f1b86d329eb8457883a9aaaed181
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30499
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9611 lov: allow lov.*.stripe{size,count}=-1 param

Since LU-7344 patch http://review.whamcloud.com/16930 was landed,
lov_stripeoffset_seq_write() and lov_stripecount_seq_write() have
incorrectly checked that lov.*.stripecount and lov.*.stripeoffset
are not negative.  In fact they can both be "-1" to indicate that
the filesystem-wide default value should be used. These parameters
can also be set internally if using "lfs setstripe -c -1 $MOUNT"
or "lfs setstripe -i -1 $MOUNT" to set the system wide default,
generating console errors on the MDS from class_process_proc_param():

    lov.: error writing proc entry 'stripecount': rc = -34
    lov.: error writing proc entry 'stripeoffset': rc = -34

Fix these functions to allow "-1" as a valid value.

Lustre-change: https://review.whamcloud.com/27946
Lustre-commit: f93276d9a5b5df475c9f6007265692632d8e7222

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I295d2591d535b039634689524a29725e963ebbe5
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30407
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9474 tests: add the stack_trap() utility function

This patch implements the stack_trap() function and uses it in
test_24d of sanity-hsm (where the need for it first arose).

The stack_trap() function is meant to be used to register multiple
cleanup actions on a single sigspec (most likely EXIT). Over time,
any call to bash's trap builtin in tests should be replaced by a call
to stack_trap().

Note that cleanup functions do not need to start with "trap - EXIT".
Each test is run in a subshell of its own (in run_one_logged()) and
therefore traps do not leak from one test to another. This is true
with or without this patch.

This is a port to b2_10 of
Lustre change: https://review.whamcloud.com/#/c/29653/
Lustre commit: 1dfda720c080e39890e5a659a30c8988ca498a7c

Test-Parameters: trivial testlist=sanity-hsm,sanity-hsm
Change-Id: I47f92a963f094d3d7055592a748e81ce357bff75
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-on: https://review.whamcloud.com/29653
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dominique Martinet <dominique.martinet@cea.fr>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
(cherry picked from commit 1dfda720c080e39890e5a659a30c8988ca498a7c)
Reviewed-on: https://review.whamcloud.com/30371
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10330 tests: fix version check in sanity.sh test_101g

The 16MB RPC patch is available in 2.7.x, so one more version
check needs to be added to 2.7.17 or later to avoid interop
failure.

Test-Parameters: trivial

Change-Id: I6eeefbf74017aaeeb8998accd6da04d8de75606c
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-on: https://review.whamcloud.com/30365
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9872 tests: modify check space requirements for NFS test

In parallel-scale-nfs testing, the Lustre clients NFS mounts
the Lustre file system. In some cases, the file system looks
like it has enough space (du) to run NFS testing, but one or
more OSTs is full or is almost full. Since the default striping
for Lustre is a single stripe, some NFS tests will fail because
the file it is writing gets written to that almost full OST.

Increase the default stripe to all OSTs for all
parallel-scale-nfs tests.

Lustre-change: https://review.whamcloud.com/29786
Lustre-commit: 0b8e9558e88930814857c97dfe2394f8c8e24a9a

Test-Parameters: trivial testlist=parallel-scale,parallel-scale-nfsv3,parallel-scale-nfsv4
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Iff187dc6c158d9063cd36ced0801b88cdc8814ae
Reviewed-by: James Casper <jamesx.casper@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30353
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9509 tests: fix compilebench timeout

With ext4 and lustre found compilebench always takes more time in
create dir operation whenever previous same test aborted without
deleting directory. As per test board result, found compilebench
test ran from suite parallel-scale-nfs.sh timedout before running
from suite parallel-scale.sh, so 1st timedout compilebench not
clear data, rerunning compilebench again causes timeout.

Fix is creating different directory on each run of compilebench.

Lustre-change: https://review.whamcloud.com/27131
Lustre-commit: d15fe1fbb261107b021cd6ebe8b8c435d2148e0f

Seagate-bug-id: MRP-3753
Signed-off-by: Jadhav Vikram <jadhav.vikram@seagate.com>
Reviewed-by: Alexander Lezhoev <alexander.lezhoev@seagate.com>
Reviewed-by: Ashish Purkar <ashish.purkar@seagate.com>
Change-Id: I9b47e28fe5c638f4167c05fb68e46ec5e90de09e
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30352
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9997 ldlm: check lock cancellation in ldlm_cli_cancel()

In that case, the assert for 'list_empty(&lock->l_bl_ast)' will fail
because the lock is already in a cancel list.

This patch checks if the lock is already being canceled in prior.

Lustre-change: https://review.whamcloud.com/29080
Lustre-commit: f4ea4017b4d6a420146b2959f9feb11866858b4a

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I04b9df789cf3b3b5e6dd61a647376307f9975f4a
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30339
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9894 lfsck: Print correct lfsck start info

When printing which scan type was started, lfsck iterates
through an array of types and compares the given type to
the types in the array. Type numbers are a bit mask.

However, LFSCK_TYPES_SUPPORTED and LFSCK_TYPES_DEF are the
full mask, and they are the first in the array. This means
we always match LFSCK_TYPES_SUPPORTED, which results in
always printing "all" as the type of scan.

Reorder the struct to increasing bit order to fix this.

Test-Parameters: trivial testlist=sanity-scrub,sanity-lfsck
Lustre-change: https://review.whamcloud.com/28604
Lustre-commit: 0c6805b2a0d575331dff0c22a657da91f61a1809

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: Ibc82369ff6531b56ceda4b1c0a833720dc61f3f4
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30265
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-6210 utils: Use C99 initializer for lfsck_types_names

This patch makes no functional changes. The lfsck_types_names
struct initializer in lustre_lfsck.c is updated to C99 syntax.

C89 positional initializers require values to be placed in the
correct order. This will cause errors if the fields of the struct
definition are reordered or fields are added or removed. C99 named
initializers avoid this problem, and also automatically clear any
values that are not explicitly set.

Test-Parameters: trivial
Lustre-change: https://review.whamcloud.com/27791
Lustre-commit: a1d62378ed98e2663c1f95f955433e2a5b13d68c

Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I8a3b44b56263b00a926550e4122193300c0c399f
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30264
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9860 tests: Add conf-sanity tests to ALWAYS_EXCEPT list

The following tests fail when run with a separate MDS and MGS:
conf-sanity tests 33a, 43b, 53b, 54b, 70e, 80, 84, 87, 100,
102, 103, 104, 105 and 107.
We need to add these tests to the ALWAYS_EXCEPT list
when running with a separate MDS and MGS.

Test-Parameters: trivial combinedmdsmgs=false testlist=conf-sanity envdefinitions=SLOW=yes
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I1b17714216e14ad04eb9a492cb5f1aa4ed82bd1a
Reviewed-on: https://review.whamcloud.com/28497
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Dilip Krishnagiri <dilipx.krishnagiri@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit 5297bfa671bd2cbb18fb3188c4d9e3b8edf3284a)
Reviewed-on: https://review.whamcloud.com/30216
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9899 tests: mount client on MGS for ost-pools

When a Lustre configuration has the MGS and MDS on separate
nodes, the file system must be mounted on the MGS to allow
OST pools to work properly.

Add the ability to mount the file system on the MGS when
necessary for the Lustre test suite ost-pools.sh.

Test-Parameters: trivial testlist=ost-pools
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Iff0663a38b92bb8e71c313897b12fca98fdae932
Reviewed-on: https://review.whamcloud.com/28638
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Arshad Hussain <arshad.hussain@seagate.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit bb4c3bc38eeeeec111c14c024669465addb30c70)
Reviewed-on: https://review.whamcloud.com/30215
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10255 tests: Fix wrong comparison operator in sanity 129

Currently sanity 129 uses numerica (-eq) operator to check whether
the has_warning variable is true or false depending on whether
expected warnings have been detected. This is wrong since bash
doesn't really have bolean types and true/false are not evaluated to
1/0. This results in the following bash error when the test is run:

./sanity.sh: line 9244: [: true: integer expression expected

Fix it by using the correct string operator '=' to check for
correct value.

Test-Paramters: clientdistro=sles12sp2 testlist=sanity
Test-Paramters: clientdistro=sles12sp3 testlist=sanity

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Change-Id: I071dcb3001ea9b2e5806d346dcd48bfcaf3297e9
Reviewed-on: https://review.whamcloud.com/30151
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Jenkins
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10217 test: Make sanity 248 work with newer dds

This patch fixes 2 problems when working with value from dd.

Firstly, extraction of the readings from dd rely on them always
being in field number 6. However, that's not true for newer dd
versions where the output has changed and the time reading is now in
field number 10. So to make the code more generic and work with both
old and new outputs switch to using an egrep regular expression to
extract the correct field.

Secondly, dd can output its time in scientific notation, with
decimal and floating point portion being separated by a comma rather
than a dot:
4096 bytes (4,1 kB, 4,0 KiB) copied, 9,1229e-05 s, 44,9 MB/s

Unforuntately this is not understood by bc so it needs to be
processed by substituting the comma for a dot and also expanding
the scientific notation to multiplication of 10 to the correct power.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Change-Id: I83ec659874ffe702c68f94f828827ff93325db72
Reviewed-on: https://review.whamcloud.com/30144
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9660 ptlrpc: do not wakeup every second

Even if there are no RPC requests on the set, there is no need to
wake up every second. The thread is woken up when a request is added
to the set or when the STOP bit is set, so it is sufficient to only
wake up when there are requests on the set to worry about.

Lustre-change: https://review.whamcloud.com/28776
Lustre-commit: e81847bd06519dd9847c31244abe6da978bcd016
Change-Id: Iac01d8c46e8645ecb6303ce72e0f6c59f16dcd5d
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29969
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10011 utils: suppress annoying messages for project quota

See following output:

[wsl@w003 ~]$ lfs quota /lustre1
Disk quotas for usr wsl (uid 14434):
      Filesystem  kbytes   quota   limit   grace   files   quota limit
grace
        /lustre1       0       0       0       -       0 0       0       -
Disk quotas for grp se (gid 1000):
      Filesystem  kbytes   quota   limit   grace   files   quota limit
grace
        /lustre1   44028       0       0       -      40 0       0       -
Unexpected quotactl error: Operation not supported
Disk quotas for prj <unknown> (pid 0):
      Filesystem  kbytes   quota   limit   grace   files   quota limit
grace
        /lustre1     [0]     [0]     [0]       -     [0]     [0] [0]       -
Some errors happened when getting quota info. Some devices may be not
working or deactivated. The data in "[]" is inaccurate.

Since project quota is disabled in default, unsupported project
quota messages will be annoying here.

Even Project quota is enabled, project id 0 will be outputed here,
it dose not make much sense either, instead, lfs quota <mnt> will only
output current user/group quota information in default

Lustre-change: https://review.whamcloud.com/29107
Lustre-commit: 0ba70b538768871f8504aca1ed6a24472f46a87d

Change-Id: Ie76ba14c4c4486a9246aafed0e8538eaae85ee98
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29600
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10098 scripts: Fix mounted check in Lustre RA

The "Lustre" resource agent for pacemaker can mis-identify a resource
as being mounted if it's mountpoint is a substring match for anything
else in /proc/mounts. Change the lustre_is_mounted() function to
check to make sure it's a lustre fs mounted at mountpoint and the
"source" (i.e. device) is the target we expect.

Lustre-change: https://review.whamcloud.com/29351
Lustre-commit: df96fab178534adb34067b20490a2d0629aabe2f

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ib877b0dc3d3ce0d93fd4663aa2418ac21d670428
Reviewed-by: Malcolm Cowe <malcolm.j.cowe@intel.com>
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29352
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

New release 2.10.2.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I495068f728dfe75e82f2498a3fbdf9e15f06c58c

New tag 2.10.2-RC2.

Second release candidate for 2.10.2 release.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I0ced71f744d66a8abc3ce7e51a61fbc45dc6c6e7

LU-10220 mdd: fix buf alloc in mdd_changelog_data_store_by_fid

Fix allocation of mti_big_buf by call to lu_buf_check_and_alloc()
in mdd_changelog_data_store_by_fid().
reclen must take the header size of struct llog_changelog_rec into
account.

Maybe no memory corruptions were seen before because the buffer size
allocated in a previous call to mdd_declare_changelog_store() was
covering the need. But audit will add more information in changelog
records, provoking memory corruptions without this fix.

Lustre-change: https://review.whamcloud.com/30014
Lustre-commit: f173f93032c62bd1f95330f331d3d93ce5a31598

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Id0a06c412b54c0ae12c15d53f3e166e3e5d9ed68
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30356
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-8497 osp: handle remote -ENOMEM from osp_send_update_req()

In osp_send_update_req() detect an unsent request by checking
rq_queued_time == 0 rather than rq_set == NULL, which is always true
after returning from ptlrpc_queue_wait().

Lustre-change: https://review.whamcloud.com/30083
Lustre-commit: cef8983c8b5bf51b58df23a779769cc4b8ca8db5

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ief959b71600157a9c3521775cc06994326e50c51
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30354
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9941 lov: lsm_is_composite isn't right

LOVEA magic containing LOV_MAGIC_MAGIC will also be regarded as
a composite magic.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I3ef37ee80364b2a8f27831e3c53fb88b464f2039
Reviewed-on: https://review.whamcloud.com/28845
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit 385518f67eb917335de0c1d727941094f4ab5874)
Reviewed-on: https://review.whamcloud.com/30342
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>

New tag 2.10.2-RC1.

First release candidate for 2.10.2 release.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I207f36c4e6815dfb06ba7b9622e939ade167eb89

LU-10234 tests: use wait correctly in sanity-hsm test_3[4-6]

Test 34, 35 and 36 did not properly check the return code of wait
and potentially missed regressions.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I21e8b16b55b946a8e5837549c3a3698379132e48
Reviewed-on: https://review.whamcloud.com/30102
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dominique Martinet <dominique.martinet@cea.fr>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit eec4b8985f5eaacd13f0e0db7328dcdd610655bc)
Reviewed-on: https://review.whamcloud.com/30208
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10086 libcfs: use dynamic minors for /dev/{lnet,obd}

Request dynamic minor allocation when registering /dev/lnet and
/dev/obd. Remove the obsolete create-device-if-not-found code from
register_ioc_dev().

Lustre-change: https://review.whamcloud.com/29741
Lustre-commit: e446c1667dc9136e3a1cb3603c3851975b68e0ac

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I59c70912b4729f58a76dc6107b3e1d7379c6d7a3
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29945
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-10241 kernel: kernel update RHEL6.9 [2.6.32-696.16.1.el6]

Update RHEL6.9 kernel to 2.6.32-696.16.1.el6

Test-Parameters: clientdistro=el6.9 mdsdistro=el6.9 \
ossdistro=el6.9 mdtfilesystemtype=ldiskfs \
ostfilesystemtype=ldiskfs testgroup=review-ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ib2215b56e647fb8e19bc8b7e344a94564c7a8c7f
Reviewed-on: https://review.whamcloud.com/30131
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10166 mdc: invalid free in changelog reader

Use kthread_stop() to instruct the producer thread to exit
when the device is closed, and only then: release the CRS
structure.

The previous implementation left small time windows open,
during which the producer threads could free a structure
before the consumer thread was completely done with it.

Change-Id: Id2038aa9b7fcfd2c3347f628e749f9d2c265ac6e
Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-on: https://review.whamcloud.com/29818
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit 89e52326b5bd3d2716ce6ec5d9f9d787947d91a1)
Reviewed-on: https://review.whamcloud.com/30207

LU-10040 nodemap: add nodemap idmap correctly

There are 3 situations when adding an idmap {id_client:id_fs} to
a nodemap tree:
- both id_client and id_fs are matched, that means this idmap
   already exists, so return -EEXIT;
- neither id_client nor id_fs is matched, that means this is a
   new idmap, so insert it;
- only "id_client" or "id_fs" is matched, since idmap uses
   "id_client" as index key, we need to delete that old idmap and
   its index by its id_client, and then insert it again. In the
   original implementation, idmap_insert() calls rb_replace_node()
   to replace the old idmap without re-sorting its both id_client
   and id_fs, so that this new added idmap can't be found by
   idmap_search() due to its wrong left and right nodes, and can't
   be deleted either.

Also, this patch improves the following code:
- nm_idmap_lock: use type rw_semaphore instead of rwlock, because
   nodemap_idx_idmap_del() may sleep on FS operations while this
   lock is being held.
- Add "update_idmaps" to sanity-sec.sh test_15 to verify this fix,
   and improve other part of this test case as well to make it run
   correctly.

Test-Parameters: testlist=sanity-sec
Signed-off-by: Stephan Thiell <sthiell@stanford.edu>
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Icf777f14c2e1dd56fa5cd0eb56666240e206d199
Reviewed-on: https://review.whamcloud.com/29364
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
(cherry picked from commit 253ccbd55ffe7fcdc405c9fcc4f72a47578920fe)
Reviewed-on: https://review.whamcloud.com/30206

LU-10041 tests: sanity/51 to cleanup properly

sanity/51 should cleanup properly in case of failure

Test-Parameters: trivial testlist=sanity ostfilesystemtype=zfs mdtfilesystemtype=zfs
Test-Parameters: trivial testlist=sanity mdscount=2 mdtcount=4

Lustre-change: https://review.whamcloud.com/29246
Lustre-commit: 0d5ceb891e904cc9bab3a6b5ce0944b8bab06975

Change-Id: Ib1c0e1a116a6802f2bf25d665be5c076ae07c8b8
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/30023
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10204 tests: Fix line continuation

There is a simple missing in one of the error paths in
ladvise_willread_performance which results in an unknown command
error. Fix it by adding the missing backslash.

Test-Parameters: trivial
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Change-Id: Ifff854c6cf5c1add71fd1571cb4de01d56204fde
Reviewed-on: https://review.whamcloud.com/29982
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-10007 pacemaker: Use lctl and load lustre

When scripts are started, load lustre module.
Use lctl instead of directly accessing health_check file.

Test-Parameters: trivial
Lustre-change: https://review.whamcloud.com/29144
Lustre-commit: 6e299c2a3bd7566a322cf9bc9494f6850d2c1b7e

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I4a81248939464e498006dc2c4072d44685add018
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29946
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-10015 o2iblnd: fix race at kiblnd_connect_peer

cmid will be destroyed at OFED if kiblnd_cm_callback return error.
if error happen before the end of kiblnd_connect_peer, it will touch
destroyed cmid and fail as
(o2iblnd_cb.c:1315:kiblnd_connect_peer())
ASSERTION( cmid->device != ((void *)0) ) failed:

Lustre-change: https://review.whamcloud.com/29134
Lustre-commit: 576551cb97c6a5efb27d8afc59ad60550dde51ce

Seagate-bug-id: MRP-4592
Signed-off-by: Alexander Boyko <alexander.boyko@seagate.com>
Change-Id: I83eb5bceeb567acef0316498b936d25d6c6ccd95
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29881
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9951 lustre_compat: add wrapper function for posix_acl_update_mode

posix_acl_update_mode is introduced in kernel 4.9, add the precheck
of it, if not exists, use inline wrapper function instead.

Lustre-change: https://review.whamcloud.com/28871
Lutre-commit: ec6bdd55bb2128dcf9779017ef9623461afe05f8

Change-Id: I8a1476d611c387a88efef5d5b8707edf5feacca8
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29880
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-10047 tests: stop skipping test_102 subtests

Remove obsolete version check on tar.
All tar versions in supported distros are now capable of --xattrs.
But not all tar version have --xattrs-include, so must test for and
adapt for that.

Lustre-commit: 036641a1e1f6d5d010873267b669528890f19fe4
Lustre-change: https://review.whamcloud.com/29288

Test-Parameters: trivial envdefinitions=ONLY=102 testlist=sanity
Test-Parameters: trivial clientdistro=el6.9 envdefinitions=ONLY=102 testlist=sanity
Test-Parameters: trivial clientdistro=sles12sp2 envdefinitions=ONLY=102 testlist=sanity

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I5e76bd1a762c4e01cf8a3a33789ca3a30c15abb0
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-on: https://review.whamcloud.com/29823
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9663 tests: replace “ls -s” with “stat -c '%s'” to get file size

This patch improves acl/2561{,_zfs}.test to use “stat -c '%s'” to
get file size instead of using “ls -s”, which prints the allocated
size of the file in blocks.

Test-Parameters: trivial \
envdefinitions=ONLY=103 \
mdtfilesystemtype=zfs ostfilesystemtype=zfs \
testlist=sanity

Lustre-change: https://review.whamcloud.com/27961
Lustre-commit: 45b8c6cf6916cf012ad0512f9cf249c152b45236

Change-Id: I8bfea5197102baeece244e20731c90172c5cc2f1
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29816
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9752 man: Reference zgenhostid instead of genhostid

In ZFS 0.7.0, they added zgenhostid(8) to be used in place of
Redhat's genhostid, so that there would be a platform agnostic
way to generate /etc/hostid.

Test-Parameters: trivial
Lustre-change: https://review.whamcloud.com/29327
Lustre-commit: a1eb6de081473545fbd5c1fe33e209fe391bf708

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I691266d04f91d5fa7c50b72948c801afa69d647d
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29805
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9578 llite: use security context if it's enabled in the kernel

if it's disabled, then Lustre stop to work properly (can not create
files, etc)

Lustre-change: https://review.whamcloud.com/27364
Lustre-commmit: 61f26ea47f96632a90702f2b404e845a01498612

Change-Id: I1e431ec95a2b0613b43893567eb6d1a64ec832de
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29737
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10141 llapi: integer overflow in llapi_changelog_start

Use the appropriate type to store and check the return value from lseek.
This prevents from misinterpreting high offsets as errors.

Lustre-change: https://review.whamcloud.com/29674
Lustre-commit: aaa808e7a40198581956a1d648f413cc30e830ed

Change-Id: I15e92be3454af20ee6611c2a7ddfc1b597d639c2
Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Thomas LEIBOVICI <thomas.leibovici@cea.fr>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29735
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-10150 build: Build with ZFS 0.7.3

Update ZFS and SPL version to 0.7.3
Also Fixup ChangeLog

Changelog: https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.7.3

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I88f0f2601262c0c535431d256a159d9e9d6ebd75
Reviewed-on: https://review.whamcloud.com/29713
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9840 lod: add ldo_dir_stripe_loaded

Add ldo_dir_stripe_loaded flag to avoid loading
stripes mulitple times especcially for non-stripe
directory.

Lustre-change: https://review.whamcloud.com/28962
Lustre-commit: 744fe412fa101609a38f7ccc77efc4f1c540e008

Change-Id: Ia9360aac9e24706e401184c75fae4ec7f8ec46d9
Signed-off-by: Di Wang <di.wang@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-on: https://review.whamcloud.com/29143
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10168 kernel: kernel update [SLES12 SP2 4.4.90-92.45]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
mdsdistro=sles12sp2 ossdistro=sles12sp2 \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: If24eddeeadc825b1b3073439304c0958c0ae1b74
Reviewed-on: https://review.whamcloud.com/29840
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10132 llite: handle xattr cache refill race

In ll_xattr_cache_refill() if the xattr cache was invalid (and no
request was sent) then return -EAGAIN so that ll_getxattr_common()
caller will fetch the xattr from the MDT.

Lustre-change: https://review.whamcloud.com/#/c/29654/
Lustre-commit: 9cc8c3a20c547ec75325dde3dd17f4b1dcc66348

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ia9ec7424e8786d92bdecf4897fafcf71d5061fb1
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29795
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-10164 kernel: kernel update [SLES12 SP3 4.4.92-6.18]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp3 testgroup=review-ldiskfs \
mdsdistro=sles12sp3 ossdistro=sles12sp3 \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I4b046f0ac64b75545cd0e2a3a44e9117f4d79ef4
Reviewed-on: https://review.whamcloud.com/29825
Tested-by: Jenkins
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10142 kernel: kernel update RHEL7.4 [3.10.0-693.5.2.el7]

update RHEL 7.4 kernel to 3.10.0-693.5.2.el7

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I6f22d9d96b86d7bd860a400cd803fad26a6b5d1a
Reviewed-on: https://review.whamcloud.com/29728
Tested-by: Jenkins
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10131 llite: Update i_nlink on unlink

Currently, the client inode link count is not updated on
last unlink.  This is fine because the dentries are all
gone and the inode is eligible for reclaim, but it's still
incorrect.  This causes two problems:

1. Inode is not immediately reclaimed
2. i_nlink count is > 0 for a fully unlinked file, which
confuses wrapfs

On last unlink, the MDT sends back attributes.  Use the
nlink count from these to update the client inode.

Remove null check inherited from ll_get_child_fid, because
the inode should never be null on an unlink.

Re-enabled test 76, which passes with this patch.
Removed slab allocator tuning from test_76, because slab is
no longer the default Linux allocator.

Lustre-change: https://review.whamcloud.com/29651
Lustre-commit: 947b22b08d3725852a811a9e330562dd986d7336

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: Ib253b5cf3d35188554cf8fc33a8a3d4b8bb237e8
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29785
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10041 osd: osd-zfs to choose dnode size

depending on dnodesize property it can be:
legacy (512 bytes), auto (512 bytes to 16K) or absolute
size (512, 1024, 2048, 4096, 8192, 16384).

Lustre-change: https://review.whamcloud.com/29242
Lustre-commit: d993de7b40cd9625b48e5361ae29f77bfceb207a

Change-Id: Iea35d8ae850523440272467320410850821f484c
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29770
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9782 osd-ldiskfs: avoid extra search

Extent tree grow greatly durin random IO test with small block size.
osd_is_mapped responsible to large cpu consumption in this case.

|          |
|          |--94.49%-- ldiskfs_es_find_delayed_extent_range
|          |          ldiskfs_fiemap
|          |          osd_is_mapped
|          |          osd_declare_write_commit
|          |
|          |--5.49%-- ldiskfs_fiemap
|          |          osd_is_mapped
|          |          osd_declare_write_commit
|
|--21.80%-- ldiskfs_es_find_delayed_extent_range
|          |
|          |--100.00%-- ldiskfs_fiemap
|          |          osd_is_mapped

let's cache a osd_is_mapped result to avoid extra search in extent
tree,

Lustre-change: https://review.whamcloud.com/28145
Lustre-commit: 1eb0573fdeb7ef8ed667974fd30d4d110b8166c9

Seagate-bug-id: MRP-4474
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@seagate.com>
Change-Id: I63d480bfc7c6b7599b80ceeec9447b227a1610c8
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29739
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-7990 llite: increase whole-file readahead to RPC size

Increase the default whole-file readahead limit to match the current
RPC size. That ensures that files smaller than the RPC size will be
read in a single round-trip instead of sending multiple smaller RPCs.

Lustre-change: https://review.whamcloud.com/26955
Lustre-commit: 627d0133d9d7bef908544537a8a2b2093719fc72

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I3bdb1c7f92c546d58951a9e6b783af23c83ebbe5
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29738
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-4705 mdc: improve mdc_enqueue() error message

Include the parent/child FIDs and name in the mdc_enqueue()
debug message.

Test-Parameters: trivial

Lustre-change: https://review.whamcloud.com/28978
Lustre-commit: 471c5303eb29d5ea1ba5a683173bda63095dae78

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I7b84921a52a4650be70fe87eea691ba2217bb3a6
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29736
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9814 ldiskfs: restore simple_strtol in prealloc

Since kstrtol needs a null-terminor string so we
back to use simple_strtol in prealloc patches.

Lustre-change: https://review.whamcloud.com/28553
Lustre-commit: 685ef61f0fb9849e3f7a32c4ebf2980d3300afb0

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I331308d830fbeef9c00156bb8c14b43651d66420
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29733
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10029 osd-ldiskfs: make project inherit attr removeable

Inherit attribute should be clearable now

Lustre-change: https://review.whamcloud.com/29189
Lustre-commit: 9d068048603fefbab1eea8d8bb5a246ce4375dca

Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I878fde0dc134c9820436ee80979d87e6dacfb70d
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29688
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9140 nrs: measure the runtime of dd directly

This patch changes the way to measure the runtime of "dd". Instead
of parsing the output of "dd", we use date command to calculate
the runtime of dd directly, avoiding the parsing failure caused
by changed output format of "dd".

Lustre-change: https://review.whamcloud.com/27878
Lustre-commit: 66abf6ffe9f032aaa7c271ee2bd1f9b569058bd5

Change-Id: Ibd2e3963f791404ee927981238227012cf4dbf2c
Test-Parameters: trivial testlist=sanityn
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29686
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9260 test: Use the correct mount device when test against lustre

The changes pass the MGSNID:/FSNAME into test, instead of
using the default loop device when testing against lustre.
The corresponding changes to the Posix test suites are also needed
to make the testing pass. Related changes apply to toolkit.

Test-Parameters: trivial testlist=posix

Lustre-change: https://review.whamcloud.com/28661
Lustre-commit: 7b59ed3ab3c9bfae95b9904982869d31a7e65770

Change-Id: I32fc5a401fdc53ed133a78dc4c84b4a7e2a5ad19
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29628
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9888 tests: Update disk2_7-zfs.tar.bz2 for quota

Add a blimit file with the larger 40960 limit in it, as this seems to
be how the image was created, but the default (without this file) is
20K.
Also add an ilimit file with 4 (default for new images). Old default
value is 2.

This patch rolls in LU-9826 fix for larger ZFS
Lustre-change: https://review.whamcloud.com/28464
Lustre-commit: 6ab9866730c1bd2344869502586fc322f6ed7b71

This patch on master
Lustre-change: https://review.whamcloud.com/28820
Lustre-commit: f37b25fbb4acf3f96ebc61bb4d9137a8f0401e35

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I33921ac58a5252f3259145d5e00faedcd21559f9
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29068
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-7988 hsm: update many cookie status at once

Instead of calling mdt_agent_record_update, which calls
cdt_llog_process, once for every HAL, build a list of the cookies to
update with their status and call mdt_agent_record_update just once
per seconds at most.

Update mdt_agent_record_update to take a status for every cookie.

Test-Parameters: trivial testlist=sanity-hsm

Lustre-change: https://review.whamcloud.com/19584
Lustre-commit: f3a415289b560b5f422efe2bd08b3b7cff113cf0

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: Ie4afd667727e07570ed6a2d51e8dfaea8302b97b
Signed-off-by: Ben Evans <bevans@cray.com>
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28973
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-3719 ldiskfs: adjust s_mb_prealloc_table_size correctly

When mb prealloc table item value is not valid, the
s_mb_prealloc_table_size should be adjust accordingly.
Port patch to rhel7 & sles12.

Lustre-change: https://review.whamcloud.com/27748
Lustre-commit: b9ecf2671cba08530cc49d47713a360f2c6c25ca

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alexander Boyko <alexander_boyko@xyratex.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I5aa3a32a6a3aedc70160409c6443746fd2ccbbc9
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28228
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9462 doc: update lfs setstripe man page and usage

Update the lfs-setstripe.1 man page formatting and content.
Update the "lfs setstripe" usage message to be in "common use" order.

Test-Parameters: trivial
Reviewed-on: https://review.whamcloud.com/27066
Lustre-change-by: Andreas Dilger <andreas.dilger@intel.com>
Lustre-commit: aa83ef5a9ec470d2f6af07d47b3dd763f4a0e7ce

Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ia761c7562fa398a8c1bb4354d09757cd5f3ebbe5
Reviewed-by: Jian Yu <jian.yu@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29627
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9158 test: Use project ID for project quota for quota_scan

Use project ID instead of quota user for project quota in
function quota_scan. This is not a fix for the rebalancing error,
only correct the test script issue.

Lustre-change: https://review.whamcloud.com/28957
Lustre-commit: 3a2f24fefbc8c4b6e98dfd6ba9957326ea0e7a97

Change-Id: I9165ada17de0f32ac38f720a5e9e9be46363b41f
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29626
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9741 test: Correct check of stripe count for directories

With the progressive file layout feature, the 'lfs getstripe -d'
output will report stripe count for each component. Since
sanity test 27w checks that there is only one "stripe_count"
line in the 'lfs getstripe -d' output, this needs to be changed
to check that there is at least one "stripe_count" reported.

Test-Parameters: trivial
Lustre-change: https://review.whamcloud.com/27980
Lustre-commit: 1cc354d559d1df90a2eaa886eb0e3956bf385a9e

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I51f141b2542e65d1bc296cb2cb14c12f22afdcec
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29625
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9672 gss: fix expiration time of sunrpc cache

Expiration time of sunrpc cache is misinterpreted. Downcal
and response from user space must provide an epoch time,
not a duration.
And on kernel side, expiry must always be expressed in seconds
from boot, as set when retrieved from get_expiry().

Lustre-change: https://review.whamcloud.com/27667
Lustre-commit: f35425801545e0b47279597983cfbd02e837c45f

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I35c58a040a62410374dee0be3ae5bed7956cd985
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29624
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9967 tests: run-llog to cleanup properly

move ignore_errors so that setup's error is ignored, the script
continues with detach command needed to release the module.

Lustre-change: https://review.whamcloud.com/29057
Lustre-commit: e3921c4727c1d2e9866e0013fd37a7a4d4616621

Change-Id: I899de7a321fa927ca9884a6e4f9e8785bbc46f7c
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29366
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9887 lfsck: calculate LFSCK speed properly

Originally, we used do_div(a,b) to calculate the LFSCK average
speed, and got the result from the parameter @a. But later, we
replaced do_div(a,b) with div_u64(a,b). The latter one doesn't
stores the quotient in the parameter @a, instead, the quotient
is returned via the function return value. The patch fixes the
LFSCK logic to obtain the LFSCK average speed from div_u64(a,b)
return value.

Lustre-change: https://review.whamcloud.com/28617
Lustre-commit: cf800c062c8c6424c442509139297095f8a708db

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I442fb8f7e6c51a4853ea37694e3c221f97e26b19
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29294
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9963 test: add parallel-scale tests to ALWAYS_EXCEPT

add the following parallel-scale tests

parallel_grouplock :
test_parallel_grouplock: test failed to respond and timed out
Associated Jira ticket LU-9429 is in open state.
to ALWAYS_EXCEPT list.

Test-Parameters: trivial testlist=parallel-scale

Lustre-change: https://review.whamcloud.com/28914
Lustre-commit: 38c78e21ca7abe4bd6e6500d88a692e7a56a7712

Change-Id: I25709af1ab49a30498a89e5369521582c5ab6cf8
Signed-off-by: dilip krishnagiri <dilipx.krishnagiri@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: James Casper <jamesx.casper@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29146
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9802 pfl: swapping lcm_entry_count correctly

It's a u16 integer so it should use le16_to_cpu() instead of
le32_to_cpu().

Test-Parameters: trivial

Lustre-change: https://review.whamcloud.com/28256
Lustre-commit: 21ea1f7629535090d64ee3355c491a9e35256f23

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I43c31a76d78aa294a3e3296a1bb69f4d6fb1423d
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29141
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10023 kernel: kernel update [SLES12 SP3 4.4.82-6.9]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp3 testgroup=review-ldiskfs mdsdistro=sles12sp3 ossdistro=sles12sp3 mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs
Change-Id: Ie65afb3d7356c4679f4f37f4af324e955261b5af
Signed-off-by: Bob Glossman <bob.glossman@intel.com>

LU-9956 kernel: kernel upgrade [SLES12 SP3 4.4.82-6.3]

Minor linux version upgrade, but SP2 and SP3 use linux 4.4 versions.
Some new kernel patches, a few revised ldiskfs patches.
All new target and config files.
Some autoconf changes to adapt to new or altered kernel APIs.

Test-Parameters: clientdistro=sles12sp3 testgroup=review-ldiskfs mdsdistro=sles12sp3 ossdistro=sles12sp3 mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I99e2b6848197ea19402fa415fdb562d03e87d947

LU-10119 scripts: Correct shebang/hashpling format

Shebang/hashpling should not have a space between the number sign and
exclamation mark.

Test-Parameters: trivial
Lustre-change: https://review.whamcloud.com/29605
Lustre-commit: dd23aa4a64c776784cee05f69560c00b36440dc7

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I4a31d7dd7579bfaf231284c678e4100acea77d9b
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29689
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9416 hsm: add kkuc before sending registration RPCs

This avoids a situation where the registration completes and the CDT
sends HSM actions just before the kkuc registration happens. In this
case the client drops the actions because there are no CT pipes in the
kkuc list.

Lustre-change: https://review.whamcloud.com/28751
Lustre-commit: bdb04079570dd2de3b9310221e0206d64efb9ecc

Change-Id: Icbd6575f04c0ca7e8f731ee8481ec72a9ff4f2e1
Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29687
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9574 llite: pipeline readahead better with large I/O

Fixed a bug where next readahead is not set correctly when
appplication issues large I/O;
Extend the readahead window length to at least cover the size of
current I/O.

Lustre-change: https://review.whamcloud.com/#/c/27388/
Lustre-commit: e94dcdcf70526177633b71cfa1c6cb68c8f28ef0

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I43c5e4f25ea30d4a36263db2588bde0401122990
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29016
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-10000 kernel: kernel update [SLES12 SP2 4.4.74-92.38]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
mdsdistro=sles12sp2 ossdistro=sles12sp2 \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I825369d9cafd4f75dc193f614d5adfe87769fcd1
Reviewed-on: https://review.whamcloud.com/29069
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-9558 tests: add completion.h header to kinode.c

When building lustre on a ARM fedora box running a 4.11 kernel
kinode.c failed to build due to the completion.h header missing.
A simple including of this header fixes everything.

Test-Parameters: trivial

Lustre-change: https://review.whamcloud.com/28998
Lustre-commit: 982d660edbcacd528579f25ed2a3ae076a0134f3

Change-Id: I2a193346b214616a6f5dee5049466fb55567e396
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Frank Zago <fzago@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/29567
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-7802 ldlm: No -EINVAL for canceled != unused

If any locks are removed from or added to the lru, the
check of "number unused vs number cancelled" may be wrong.
This is fine - do not return an error or print debug in
this case.

Lustre-change: https://review.whamcloud.com/28560
Lustre-commit: a5081b7362e44b8d38aee1112f9a7d3aae1642c0

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I2ee9b8d86fbd6c9bd2c29e3472e3d410ee303374
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28975
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>