LU-10978 utils: preserve lustre_rsync state

In lustre_rsync, if debugging is enabled then we zero-out info and ext
in each iteration of the main loop of lr_replicate(). This may prevent
the consumed changelog entries from being cleared at the end of
lr_replicate().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ibd089524ca6cd6f488fc3558882cc431fdf65de7
Reviewed-on: https://review.whamcloud.com/32246
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10980 test: correct the version code check in sanityn.sh

The following patch was landed to 2.11.50 instead of 2.10.58, so
the version code check in sanityn.sh test_77ja should be corrected:
- Lustre-commit: e0cdde123c14729a340cf937cf9580be7c9dd9c1
- Lustre-change: https://review.whamcloud.com/27608

Test-Parameters: trivial testlist=sanityn envdefinitions=ONLY=77ja \
serverjob=lustre-b2_11 serverbuildno=2

Change-Id: I619d5fd6fa2297663084d3b5b8bdbc1b4642cdc1
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-on: https://review.whamcloud.com/32239
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10979 test: correct the version code check in sanity-sec.sh

The following patch was landed to 2.11.50 instead of 2.10.59, so
the version code check in sanity-sec.sh test_27a/b should be
corrected.
- Lustre-commit: 5b64d9fb0d5c5f292548c23e7841cc30f7a8423e
- Lustre-change: https://review.whamcloud.com/#/c/31450

Test-Parameters: trivial testlist=sanity-sec envdefinitions=ONLY=27 \
serverjob=lustre-b2_11 serverbuildno=2

Change-Id: I641ada8fb75d6347873a86ef453ebafd029876af
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-on: https://review.whamcloud.com/32238
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-9325 libcfs: handle complex strings in cfs_str2num_check

Originally the function cfs_str2num_check used simple_strtoul
but has been updated to kstrtoul. The string passed into
cfs_str2num_check can be a very complex, for example we could
have 10.37.202.[59-61]. When simple_strtoul was used the first
number until we hit a non-digit character could be extracted
but testing showed that kstrtoul will not return any value if
it detects any non-digit character. Because of this change in
behavior a different approach is needed to handle these types
of complex strings. The use of sscanf was investigated to see
if it could be used to extract numbers from the passed in
string but unlike its glibc counterpart the kernel version
also just reported a error with no results if a non-digit value
in the string was encountered. Another possible approach would
be to use __parse_int directly but that class of functions is
not exported by the kernel. So the approach in this patch is
to scan the string passed in for the first non-digit character
and replace that character with a '\0' so kstrtoul can be used.
Once completed the original character is restored. We also
restore a original behavior that was removed to return 0 when
we encounter any non digit character before the nob count.

Linux-commit : 3ad6152d766039cb8ffd8633d971fb79402e5464

Change-Id: If73d605499e2f05224a14417b0029036db38c8ba
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32217
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10907 tests: improve nodemap_test_setup()

Do wait_nm_sync() after modifying default.admin_nodemap, otherwise
it might cause the latter sync to fail.

Also, this patch checks the number of clients for sanity-sec.sh
test_23a, 2 clients at least for nodemap c0 and c1.

Test-Parameters: trivial testlist=sanity-sec

Change-Id: I7c7d5929b2f2b5aaaa0e7e884fa296fdf3cc1b57
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-on: https://review.whamcloud.com/32018
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10855 llog: remove llog_cancel()

Remove llog_cancel() and replace its sole use with a direct call to
llog_changelog_cancel(). Simplify the prototype of
llog_changelog_cancel() according to its use.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I32f5adf34ca10b918e5860e03ef0d8028c941697
Reviewed-on: https://review.whamcloud.com/31795
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10699 hsm: add local object storage to MDTs

Add local object storage (LOS) setup and cleanup to MDTs. LOS will be
used for the HSM actions index.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I1f9590350beeb49ebf14bfc602923c4cc8f6a15f
Reviewed-on: https://review.whamcloud.com/31758
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9325 obdclass: handle strings correctly in lmd_find_delimiter

The feedback from the pushing of the lmd_find_delimiter() work
upstream was not to directly parse the string data passed in.

Change the return value to a bool and return true when the
character is found. Currently even though this function is
named lmd_find_delimiter() it return 1 when nothing is found
which is counter intuitive. Use strcspn() to determine the
position where the first delimiter is found. Add a test to
make sure the mount string is valid.

Change-Id: I7ef53f29a6d3284acd01225c115712d3674e8435
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/26558
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9230 ldlm: speed up preparation for list of lock cancel

Keep the skipped locks in lru list will cause serious
contention for ns_lock. Since we have to travel them
every time in the ldlm_prepare_lru_list(). So we will
use a cursor to record position that last accessed
lock in lru list.

Change-Id: Ibda36a90e54076cb785a65910b34300639b3e140
Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Signed-off-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-on: https://review.whamcloud.com/26327
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10898 tests: enable conf-sanity 32a/32d

Stop zed service because it hold zfs module open.
Export zpools before trying to rmmod zfs.

Test-Parameters: trivial testlist=conf-sanity
Test-Parameters: trivial testlist=conf-sanity mdtfilesystemtype=zfs ostfilesystemtype=zfs
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Id1b891cadb91d9e3631a2d067c9d76a2965c34ff
Reviewed-on: https://review.whamcloud.com/31991
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10843 mgs: allow snapshot after MGS remount

If the MGS is unmounted/mounted without the MDS reconnecting,
the fsdb FSNAME-barrier would not be created.

This change allows mgs_barrier_freeze (called from snapshot_create)
to attempt the create the required fsdb.

This adds a test to sanity-lsnapshot.sh for this issue.

Test-Parameters: testlist=sanity-lsnapshot mgtfilesystemtype=zfs ostfilesystemtype=zfs mdtfilesystemtype=zfs combinedmdsmgs=false ostcount=2 standalonemgs=true
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I2432cc0bdaddb07f024744065ca2ced77288fd7b
Reviewed-on: https://review.whamcloud.com/31779
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10472 osd-ldiskfs: add T10PI support for BIO

This patch enables data integrity check in osd-ldiskfs when submiting
bio.

A fault injection mechanism is added to make sure the data integrity
check works well. On a OST with T10PI feature enabled, following
results are expected:

$ lctl set_param fail_loc=0x243
fail_loc=0x243
$ dd if=/dev/zero of=/mnt/lustre/file bs=1048576 count=100
dd: error writing '/mnt/lustre/file': Invalid or incomplete
multibyte or wide character
34+0 records in
33+0 records out
34603008 bytes (35 MB) copied, 0.510675 s, 67.8 MB/s

When doing fault injection, the write operation will wait until the
value is returned from BIO. Otherwise, returned error number may not
be returned to the application.

This implies a problem: because of the async submit of BIO, even the
OST has T10PI enabled, the application might not be able get error
notification when data corruption happens. However, there is nothing
we can do to improve this (unless write performance is not important),
because async commit is essential for good performance.

Change-Id: I76cc14b42feed835158100d35f65aedae0d79a5c
Signed-off-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/30792
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>

New tag 2.11.52

Change-Id: Ie2c8d2a2d9d655f0646324780b8e02f0d031c4ba
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10805 libcfs: timer_setup() API changes

Linux kernel 4.15 replaced setup_timer() with the new
interface timer_setup().
Introduce cfs wrappers to handle the API changes.

Linux-commit: e99e88a9d2b067465adaa9c111ada99a041bef9a

Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: Ib79495f9ab7e955d6f72f1e9390cec0e23e2d641
Reviewed-on: https://review.whamcloud.com/31790
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10560 llite: remove assigning f_version in ll_readir.

Backport upstream patch:
 commit: 8ec426c7019ed9600d9dc0cf758445adcdbfc14e
         lustre: don't set f_version in ll_readdir

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I296ce2cf8a7f4f91cf051f281206a7af60b7fb38
Reviewed-on: https://review.whamcloud.com/32268
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10988 lfsck: load object attr when prepare LFSCK request

It will avoid empty attrs in LFSCK request (lfsck_namespace_req).
The patch also shows invalid mode for dt_mode_to_dft() for debug.

Other cleanup for lfsck_namespace_striped_dir_rescan().

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I2874160d9a0c9e3084d0d3d7f365940108c82018
Reviewed-on: https://review.whamcloud.com/32245
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10944 kernel: kernel update [SLES12 SP3 4.4.126-94.22]

Update target, kernel_config, and ldiskfs files for new version
ldiskfs for sles12sp2 and sles12sp3 are no longer identical,
so revise some build files for sles12sp2 here too.

Test-Parameters: clientdistro=sles12sp3 testgroup=review-ldiskfs \
  mdsdistro=sles12sp3 ossdistro=sles12sp3 \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: If554b24644f763462002d81309f8edfa016210e8
Reviewed-on: https://review.whamcloud.com/32139
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10924 tests: conf-sanity test_116 server version check

Patch "LU-10520 mkfs: enable extents for big MDT" adds test
that check if extents are enabled with 64bit option. But
Lustre FS before this patch sets ^extents opiton. So interop
testing fails.

This patch adds version check for MDT to conf-sanity test_116.

Signed-off-by: Artem Blagodarenko <artem.blagodarenko@gmail.com>
Cray-bug-id: LUS-5812
Change-Id: Ia42597174944ec1627dfe0f6a7cf8149e674cecc
Reviewed-on: https://review.whamcloud.com/32050
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10560 osd: bi_bdev is replaced by bi_disk

The member bi_bdev is replaced by bi_disk in struct bio.
 - upstream commit: 74d46992e0d9dee7f1f376de0d56d31614c8a17a
   block:  replace bi_bdev with a gendisk pointer and partitions index

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Ice1fb53f8371fb744af5dbac6c076ce817770213
Reviewed-on: https://review.whamcloud.com/31975
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-10857 tests: allow to disable project quotas

Project quotas is disabled by default, sanity-quota.sh
enables it always if project quota supported.
Patch adds the possibility to disable project quotas
for regular user/group quota testing by using
ENABLE_PROJECT_QUOTAS=false.

Test-Parameters: trivial testlist=sanity-quota envdefinitions=ENABLE_PROJECT_QUOTAS=false
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Change-Id: I48176898fd940d66d0ebee4ef085a0bcece02ee9
Reviewed-on: https://review.whamcloud.com/31796
Tested-by: Jenkins
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 ptlrpc: use workqueue for pinger

lustre has a "pinger" kthread which periodically pings peers to ensure
all hosts are functioning.

This can more easily be done using a work queue.

The SVC_EVENT functionality to wake up the thread can be replaced with
mod_delayed_work().

Change-Id: I6064e9baea186323ab2eb0cca61ed23fcf8b55f7
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/31728
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8066 mdc: move mdc-specific procfs files to sysfs

This moves max_rpcs_in_flight and max_pages_per_rpc to
/sys/fs/lustre/mdc/.../

Linux-commit : 2ee26222d497cf75eb9a02f324dee8b6b16e1e67

Add better error handling.

Change-Id: If6b50d2a3ef26325df0018ffb990a74ce2b75c10
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/30961
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8066 fld: move all files from procfs to debugfs

Port of the upstream client work done by Dmitry Eremin.

Linux-commit: 827650494fbe9390052502d0498c8b90b2e329ec

Move all server side procfs handling to debugfs.

Change-Id: I6b0311d148576dec1ab1ef7ca1afce0ab2e4ed3b
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: Dmitry Eremin <dmiter4ever@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/26651
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9857 lmv: dir page is released while in use

When popping stripe dirent, if it reaches page end,
stripe_dirent_next() releases current page and then reads next one,
but current dirent is still in use, as will cause wrong values used,
and trigger assertion.

This patch changes to not read next page upon reaching end, but
leave it to next dirent read.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I28865d5719dfe95c0efba2f5543661a07aca6bd2
Reviewed-on: https://review.whamcloud.com/32180
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7763 tests: improve sanityn 33d

Sanityn.sh 33d verifies that CoS (Commit-on-Sharing) for DNE, but if
transaction commits before unlock, CoS will not be triggered. To
avoid this, do all tests twice, it's unlikely that both will fail.

Test-Parameters: trivial testlist=sanityn mdscount=2 mdtcount=4
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I590327111f7be2df233568ce408ee56a5af1d73a
Reviewed-on: https://review.whamcloud.com/31946
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10992 llite: decrease sa_running if fail to start statahead

Otherwise the counter of ll_sb_info::ll_sa_running will leak as
to the umount process will be blocked for ever.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I6b3a0dfb39f120a0c58184a22211b52376b41bd9
Reviewed-on: https://review.whamcloud.com/32287
Tested-by: Jenkins
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-10900 osd: wrong assertion in osd_transfer_project

When project ID feature is not enabled on ldiskfs, the project
ID of any inode should be zero. osd_transfer_project() made
the opposite assertion.

Fixes: LU-10565 osd: move ext4_tranfer_project to osd
Test-Parameters: trivial testlist=sanity-quota testgroup=review-ldiskfs \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I8c065b9453e0e2b3f9f26e39fc82e8e73902df91
Reviewed-on: https://review.whamcloud.com/31947
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1644 ptlrpc: fix return type of boolean functions

Some functions are returning type int with values 0 or 1 when
they could be returning bool.  Fix up the return type of:

    lustre_req_swabbed()
    lustre_rep_swabbed()
    ptlrpc_req_need_swab()
    ptlrpc_rep_need_swab()
    ptlrpc_buf_need_swab()

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I758afbf2d50ee9c0930d3080016803a299ecab07
Reviewed-on: https://review.whamcloud.com/32088
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10175 ldlm: IBITS lock convert instead of cancel

For IBITS lock it is possible to drop just conflicting
bits and keep lock itself instead of cancelling it.
Lock convert is only bits downgrade on client and then
on server.
Patch implements lock convert during blocking AST.

Test-Parameters: mdssizegb=20 testlist=sanity-dom,dom-performance,racer,racer,racer
Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: Id073f14ec3283479a9a48fd80090ecd3b0967b6d
Reviewed-on: https://review.whamcloud.com/30202
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10985 mdt: properly handle unknown intent requests

Invalid intent requests should be rejected early on,
so the later code does not make any assumptions about
validity of various structures like extended pills
and such.

Change-Id: I74de899ae6cea7017ae8ca9a8c488ca801a38c5d
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/32237
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-10575 build: make lustre-test dpkg depend on mpi-default-dev

Currently Ubuntu testing fails to run MPI based applications in
the lustre test suite. This is due improper dpkg setup. The
solution is to make lustre-test to depend on mp-default-dev
meta-package which will pull MPICH or OpenMPI depending on the
platform.

Test-Parameters: trivial testlist=performance-sanity clientdistro=ubuntu1604

Change-Id: I6b162d4d7b2485289d3e1c4dc23bcd6263fee682
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32179
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Thomas Stibor <t.stibor@gsi.de>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10413 ldlm: expose dirty age limit for flush-on-glimpse

Glimpse request may cancel old lock and cause data flush.
That helps to cache stat results on client locally early.
The time limit was hardcoded to 10s and is exposed now
as ns_dirty_age_limit namespace value, it can be set/check
via /sys/fs/lustre/ldlm/namespaces/<namespace>/dirty_age_limit

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Ie5b74eeb1720d552ab3f667e38af3925855f40ac
Reviewed-on: https://review.whamcloud.com/32113
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8837 obd: remove linux directory

With only two files in obdclass/linux we can move the code around.
Rename linux-obdo.c to obdo_server.c since its only server code.
Move the ioctl handling in linux-module.c to class_obd.c and
rename linux-module.c to obd_sysfs.c since it only contains sysfs/
debugfs code.

Test-Parameters: trivial

Change-Id: Ie4b99cf37f83158f9d43eb38c971c1602f805c6f
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32100
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10922 osd-zfs: return -ENOENT if object destoryed

Originally, if the target object does not exist or just destroyed,
the osd_xattr_get_internal() will return -ENOENT to the caller, it
is right. But one of the callers: osd_xattr_get(), converts it as
-ENODATA before returning to the upper layer by wrong.

This patch fixes it.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I9bfd03c53dda165af37a4ae62b4efbc5875833e5
Reviewed-on: https://review.whamcloud.com/32042
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10902 mdd: force disable changelogs early and safely

In mdd_changelog_user_purge() changelogs were disabled too late
allowing for new records to be unnecessarily gathered.
Also, a race was possible, causing changelogs to be disabled even
with registered users.

sanity/tests_160h has been added to check race is fixed.
Also, changelog_[unregister,clear]() functions in test-framework.sh
had to be modified to handle facets in same order than
changelog_register() to allow the race simulation to work in
DNE environment.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I78f07068e89b1cf2cf0c83dd2068f82f8a4a6dd5
Reviewed-on: https://review.whamcloud.com/32007
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10850 fld: handle empty ranges in fld_index_init()

In fld_index_init(), ignore the empty range corresponding to the
zeroed-out phantom record that is included in newly created ldiskfs
IAM indexes. (This phantom record will be removed in a subseuent
patch.) Ensure that lfs_new will be set when the FLD index was empty
of valid ranges. (Previously it was properly set on ZFS backed targets
but not on ldiskfs.)

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I9850eac686e1488ffbd627f100075ce6f29200d8
Reviewed-on: https://review.whamcloud.com/31871
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10856 tests: remove deprecated lfs getstripe -M usage

The "lfs getstripe -M" and "lfs mv -M" options are deprecated
since 2.9 and should not be used in the tests.

Clean up some of the nearby tests to match current coding style:
- lower-case local variable names
- tabs for indentation
- replace $GETSTRIPE/$SETSTRIPE with "$LFS getstripe/setstripe"

Test-Parameters: trivial testlist=replay-dual,sanity-flr
Test-Parameters: testlist=recovery-small,replay-single mdtcount=2
Test-Parameters: testlist=testlist=sanity,sanityn,sanity-hsm mdtcount=2
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Id86395a03379bef2de24dfa92642c928f51cab07
Reviewed-on: https://review.whamcloud.com/31808
Tested-by: Jenkins
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Elena Gryaznova <c17455@cray.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10805 ldlm: convert ACCESS_ONCE() to READ_ONCE()

Linux kernel 4.15 removed ACCESS_ONCE, use READ_ONCE instead.

Linux-commit: b899a850431e2dd0943205a63a68573f3e312d0d

Test-Parameters: trivial
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I139e4cab95b99f362a85c37efe236137835f9c54
Reviewed-on: https://review.whamcloud.com/31789
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-10838 tests: replay-single/89 to wait for recovery

with a single client (e.g. local setup) test 89 should wait
for recovery completion. client_up (i.e. lfs df) is not enough
as the new client's OSC will be skipped being not active.

Change-Id: I859edfc5a38cbac44377c7660fabb87db55978e9
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/31729
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10624 tests: conf-sanity needs larger ZFS filesystems

otherwise some tests can take a lot (like ~2000 seconds
for test 106) due to very limited space and short
transactions. with the patch 106 passes in ~440 seconds.

Change-Id: I0597072cf72315411b7e656b1e87a0fceabb171f
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/31192
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-8066 lov: Move suitable variables from procfs to sysfs

Migrate procfs files with simple output to sysfs. Create
sysfs symlinks for target_obds.

Linux-commit : cb1debff6d8b254536ec6ff642a2bdd22c86cb85

Change-Id: I97e403694fd1aab9f70363a2961beb08c6959447
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/30960
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8066 lmv: Move suitable entries from procfs to sysfs

Linux-commit: b5fa70d76fb1c532dd4468b8083e2945c9df9ddb

Change-Id: I89ffcf5a8efe870a3bd02462681b24952a788dab
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/28460
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8293 lnet: Add insserv header to lnet init script

The lnet init script does not contain header information as described
by the insserv man page. This header information is needed to ensure
the lnet init script is not run until openibd has been able to start

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I6f778e827f88ce34199dff70be5d5089f0ba51b9
Reviewed-on: https://review.whamcloud.com/20835
Tested-by: Jenkins
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10932 libcfs: properly handle failure cases in SMP code

While pushing the LU-8703 and LU-7734 work upstream some bugs were
pointed out by Dan Carpenter in the code. Due to single err label
in cfs_cpt_table_alloc() and cfs_cpu_init() a few items were being
cleaned up that were never initialized. This can lead to crashed
and other problems. In those initialization function introduce
individual labels to jump to only the thing initialized get freed
on failure. Lastly in cfs_cpt_table_alloc() handle the failure
case instead of the passed case which is the perferred style.

Change-Id: I969f4e327888042a517bc321b68d55fc691b074e
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32085
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6349 mds: remove obsolete MDS_VTX_BYPASS flag

The MDS_VTX_BYPASS flag is only set and never checked.  This is true
since 2.3.53-66-g54fe979 "LU-2216 mdt: remove obsolete DNE code", but
it was already obsolete for a long time before that.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ia9510146326e71a134fef85dd8febf7b752cab07
Reviewed-on: https://review.whamcloud.com/31984
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4684 ptlrpc: add dir migration connect flag

Add dir migration connect flag to prevent collision with other
features. Though dir migration code exists, it will be reworked,
and the new RPC protocol won't be compatible with current one.

Also handle the previously-added OBD_CONNECT2_FLR flag.

Test-Parameters: trivial
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Ie3eaec8602664170b52e3778b76059a0899ee7b1
Reviewed-on: https://review.whamcloud.com/31914
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-9019 libcfs: remove the remaining cfs_time wrappers

In the lustre code various small bits are left of the libcfs time
wrappers. Remove all the remaining wrappers except for the inline
function cfs_time_seconds(). For cfs_time_seconds() we have to
move to nsec_to_jiffies() since msecs_to_jiffies() has become an
inline function in jiffies.h which means HZ can be different on
the installation node verses what the target node is configured
for. Since nsec_to_jiffies() is a normal built in kernel function
we can avoid the LU-5443 issues. For cfs_duration_secs() we use
the internal kernel function jiffies_to_msec() since only the
newest kernels have jiffies_to_nsec() and that function is just
a wrapper around jiffies_to_msec(). All the reset of the libcfs
time abstractions are gone.

Change-Id: I166973071304f1d55a15b1e21fcfbe434ff58199
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/31068
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9780 tests: Testing Round-Robin allocation

Adding a test for fix in LU-977 which shows that
in absence of protection to lqr_start_idx there is
possibility of imblance in allocating objects on OSTs
with round-robin algorithm.

This test checks the protection of lqr_start_idx by using
a new reproducer, rr_alloc which uses MPI to create files in
parallel, and checking the even file distribution over OSTs.

Distribution check formula is adjusted as per implementation,i.e
some factors like - exhaustion of pre-created objects and counter
‘lrq_start_count’(lod_qos.c) reseed are also taken care so that
object allocation is not affected by these factors.

Test-Parameters: trivial osscount=3 clientcount=6                  envdefinitions=ONLY=rr_alloc testlist=parallel-scale
Signed-off-by: Ashish Maurya <ashish.maurya@seagate.com>
Signed-off-by: Rahul Deshmukh <rahul.deshmukh@seagate.com>
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: MRP-2723
Reviewed-by: Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Vikram Jadhav <jadhav.vikram@seagate.com>
Change-Id: I55f798c6dc8e607f002365f4a22ccf59a454fe1d
Reviewed-on: https://review.whamcloud.com/28075
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3665 tests: Cleanup echo client after obdfilter-survey

Some failures of obdfilter-survey do not cause an error in
obdfilter-survey.sh In some cases obdfilter-survey did not
cleanup echo clients it had created and that could hang
umount of osts.
Change test-framework.sh::cleanupall to remove echo clients before
trying to umount to prevent the echo clients from holding the OST or
MDS/MGS open forever.

Test-Parameters: trivial testlist=obdfilter-survey osscount=1 ostcount=2 mdscount=1 mdtcount=1
Test-Parameters: trivial testlist=obdfilter-survey mdtfilesystemtype=zfs ostfilesystemtype=zfs osscount=1 ostcount=2 mdscount=1 mdtcount=1
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I63ae59da84101c782aa9d5e7216cce3b3b1ff2fe
Reviewed-on: https://review.whamcloud.com/9350
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6655 ptlrpc: skip delayed replay requests

During recovery, there could be some delayed replay requests
after the final recovery completion ping request was handled,
and it should be skipped.

Change-Id: Ie0d5ff92c75f9d078b8ae28e899d4a821113194f
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: https://review.whamcloud.com/23205
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1644 mgc: remove obsolete IR swabbing workaround

The OBD_CONNECT_MNE_SWAB check was added to the MGC for compatibility
with servers in the 2.2.0-2.2.55 range (in 2012) with big-endian
clients.  2.2 was not an LTS release and is no longer being used.

Remove the checks on the client for OBD_CONNECT_MNE_SWAB being set,
and assume that the server does not have this bug.  This will allow
the removal of the rest of this workaround from the server code once
there are no more clients depending on the presence of this flag.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ie128745e3af8e9d41454e095922e0f6bc8fcab07
Reviewed-on: https://review.whamcloud.com/32087
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10910 mdd: deny layout swap for DoM file

Layout swap is prohibited for DoM files until LU-10177
will be implemented. The only exception is the new layout
having the same DoM component.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I8b9deaea7aab0d5694a4c5d9fe2f9f36d2cdd382
Reviewed-on: https://review.whamcloud.com/32044
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10909 utils: enable non-shared libmount_utils_ldiskfs.a

It depends on libcfs.a (due to cfs_abs_path()) so that library must
come after it in the list of libraries provided to the linker.

Test-Parameters: trivial
Change-Id: Id3026705ebac65b74d1ea48964c25aa6c4834552
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/31970
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-930 doc: include status in "lfs df" man page

Describe the "DRSI" target states in the lfs-df.1 man page.

Improve the overall description of how "lfs df [-i]" works, with
estimation of total/free inodes for ZFS, and show actual output.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I2ae4190c9ebd3467b5243df89938501d301cab07
Reviewed-on: https://review.whamcloud.com/31998
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Joseph Gmitter <joseph.gmitter@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10903 obdecho: use OBD_ALLOC_LARGE for lnb

When allocating the niobuf_local, if there are a large number of
(potential) fragments this allocation can be quite large.  Use
OBD_ALLOC_LARGE(lnb) and OBD_FREE_LARGE(lnb) to avoid allocation
errors and console noise.  This was causing sanity test_180c to
fail in a VM on occasion, and could also be problem in real use.

Tidy up test_180[abc] to use stack_trap to handle echo device
cleanup rather than having a series of conditional checks.
Use "error" to report errors rather than error numbers.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ib0e721f58fc0a62acca29b31fb4dfac5021cab07
Reviewed-on: https://review.whamcloud.com/31964
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10886 build: fix warnings during autoconf

Quiet warnings in configure checks to avoid confusion when debugging
autoconf/configure scripts, and to avoid potential errors during
detection.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I2916a120b57fb9fa529bf7050cf65511233ebbe5
Reviewed-on: https://review.whamcloud.com/31904
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8200 test: improve sanityn.sh 33c

If transaction is committed before unlock, lock won't be saved. So
trigger Sync-Lock-Cancel twice in sanityn.sh 33c, it's unlikely to
fail both.

Test-Parameters: trivial clientcount=2 mdscount=2 mdtcount=4 \
osscount=1 ostcount=8 mdtfilesystemtype=zfs ostfilesystemtype=zfs \
testlist=sanityn
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I5c99e5a8261df8c9f463aea7ed67df95baaf3e6c
Reviewed-on: https://review.whamcloud.com/31673
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10699 hsm: simplify mdt_hsm_{add,get}_actions()

Split the mdt_hsm_get_actions() functionality of
hsm_find_compatible_cb() into a new llog callback
(hsm_find_action_cb()). Simplify hsm_find_compatible() since it only
needs to handle adding new requests. Simplify the MDS_HSM_ACTIONS
handling code.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ie1e7eac85397d82ad9cc4790582bf8ba51713e70
Reviewed-on: https://review.whamcloud.com/31385
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8066 osc: move suitable values from procfs to sysfs

All single-value controls are moved from /proc/fs/lustre/osc/.../
to /sys/fs/lustre/osc/.../

Linux-commit : aab38b00ac19347bf982cf42c71aab14a9301dee

Change-Id: I69744804bc424ab171da8f649336a3ad450f0d05
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/30962
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2711 tests: conf-sanity clean up environment in test

Several Lustre test suites execute code or set variables
between tests even if the preceding test is not executed.
In the case of conf-sanity, the code between tests 23a and
23b and between 43b and 44 are there to clean up the environment
from the previous test. Properly clean up the file system
configuration/environment at the end of test 23a and 43b.

Test-Parameters: trivial testlist=conf-sanity

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I250bb2067d0c966f866b404025dc6240b16f301e
Reviewed-on: https://review.whamcloud.com/30703
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5170 lfs: Standardize error messages in lfs_path2fid()

Error messages in lfs_path2fid() are updated to a standard format.
Messages are prefixed with the name of the utility and the command
that caused the error.  User-provided values are delimited with
single quotes.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I32ab8650ddced9569837aa3d106ef1708c974bce
Reviewed-on: https://review.whamcloud.com/30670
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-5170 lfs: Standardize error messages in lfs_fid2path()

Error messages in lfs_fid2path() are updated to a standard format.
Messages are prefixed with the name of the utility and the command
that caused the error.  User-provided values are delimited with
single quotes.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I124f3e5bfad120abe701dce592da6005d53112c5
Reviewed-on: https://review.whamcloud.com/30668
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-5170 lfs: Standardize error messages in lfs_data_version()

Error messages in lfs_data_version() are updated to a standard
format.  Messages are prefixed with the name of the utility and the
command that caused the error.  User-provided values are delimited
with single quotes.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Idb0939c4fa23fc409d965183e5fe5dddcab6da4f
Reviewed-on: https://review.whamcloud.com/30667
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-10825 lnet: add ip2nets syntax handling for peer

Allow peers to be added using ip2nets syntax, from
both command line and YAML block.

Command line example:
lnetctl peer add --ip2nets 10.10.10.[3-6,9]@tcp
lnetctl peer del --ip2nets 10.10.10.[3-6,9]@tcp

YAML example:
peer:
    ip2nets:
      - nid: 30.10.10.[3-8]@tcp
      - nid: 40.10.10.[9-40]@tcp

Test-Parameters: trivial
Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I904be8d496ad2be277c3d21dc7f72cbc7ed02b50
Reviewed-on: https://review.whamcloud.com/31786
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10825 libcfs: generate ip addresses

Add infrastructure API cfs_ip_addr_range_gen() to generate a
maximum of 'count' IP addresses from an expression.

Test-Parameters: trivial
Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I479dd7128eef404106a7863124a38c501150ba9e
Reviewed-on: https://review.whamcloud.com/31785
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8066 ldlm: move all remaining files from procfs to debugfs

Move all files except stats. It will be moved later after change
type of obddev->obd_proc_entry member.

Linux-commit: 700815d47f9da0477229f009b6fa235f20da1e21

Clean up the helper functions used to implement "dump_granted_max" in
debugfs.

Change-Id: Id0a0b2c663295c7164c52f89d81525dbefbb992a
Signed-off-by: Dmitry Eremin <dmiter4ever@gmail.com>
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/29255
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8066 mgc: migrate away from procfs

Move mgc to using the new sysfs ping and conn_uuid. Move all
sysfs/debugfs handling to lprocfs_mgc.c and implement proper
error handling.

Change-Id: Iecbfac0afedaef9a2b6be33ce026d61008a33136
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/29250
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-5170 lfs: Standardize llapi messages for lfs_find()

Error messages in the llapi functions called by lfs_find()
are updated to a standard format.  Messages are prefixed with
the name of the utility and the command that caused the error.
User-provided values are delimited with single quotes.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ia324887964b4a89cc5d4f92d3f3a7fa421e03dca
Reviewed-on: https://review.whamcloud.com/28997
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-5170 lfs: Standardize error messages in lfs_mv()

Error messages in lfs_mv() are updated to a standard format.
Messages are prefixed with the name of the utility and the command
that caused the error.  User-provided values are delimited with
single quotes.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Id3d276024a2dcd1ca9169123f9d9193a846d7c85
Reviewed-on: https://review.whamcloud.com/28239
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-5170 lfs: Standardize error messages in set_time()

Error and warning messages in set_time() are updated to a standard
format.  Messages are prefixed with the name of the utility and the
command that caused the error.  User-provided values are delimited
with single quotes.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I4b5efa394b49f4a0af0c073ad707c2b8c6faf6b0
Reviewed-on: https://review.whamcloud.com/28234
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-10214 lnet: allow expressions for route config

Support the ip2nets syntax for route gateway configuration. Only support
a maximum of 128 gateways per configuration command. This upper limit is
to prevent a large number of routes to be configured by mistake. This
feature is available from both command line and YAML interfaces.

Command line examples:
lnetctl route add --net tcp1 --gateway 12.3.4.[2-6]@tcp
lnetctl route del --net tcp1 --gateway 12.3.4.[2-6]@tcp

YAML examples:
route:
    - net: tcp1
      gateway: 12.3.4.[2-20]@tcp

Test-Parameters: trivial
Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I0d8a0f94cce7140602a64f13f0401ef209f3ca57
Reviewed-on: https://review.whamcloud.com/30511
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10926 llite: avoid panic in ll_set_acl()

While pushing the xattr work upstream Dan Carpenter noticed a bug
in ll_set_acl() that could panic a node. The problem is in the
case of default the code jumps to out: which calls the function
forget_cached_acl(). If you call forget_cached_acl() with a type
not related to ACLs it will crash the node via a BUG(). To avoid
this just test rc after the switch block and return since neither
the acl or xattr have been modified.

Change-Id: Ib6f3e5e818635c56cc4abbf37e7c63e6897a72a6
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32045
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

New tag 2.11.51

Change-Id: I734b61e40f2565433f2c84b33ad108e32e7ead82
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10901 build: Update ZFS/SPL to 0.7.8

This skips 0.7.7 due to regression fixed in 0.7.8

https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.7.8
https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.7.7

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Icd6a21793a5f7502c88121def8d86d9aa48c6ae8
Reviewed-on: https://review.whamcloud.com/31949
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10560 libcfs: Fix inconsistent declaration for cfs_kernel_write()

The header file buffer should be const void * to match the
implementation.

Test-Parameters: trivial

Change-Id: I3a09b84e4c3dbe39870be04ae98c13e9e6221a6d
Fixes: b9a32054600a8d63948cced361191aa6ae7ea8f2
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Reviewed-on: https://review.whamcloud.com/31948
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10869 build: package configuration files for Ubuntu / Debian

For a long time Lustre never added /etc configuration files to its
debian packages. It could get away with but now you can see it
fail conf-sanity test 76a. This test the lctl set_param -P which
uses udev events to set the tunables for lustre. In order for it
to work a default udev rule has to be added to 99-lustre.rules.
Beside the missing 99-lustre.rules and in all the other files used
for configuration. Lastly create conffile which is the way Debian
handles potential stomping of configuration files. When installing
with apt-get install it will ask the person installing if they
want to over ride specific files.

Test-Parameters: clientdistro=ubuntu1604 trivial testlist=conf-sanity
Test-Parameters: envdefinitions=ONLY=76a testlist=conf-sanity

Change-Id: Ic0aaf2bba531ce23a3e23ef070a1501032ad1c9f
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/31850
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Thomas Stibor <t.stibor@gsi.de>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10805 lnet: switch to module_param

From Linux 4.15 kernel the set/get function prototypes
for module_param_call has changed. This triggers compile
error:
  CC [M]  lustre-release/lnet/lnet/api-ni.o
In file included from lustre-release/lnet/lnet/api-ni.c:38:0:
include/linux/moduleparam.h:232:22: error:
initialization from incompatible pointer type [-Werror]
  static const struct kernel_param_ops __param_ops_##name =
We can switch to module_param and cfs_kernel_param_arg_t since
they are already available in libcfs.

Linux-commit: b2f270e8747387335d80428c576118e7d87f69cc

Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: Ie6f0f3a31fd22904e03e8300f45f0a8684265abd
Reviewed-on: https://review.whamcloud.com/31788
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10841 ldlm: ASSERTION(lock->l_granted_mode!=lock->l_req_mode)

Policy processors can unlock resource to send BL AST,
so cached next list entry can become invalid.

Move sending BL ASTs to ldlm_reprocess_queue()
in case of LDLM_PROCESS_RECOVERY.

Cray-bug-id: LUS-5689
Change-Id: Ib9b757576461b2f74aaa916b4b62538a9abfa0dd
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-on: https://review.whamcloud.com/31726
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10699 hsm: remove struct hsm_compat_data_cb

Remove uses of struct hsm_compat_data_cb from hsm_find_compatible()
and its llog callback.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I33addc2bfd983046865286bc2c3200ed0bfd1380
Reviewed-on: https://review.whamcloud.com/31382
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5170 lfs: Standardize error messages in lfs_changelog()

Error messages in lfs_changelog() are updated to a standard format.
Messages are prefixed with the name of the utility and the command
that caused the error.  User-provided values are delimited with
single quotes.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Ib9229d13a6fa4e76ee407de33f181465adc879b6
Reviewed-on: https://review.whamcloud.com/30665
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-10732 tests: sanity-lfsck to reset speed limit

- sanity-lfsck should reset speed limit on all servers.
- Change MDS/OST size to 300M as 100M is way too small for
  ZFS and mkfs time shouldn't be a problem for ldiskfs now
- Enable test_9a
- More accurate check for test_30.

Change-Id: I6129c1248a5bb51004d446b858c2ffba1354e74d
Signed-off-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/31488
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-9586 tests: remove replay-dual 15c from ALWAYS_EXCEPT

replay-dual test 15c "remove multiple OST orphans" was added
to the ALWAYS_EXCEPT list because of bugzilla 10124. Remove
replay-dual test 15c from ALWAYS_EXCEPT list.

Test-Parameters: trivial testlist=replay-dual
Signed-off-by: dilip krishnagiri <dilipx.krishnagiri@intel.com>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I88866e485fdb037575ee4a72ed38aaa4b465df7d
Reviewed-on: https://review.whamcloud.com/27396
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Cliff White <cliff.white@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10889 ptlrpc: update req timeout if resending happened

When the server drops duplicate request processing, the client and
the server have different deadline for the same request. The server
operates with the first copy and the client operates with the second.

This patch adds request deadline updates if a duplicate request is
found.

A fix for LU-8420 changed lock callback prolong calculation to use
request deadline in case when service estimate changed since the
request has beed created. Using outdated deadline may cause
insufficient prolong timeout and subsequent client eviction.

Signed-off-by: Alexander Boyko <c17825@cray.com>
Signed-off-by: Vladimir Saveliev <c17830@cray.com>
Change-Id: I55725d396f50d864687248df46e7882290fc21ca
Cray-bug-id: MRP-3720 MRP-4289
Reviewed-on: https://review.whamcloud.com/31910
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10887 lfsck: offer shard's mode when re-create it

The namespace will re-create the lost shard of a broken stripe
directory if "-C" option specified. Under such case, the @mode
parameter should be given properly.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I2441d8ca83c932a34ef1971d334f54ecd7343b27
Reviewed-on: https://review.whamcloud.com/31915
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10368 mdc: resend quotactl if needed

In mdc_quotactl, it is better to resend the quotactl request
if reconnection or failover is triggered during the process.

Change-Id: I64f96863a6f10026aa69cba3c59095966b58b98d
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: https://review.whamcloud.com/31773
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10847 doc: fix broken ChangeLog

recent kernel update landings changed lines in the wrong
section of the ChangeLog file.
This patch moves the changes to the correct locaions.

Test-Parameters: trivial

Change-Id: I8ea3f35a8a8c6a6c7af3b23fa253cc882d0e61f3
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-on: https://review.whamcloud.com/32003
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10912 mdc: use large xattr buffers for old servers

Pre 2.10.1 MDTs will crash when they receive a listxattr (MDS_GETXATTR
with OBD_MD_FLXATTRLS) RPC for an orphan or dead object. So for
clients connected to these older MDTs, try to avoid sending listxattr
RPCs by making the bulk getxattr (MDS_GETXATTR with OBD_MD_FLXATTRALL)
more likely to succeed and thereby reducing the chances of falling
back to listxattr.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ia96323c47c91a44495b73be2d95705298c7f7ac9
Reviewed-on: https://review.whamcloud.com/31990
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4939 tests: fix typo in sanity.sh

4939 got merged with a typo at the top of sanity.sh

Signed-off-by: Ben Evans <bevans@cray.com>
Change-Id: Id9d407cb9ea2e3952f3ac02f5d675e381c66ade7
Reviewed-on: https://review.whamcloud.com/31931
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Elena Gryaznova <c17455@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10834 tests: fix cleanup_77c() defect

Check the length of string is non-zero properly.
Without this fix cleanup_77c() calls:
 "rm -f *" in case of empty $osc_file_prefix, and
 "do_facet ost1 rm -f *" in case of empty $ost_file_prefix

Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-5817
Test-Parameters: trivial testlist=sanity envdefinitions="ONLY=77c"
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Change-Id: Id3f3ec52c965b15900f7a649120b89267daa9810
Reviewed-on: https://review.whamcloud.com/31918
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10847 kernel update [SLES12 SP2 4.4.120-92.70]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
  mdsdistro=sles12sp2 ossdistro=sles12sp2 \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I24892ca2360370ede3ca7f909f2e2b1e2275870a
Reviewed-on: https://review.whamcloud.com/31764
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 obdclass: use workqueue for zombie management

obdclass currently maintains two lists of data structures (imports and
exports), and a kthread which will free anything on either list.  The
thread is woken whenever anything is added to either list.

This is exactly the sort of thing that workqueues exist for.

So discard the zombie kthread and the lists and locks, and create a
single workqueue.  Each obd_import and obd_export gets a work_struct to
attach to this workqueue.

This requires a small change to import_sec_validate_get() which was
testing if an obd_import was on the zombie list.  This cannot have every
safely found it to be on the list (as it could be freed asynchronously)
so it must be dead code.

We could use system_wq instead of creating a dedicated zombie_wq, but as
we occasionally want to flush all pending work, it is a little nicer to
only have to wait for our own work items.

Change-Id: I3d0fafcc4d3896ff8760d5d36d8cfa187f86bc7d
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/31707
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1187 idl: remove obsolete directory split flags

The directory split functionality from the old CMD (pre-DNE)
feature was never usable in production, and was removed before
the DNE 2.4 release.  Remove old flags relating to this feature.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I052ea57843ae67c18706a80221590924077bf46f
Reviewed-on: https://review.whamcloud.com/31700
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10157 lnet: make LNET_MAX_IOV dependent on page size

The default behavior of LNet is to always use 256 pages which is
LNET_MAX_IOV and that LNET_MAX_PAYLOAD is always one megabyte.
This assumes pages are always 4K in size which is not the case.
This cause bulk I/O errors when using platforms like PowerPC or
ARM which tend to use 64K pages. This is resolved by first making
LNET_MAX_PAYLOAD always one megabyte since this is what the
configuring sets it too by default and no one ever changes it.
In theory it could set it to as high as 16MB but that will cause
the I/O errors since the ptlrpc layer expects the packets to be
always 1 megabyte in size. Also it would be better to make the
maximum payload a per network setup configurations instead of for
everything. Second we make LNET_MAX_IOV equal to LNET_MAX_PAYLOAD
divided by the PAGE_SIZE. This way packets will always be the
LNET_MAX_PAYLOAD in size but the number of pages used,
LNET_MAX_IOV will vary depending on the platform it is creating
packets on.

Change-Id: Ie1dcdb195e68b44e2fa2d9b24715216d8aca4c65
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/31559
Tested-by: Jenkins
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ruth Klundt <rklundt@sandia.gov>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10766 utils: Correct error handling in llapi_dir_create

Free dirpath if namepath allocation failed.

Change-Id: If5c15219a28767e5c89b25a44b2ddaa907c5d12d
Test-Parameters: trivial
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/31510
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-10877 lu: fix reference leak

reference leak fix in dt_locate_at introduced in
f6d6a552398eb1e65857d9bf1afaaf98c8dc1a79.
Use a top level object in debug in lu_object_put to match with
lu_object_get.

Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Change-Id: I6dbeeb4f70a2914b1606ad0c5586db431a6dcd2c
Reviewed-on: https://review.whamcloud.com/31870
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Tested-by: Jenkins
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10898 tests: disable failing conf-sanity test

The test_32a and test_32d runs are failing continuously for
ZFS filesystems (module unload problem).  Disable tests while
we investigate source and fix it.

Test-Parameters: trivial testlist=conf-sanity mdtfilesystemtype=zfs ostfilesystemtype=zfs
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ia7e1fc7969c41d1fd46d6e9b078093704a1bed83
Reviewed-on: https://review.whamcloud.com/31960
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Jenkins
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-10833 tests: missing sed command

sed is complaining on missing command:
       cmd=$(echo $cmd | sed '/-n//')

Change-Id: Id1f0ed65719b58c4a51e9829177dd51996124d25
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/31704
Tested-by: Jenkins
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6632 mgs: dont remove EXCLUDE records on lctl replace_nids

conf-sanity.sh:test_66 is modified to illustrate the problem:
  add EXCLUDE records to config file. lctl replace_nids removes
  those records which leads to mounting problem
fix: Remove records marked as SKIP instead of EXCLUDE ones.

Change-Id: Ica4b23a74870d8ebcb09b240313df4d4c33bbbde
Signed-off-by: Vladimir Saveliev <c17830@cray.com>
Signed-off-by: Alyona Romanenko <alyona.romanenko@seagate.com>
Cray-bug-id: MRP-2105
Cray-bug-id: MRP-2766
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Artem Blagodarenko <c17828@cray.com>
Test-Parameters: trivial envdefinitions=ONLY=66 testlist=conf-sanity
Reviewed-on: https://review.whamcloud.com/14921
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Elena Gryaznova <c17455@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-10859 ldiskfs: fix deadlock with heavy memory preassure

On one Customer site, we hit following deadlock:

Thread 1:
ofd_object_punch
 osd_punch
  ldiskfs_truncate
   ldiskfs_inode_attach_jinode
     ...
     do_try_to_free_pages
      lu_cache_shrink
       mutex_lock -->try to hold @lu_sites_guard

kswapd thread2:
kthread
 shrink_slab
  lu_cache_shrink
    mutex_lock ---->hold already.
     ...
     dqget
      ldiskfs_acquire_dquot
       jbd2__journal_start-->blocked to wait for more credits.

Thread3:
kthread
 kjournald2
  jbd2_journal_commit_transaction-->blocked to wait Thread2 finished,
 since Thread1 add a handle into transaction.

So deadlock happens because of Thread1 wait Thread2, Thread2 wait Thread3..
but Thread3 wait Thread1....

This problem still exists even we have switched @lu_sites_guard
into a read/write lock, sine we hold write lock at lu_cahce_shrink().

Fixed the problem by making ldiskfs_inode_attach_jinode() use
GFP_NOFS.

Test-Parameters: testgroup=review-ldiskfs \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Change-Id: I0ab143fc0cdb8e1b0c490c2c25e8af483c491a81
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-on: https://review.whamcloud.com/31806
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

LU-10845 kernel: kernel update [SLES12 SP3 4.4.120-94.17]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp3 testgroup=review-ldiskfs \
  mdsdistro=sles12sp3 ossdistro=sles12sp3 \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I45c1a65c5ef5373201d8b2a1c457bfa2b37df058
Reviewed-on: https://review.whamcloud.com/31762
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>