LU-9966 tests: sanity-411 check LBUG direct

The sanity test_411 intends to catch LBUG while IO
under memory pressure. It assumes that 'dd' would
fail because memory alloction failure. But it is
not ture since RHEL7 has some problem in memory
accounting. So we can ignore the 'dd' status.

Test-Parameters: trivial

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Ie9af0fe68c85a9e2af4da6b49d0a697d00f3761b
Reviewed-on: https://review.whamcloud.com/32293
Tested-by: Jenkins
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>

LU-10772 utils: incorrect check in build_layout_from_yaml_node

In lustre/utils/lfs.c, build_layout_from_yaml_node(),
check and fail for NULL pointer 'string' before
dereferencing it.

Issue found in Static analysis.

Change-Id: I272432d3997f3bfae520ff3aa48d175309ccd386
Signed-off-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-on: https://review.whamcloud.com/31537
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11424 lnet: copy the correct amount of cpts to lnet_cpts

The incorrect size was used in the memory copy of the requested
cpts to net->lnet_cpts. This lead to the following in testing
RIP: 0010:lnet_match2mt.isra.8+0x2b/0x40 [lnet]

lnet_mt_of_attach+0x72/0x1b0 [lnet]
LNetMEAttach+0x60/0x1f0 [lnet]
ptl_send_rpc+0x26f/0xbb0 [ptlrpc]
libcfs_debug_msg+0x57/0x80 [libcfs]
ptlrpc_send_new_req+0x4c9/0x860 [ptlrpc]
ptlrpc_check_set.part.21+0x855/0x18b0 [ptlrpc]
? try_to_del_timer_sync+0x4d/0x80
? del_timer_sync+0x35/0x40
ptlrpcd_check+0x3ae/0x3f0 [ptlrpc]
ptlrpcd+0x2be/0x320 [ptlrpc]
? wait_woken+0x80/0x80

Changing the size from ncpts to ncpts * sizeof(*net->net_cpts)

Change-Id: I10832430e53ccc5b40ebce3ddfd2cf9ea330b0df
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33229
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11187 ldiskfs: don't mark mmp buffer head dirty

Marking mmp bh dirty before writing it will make writeback
pick up mmp block later and submit a write, we don't want the
duplicate write as kmmpd thread should have full control of
reading and writing the mmp block.
Another reason is we will also have random I/O error on
the writeback request when blk integrity is enabled, because
kmmpd could modify the content of the mmp block(e.g. setting
new seq and time) while the mmp block is under I/O requested
by writeback.

Linux-commit: fe18d649891d813964d3aaeebad873f281627fbc

Test-Parameters: testgroup=review-ldiskfs testlist=mmp
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I5aa9fd384a4ea25ee52f1198528fae4ecc9c28c7
Reviewed-on: https://review.whamcloud.com/33038
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-11198 utils: propagate errors for read_param

Steps to reproduce:

 $ lctl get_param -n ost.OSS.ost_io.nrs_tbf_rule
 error: get_param: reading 'ost.OSS.ost_io.nrs_tbf_rule': No such device
 $ echo $?
 0

read_param() might return errors, fix to return errors to
caller.

Change-Id: Id165424342be3109669f173b9b72c16aa8c83425
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/32925
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11414 ptlrpc: don't change buffer when signature is ready

The lm_repsize is part of buffer being used in signature calculation
and must not be changed after calculation is done.

Patch reverts related changes from commit 13372d6c and moves related
lm_repsize update into MDC where DOM read-on-open buffer is prepared

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ic277ca26ad8fb8e23ee7e4b6e81d84a1d51b4733
Reviewed-on: https://review.whamcloud.com/33223
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11420 mdt: export mdt_dom_read_open_fops

Read-on-open MDT lproc fops were removed by mistake in
commit 169738e3, restore lost changes and fix compilation
failures with gcc8

Change-Id: I89de27b76e061b67d175d026a7e563d48df78603
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33224
Tested-by: Jenkins
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11436 build: always dist debian stuff

Always try to dist debian stuff, so that even on non-dpkg-using
build targets (RHEL series), the tar dist will contain all debian
stuff, and can be delivered to users who want to build lustre on
debian series servers.

Test-Parameters: trivial
Change-Id: I8493f5a5e65078c0160c5493efee6906c87db6f1
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Reviewed-on: https://review.whamcloud.com/33246
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Jenkins
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10906 llite: create checksums to replace checksum_pages

Create llite.*.checksums, which matches llite.*.checksum_pages in
functionality. Now the llite layer have something that matches
osc.*.checksums. In time we can retire checksum_pages and change
it to its original purpose of enabling per-page checksums (which
was not implemented in the CLIO development).

Test-Parameters: trivial

Change-Id: I0e381e3626e835a448a6285cb81d27d67af93a54
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33222
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11400 flr: add 'nosync' flag for FLR mirrors

This patch allows 'nosync' flag to be set for FLR mirror components,
which makes lfs mirror resync skip on mirrors with this flag unless
mirror resync explicitly requested those mirrors to be resync.

This flag can be cleared by set '^nosync' on any component of the
mirror.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I209ad91fd9a96d82f8c83a3eb047f665ef4ccf86
Reviewed-on: https://review.whamcloud.com/33205
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11397 idl: remove obsolete RPC flags

Remove RPC flags that are no longer in use:
- OBD_MD_FLQOS has never been used in master branch
- OBD_MD_FLEPOCH unused since v2_7_50_0-38-gd5d5b349f2
- OBD_MD_REINT unused since before 1.6
- OBD_MD_FLMDSCAPA unused since v2_7_55_0-15-g353ef58b1d
- OBD_MD_FLOSSCAPA unused since v2_7_55_0-15-g353ef58b1d

Rename OBD_MD_FLGENER to OBD_MD_FLPARENT to more accurately describe
that this flag is only used to mark the parent FID in the OST obdo.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I7ef8ee5a403a7f4d0184bb279198d93e1198e9de
Reviewed-on: https://review.whamcloud.com/33202
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11380 llite: zero lum for stripeless files

In the IOC_MDC_GETFILEINFO/LL_IOC_MDC_GETINFO case of ll_dir_ioctl(),
if the file has no striping then zero out the lum buffer so that
userspace won't be confused by garbage.

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I4574b8bfdee122ec1ac571ae3714026c6fa0a7f0
Reviewed-on: https://review.whamcloud.com/33172
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>

LU-11381 tests: sanity-flr 201 skip information

sanity-flr test 201 was added to the ALWAYS_EXCEPT list
with no information about why it is being skipped. Add
the JIRA ticket number that documents why test 201
is not being run.

Test-Parameters: trivial testlist=sanity-flr
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I33cb0a51908bdba6a3f3faa35b1e337b4ff41764
Reviewed-on: https://review.whamcloud.com/33171
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>

LU-10899 osc: move obdo_cache to OSC code

The obdo_cache slab is only used by the OSC code today, so it does
not need to be allocated in obdclass on servers.  Move it to only
be allocated when the OSC module is loaded.

Rename obdo_cachep to osc_obdo_kmem to match other slab caches
created by the OSC.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I6f70adb1f5d0e76c72d797a65b413df1533ebbe5
Reviewed-on: https://review.whamcloud.com/33141
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11360 osd-ldiskfs: hold s_umount before calling sync_fs

ldiskfs_sync_fs() calls dquot_writeback_dquots(), which expects
we are holding s_umount since kernel 4.10
Linux-commit: 9d1ccbe70e0b14545caad12dc73adb3605447df0

Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: Icda84eebe0d6261e9e374fb2a005bb66e586bba8
Reviewed-on: https://review.whamcloud.com/33135
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11111 lfsck: print correct parent FID

The LFSCK debug message for repairing a striped directory incorrectly
prints the child FID twice, rather than printing the parent FID:

  MDT0002-osd: namespace repair shard 0 [0x68002f9d1:0xa47c:0x0] ...
           ... of the striped directory [0x68002f9d1:0xa47c:0x0]

Fix this to print the parent FID.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I94b530ca6eaa07ee93bb8154bba8d356ce2f2896
Reviewed-on: https://review.whamcloud.com/32935
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11164 ldlm: pass env to lvbo methods

to save on env allocation.

Benchmarks made by Shuichi Ihara demonstrated 13% improvement
for small I/Os: 564k vs 639k IOPS. the details can be found
LU-11164.

Change-Id: I797e3d7e19ef408993004a2b872842d655240525
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/32832
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10030 clio: don't use spare bits in iattr.ia_valid

Using spare bits in iattr.ia_valid is not safe as the
bits might get used for some other purpose without
us noticing.

Lustre currently used 4 spare bit, but they are all
defined in different places so this isn't immediately
obvious.

This patch brings all those bit definitions together
and defined a new op_data field (op_xvalid) to hold
the extra validity bits.

It also replaces sa_valid in struct cl_setattr_io
with sa_avalid and sa_xvalid.  Changing the name is
helpful as sa_valid already has another use within
lustre.

Modified version of:
Linux-commit: 599b9fb8d424c7907325037b5a862221b1232526

Change-Id: I475a16d557143c0741472e5873c83d71968d7cb8
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32825
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 llite: move /proc/fs/lustre/llite/xattr_cache to sysfs

Move xattr_cache file from /proc/fs/lustre/llite/*
to /sys/fs/lustre/llite/*/

This is a modified version of

Linux-commit: 070c29ca79ef5a3ba753053928371b257064b15f

Change-Id: I0fa7715cd2ae9c90af1540e406c1645a8eee3b2e
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32507
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 llite: move /proc/fs/lustre/llite/*_easize to sysfs

Move max_easize and default_easize files from /proc/fs/lustre/llite/*
to /sys/fs/lustre/llite/*/

This is a modified version of

Linux-commit: f5501d042e5dd2b9932f1cad8ad3a71ec4777926
Linux-commit: 60b65afb700626f0fe43589138c8ca9ac460d634

Change-Id: Ie445d976ba6e21a224adc1894c108a7d30ed3a8b
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32506
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 llite: move /proc/fs/lustre/llite/lazystatfs to sysfs

Move lazystatfs file from /proc/fs/lustre/llite/*
to /sys/fs/lustre/llite/*/

This is a modified version of

Linux-commit: c7fe64bdd0623f5871800ff42842741175f19562

due to the large amount of changes to the OpenSFS/Intel branch.

Change-Id: I279f3361ae7b779bbd186504f5c4072b265330f9
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32505
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8174 misc: name open file handles as such

In a number of places in the code, rename variables from "*_handle"
or "*_fh" to "*_open_handle" so that it is clear this is referencing
an open file handle rather than something else (e.g. a lock handle).

Also rename the confusingly-named mti_close_handle to mti_open_handle,
since this is referencing an open file handle, even if it is used at
close time to close the file.

   mfd_handle2mfd() -> mfd_open_handle2mfd()
   mdt_file_data.mfd_handle -> mfd_open_handle
   mdt_file_data.mfd_old_handle -> mfd_open_handle_old
   mdt_thread_info.mti_close_handle -> mti_open_handle
   mdt_body.mbo_handle -> mbo_open_handle
   mdt_io_epoch.mio_handle -> mio_open_handle
   md_op_data.op_handle -> op_open_handle
   mdt_rec_create.cr_old_handle -> cr_open_handle_old
   mdt_reint_record.rr_handle -> rr_open_handle
   obd_client_handle.och_fh -> och_open_handle

Change the resync code path to use a "lease_handle" to avoid confusion
with an open handle:

   mdt_rec_resync.rs_handle -> rs_lease_handle
   use md_op_data.op_lease_handle
   add mdt_reint_record.rr_lease_handle

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I6e2abdaee1c32d502ea4f2f4b339a6bb21528f69
Reviewed-on: https://review.whamcloud.com/26953
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6081 user: adding llapi_create_volatile_param()

This new function is similar to the existing
llapi_create_volatile_idx() function, but takes 2 additional
parameters: a stripe information, and a file mode.

llapi_create_volatile_idx() is changed to use
llapi_create_volatile_param().

Include a new man page for llapi_create_volatile_param(),
llapi_create_volatile_idx() and llapi_create_volatile_param().

Signed-off-by: frank zago <fzago@cray.com>
Change-Id: Ia50750d68b8cfa335b53b114a20fccaed49c75cd
Reviewed-on: https://review.whamcloud.com/13242
Tested-by: Jenkins
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10806 target: skip discard for a missing obt_lut

If error happens at tgt_init, obt_lut is zeroed. The error handle
path destroys export and calls tgt_grant_discard which access to
non allocated memory.

The patch skips release grant for zero obt_lut.

BUG: unable to handle kernel NULL pointer dereference at
0000000000000378
IP: [<ffffffff816b6b5c>] _raw_spin_lock+0xc/0x30
[<ffffffffc0ff7c15>] ? tgt_grant_discard+0x35/0x190 [ptlrpc]
[<ffffffffc0fcbe2e>] ? tgt_client_free+0x17e/0x3b0 [ptlrpc]
[<ffffffffc1414097>] mdt_destroy_export+0x87/0x200 [mdt]

Signed-off-by: Alexander Boyko <c17825@cray.com>
Cray-bug-id: LUS-6260
Change-Id: Iccd80cb753122727156430e2593340c32841451c
Reviewed-on: https://review.whamcloud.com/33240
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8616 utils: document specific MDT index selection

Document the "lfs setdirstripe -i" ability to specify explicit MDT
index values for striped direcotories.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I23b0b5211677bc935b37ebbbfa0a2999c03ebbe5
Reviewed-on: https://review.whamcloud.com/33216
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11392 llog: fix race llog_process_thread vs llog_add

The llog should be wrapped so it has old data on disk and zero at
bitmap.
  1. llog_process_thread reads part of llog at buffer.
  1. process a last record, checks the next record fields
  2. llog_add adds a record and marks new record at bitmap
  1. check bitmap flag and process the old record from buffer

After LU-7001 small window for the race still exists. It locates
between partial_chunk check and ext2_test_bit.
The patch fixes this race.

Cray-bug-id: LUS-6287
Signed-off-by: Alexander Boyko <c17825@cray.com>
Change-Id: I45a742af0c987aa159a22a3255a4fb9d430b6a84
Reviewed-on: https://review.whamcloud.com/33193
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10177 lfs: support DOM-to-OST migration

Migraton of DoM file to an OST file is possible with
'lfs migrate' command in general but would cause space loss
on MDS, because DOM local inode blocks are not deleted.
Patch doesn the following:
- adds truncate for local MDS inode after migration.
- check and prevent migration from non-DOM to DOM file,
  that will be done later with mirroring.
- prohibit 'lfs swap_layouts' command if either file has
  DOM component, 'lfs migrate' should be used instead

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I1ce7b09eefab5e13f81af30c007475c927bd2874
Reviewed-on: https://review.whamcloud.com/32979
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11128 ptlrpc: new request vs disconnect race

new request can race with disconnect-by-idle process.
disconnect code detect this state and initiate a new connection.

Change-Id: I6acc913c371f2ae63f27151edce457c2a13118c5
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/32980
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11373 libcfs: fix wrong check in libcfs_debug_vmsg2()

Logic here is we skip output if time is before @cdls_next
reach and increase @cdls_count. however we did it in the
opposite way:

1)libcfs_debug_vmsg2() is called for a long time, that means
current check succeed, we skip print messages and return, we
will skip all messages later too..

2)libcfs_debug_vmsg2() is called frequently, current check
fail every time, message will be bumped out always. the
worst case is we never skip any messages.

Also fix test case to cover this later which is from Andreas:

The test_60a() llog test is being run on the MGS, while the
check in test_60b() to confirm that CDEBUG_LIMIT() works properly
is being run on the client.  There has been a breakage in
CDEBUG_LIMIT() that this test failed to catch, now we need to
track it down.

Change test_60b to dump the dmesg logs on the MGS.

Fixes: fdb5d3d50
Test-parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Idb2ecfe5ddadc72d8931e6623b171b9c773ebbe5
Reviewed-on: https://review.whamcloud.com/33154
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-11071 build: create lustre resource package for Ubuntu

The new Ubuntu server support is missing on last package that RHEL
currently supports. Enable Debian/Ubuntu packaging for the resource
agent package.

Test-Parameters: trivial

Change-Id: I872ae6b1b02782155db14c5cc971607ca3e93d56
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33152
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11268 mdc: use smaller buffers for getxattr

In mdc_intent_getxattr_pack() only the xattr values buffer should be
upsized for older MDTs. This refines the interop fix from change
https://review.whamcloud.com/31990 (LU-10912 mdc: use large xattr
buffers for old servers).

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: If7c6fe7be96a6a227c746f90757b7aae1874b14e
Reviewed-on: https://review.whamcloud.com/33024
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 llite: remove redundant lookup in dump_pgcache

Both the 'next' and the 'show' functions for the dump_page_cache
seqfile perform a lookup based on the current file index.  This is
needless duplication.

The reason appears to be that the state that needs to be communicated
from "next" to "show" is two pointers, but seq_file only provides for
a single pointer to be returned from next and passed to show.

So make use of the new 'seq_private' structure to store the extra
pointer.
So when 'next' (or 'start') find something, it returns the page and
stores the clob in the private area.
'show' accepts the page as an argument, and finds the clob where it
was stored.

Linux-commit: 85b6542bbade42935386661cb45d90c4e13766b7

Test-Parameters: trivial envdefinitions="ONLY=63b" testlist=sanity
Test-Parameters: trivial envdefinitions="ONLY=118" testlist=sanity
Test-Parameters: trivial envdefinitions="ONLY=20" testlist=sanityn

Change-Id: I6a45430f7ba3007dcd70bdf09c4c7ed5a213533d
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33010
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11237 build: delete obsolete slapd-lustre.conf

Delete the obsolete slapd-lustre.conf file, as it was only used
for LDAP configuration of clients and servers back before
"zero-conf" (MGS+llog) configuration was available.  Also remove
the use of openldap in Makefile.am.

This has been obsolete since Lustre 1.4 or so.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If99ca0a62a1753ead1b29d217bc2437bd29e0850
Reviewed-on: https://review.whamcloud.com/32989
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11169 obdclass: fix old return code usage

Commit 49d8a7ccd73 from LU-9855 introduce a fail in the
obd_commitrw function which parent error code
replaced with result of exp_check_ops function.

Cray-bugid: LUS-6195

Test-Parameters: trivial
Change-Id: If8643ba39232f94ba9c523fdb9f10c1411a1838a
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-on: https://review.whamcloud.com/32906
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11020 osd: use right sync

use a sync_fs instead of journal flush only.
ext4 commit with similar change

commit 06a407f13daf9e48f0ef7189c7e54082b53940c7
Author: Dmitry Monakhov <dmonakhov@openvz.org>
Date:   Wed Jun 12 22:25:07 2013 -0400

ext4: fix data integrity for ext4_sync_fs

Inode's data or non journaled quota may be written w/o jounral so we
 _must_ send a barrier at the end of ext4_sync_fs. But it can be
skipped if journal commit will do it for us.

Cray-bug-id: LUS-5993
Signed-off-by: Alexey Lyashkov <c17817@cray.com>
Change-Id: Ie242b48d0b8ad61cf4d38ac0fb173176193716d9
Reviewed-on: https://review.whamcloud.com/32866
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 ofd: migrate from proc to sysfs

Move the ofd module from using proc for most single value files
to sysfs. Leave the more complex proc entries for now. Migrate
the sysfs and most of the proc handling to lproc_ofd.c.

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: Ief21636a0dce3fd4946f9a6971a8b09c2812200e
Reviewed-on: https://review.whamcloud.com/33059
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6864 mdc: move RPC semaphore code to lustre/osp

The "MDC RPC semaphore" is no longer used by MDC code since patch
http://review.whamcloud.com/14374 "LU-5319 mdc: manage number of
modify RPCs in flight" landed.  It is only still used by the OSP
currently.  While there are plans to remove this from the OSP as
well, it makes sense to move all of this code from MDC to OSP so
that it will also be cleaned up when that functionality lands.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I1c4c9a45f308a29600631ac7fed40e592b3ebbe5
Reviewed-on: https://review.whamcloud.com/32412
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8130 ptlrpc: convert conn_hash to rhashtable

Linux has a resizeable hashtable implementation in lib,
so we should use that instead of having one in libcfs.

This patch converts the ptlrpc conn_hash to use rhashtable.
In the process we gain lockless lookup.

As connections are never deleted until the hash table is destroyed,
there is no need to count the reference in the hash table.  There
is also no need to enable automatic_shrinking.

Linux-commit: ac2370ac2bc5215daf78546cd8d925510065bb7f

Change-Id: I576daf314c3ac31a58df02d731292e1e8bb408c6
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32036
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-4684 lmv: support accessing migrating directory

Migrating directory contains stripes of both old and new layout, and
its sub files may be located on either one. To avoid race between
access and new creations, there are 4 rules to access migrating
directory:
1. always create new file under new layout.
2. any operation that tries to create new file under old layout will
   be rejected, e.g., 'mv a <migrating_dir>/b', if b exists and is
   under old layout, this rename should fail with -EBUSY.
3. operations that access file by name should try old layout first,
   if file doesn't exist, then it will retry new layout, such
   operations include: lookup, getattr_name, unlink, open-by-name,
   link, rename.
4. according to rule 1, open(O_CREAT | O_EXCL) and create() will
   create new file under new layout, but they should check existing
   file in one transaction, however this can't be done for old
   layout, so check existing file under old layout on client side,
   then issue the open/create request to new layout.

Disable sanity 230d for ZFS backend because it will trigger lots of
sync, which may cause system hung.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Icd587eb17f4c6dc2234dcc3c7ba40759bc3e8d31
Reviewed-on: https://review.whamcloud.com/31504
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-5026 obdclass: Add lbug_on_eviction option

Add an lbug_on_eviction sysfs interface.  When it is set to a non-zero
value on a client, it will cause the client to LBUG whenever it is
evicted by the server. Note, an MDS is a client to OSTs, and every
server is a client of MGS. Thus, it is probably desireable to leave
this set to zero on servers.

Change-Id: I2f694e4bc1b1ec9daf52f8a6ac46ad87f40f810f
Cray-bug-id: LUS-2591
Signed-off-by: Ryan Haasken <haasken@cray.com>
Signed-off-by: Chris Horn <hornc@cray.com>
Reviewed-on: https://review.whamcloud.com/10257
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11329 utils: add get_maintainer.pl script

The get_maintainer.pl script can be given a patch file or list of
pathnames and will look at the MAINTAINERS file for different code
subsections and the developers that are maintaining those sections.
If no maintainer is found, it will also check the git commit history
and "git blame" to see which developers were active on the specified
file(s) and use them.

This patch adds a copy of the get_maintainers.pl and MAINTAINERS files
to the Lustre tree to make it easier to find reviewers for the code.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I5f2e717227a7c86c924e7edb889e087c7c3ebbe5
Reviewed-on: https://review.whamcloud.com/33104
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11128 ptlrpc: add debugging for idle connections

Add a "debug" parameter for the idle client disconnection so that
it can log disconnect/reconnect events to the console.

Print the idle time in the "import" file.

Enable the connection debugging for all test runs.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I47a4f73542fb4864e4f35275ecfe8907493ebbe5
Reviewed-on: https://review.whamcloud.com/33168
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11301 target: add lock in sub_trans_stop_cb()

sub_trans st_committed and st_stopped flags take different bit of
the same memory address, so both of them should be set with lock,
but sub_trans_stop_cb() doesn't, which may cause it overriden.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ic3d7d7281b3cf9bd20702be944e14f35200318f1
Reviewed-on: https://review.whamcloud.com/33169
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11372 o2iblnd: add clarifying comment to ko2iblnd.conf

The settings in ko2iblnd.conf are recommended only for
OPA but if other IB cards are present along with
OPA, then these settings are applied across all the
configured IB interfaces.

Change-Id: I6daf1f59a100db4e9c2cd12c255ad33fda274093
Test-Parameters: trivial
Signed-off-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33163
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11071 ldiskfs: fix ldiskfs patch for ubuntu18

the original ext4-corrupted-inode-block-bitmaps-handling-patches.patch
missed the changes to ext4_validate_inode_bitmap, add it back

Test-Parameters: trivial
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I42ff97be5c5680be04d69e0e818b622df6bd400d
Reviewed-on: https://review.whamcloud.com/33144
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10899 obdclass: remove unused ll_import_cachep

The ll_import_cache is not used anywhere, and can be removed.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I3f43c81ffbf1385117f9ba1f8f9881388a3ebbe5
Reviewed-on: https://review.whamcloud.com/33119
Tested-by: Jenkins
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11303 out: clean up osp_update_rpc_pack() macro

The osp_update_rpc_pack() macro was using both an operation "name"
argument as well as an "opcode" argument, but totally ignoring the
"opcode" argument.  This allowed a caller in osp_attr_set() to pass
a non-existent opcode name to the function that was ignored.

The function also concatenated the passed name string to generate
a function name to be called (which supplied its own opcode value).
This name concatenation obscures the code functionality, since the
called functions are not actually referenced directly in the code.

Instead, pass the function name as the argument, and drop the opcode
completely.  This at least makes it possible to see where the function
is being called.  Change the macro to be upper-case, so that it is
more clear that it is a macro rather than a real function.  It would
be better to replace the macro with an actual function, but this is
not very practical due to the use of variable-length argument lists.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I7bf5c1d0f7a9abab1d8de8046cc408961f3ebbe5
Reviewed-on: https://review.whamcloud.com/33107
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 llite: use more private data in dump_pgcache

The dump_page_cache debugfs file allocates and frees an 'env' in each
call to vvp_pgcache_start,next,show.  This is likely to be fast, but
does introduce the need to check for errors.

It is reasonable to allocate a single 'env' when the file is opened,
and use that throughout.

So create 'seq_private' structure which stores the sbi, env, and
refcheck, and attach this to the seqfile.

Then use it throughout instead of allocating 'env' repeatedly.

Linux-commit: fc5f499af16c247c5bff68d2625a6d1ca48b3332

Test-Parameters: trivial envdefinitions="ONLY=63b" testlist=sanity
Test-Parameters: trivial envdefinitions="ONLY=118" testlist=sanity
Test-Parameters: trivial envdefinitions="ONLY=20" testlist=sanityn

Change-Id: Id7e7f5ec1729d1e5af041adcb247ca11299ed734
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33009
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11133 dkms: Add missing ldiskfs module in dkms config

The dkms.conf file generated by the script dkms.mkconf does not
include the ldiskfs kernel module. This results in not being able to
use ldiskfs when using the lustre-ldiskfs-dkms or lustre-all-dkms
packages.
This patch adds the module in the configuration for both of these
packages.

Test-Parameters: trivial
Signed-off-by: Sebastien Piechurski <sebastien.piechurski@atos.net>
Change-Id: Icc98fba208c78c3061fe591e3315af1eb9199f4f
Reviewed-on: https://review.whamcloud.com/32800
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 llite: move /proc/fs/lustre/llite/statahead_{max, agl} to sysfs

Move statahead_max and statahead_agl files from /proc/fs/lustre/llite/*
to /sys/fs/lustre/llite/*/

This is a modified version of

Linux-commit: 4081a3201ade8b3be715ebb0b7879a2bfe5fcc98

due to the large amount of changes to the OpenSFS/Intel branch.

Change-Id: Ic16d48d0cf1baabb462952de9739544923eba8cd
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32504
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 llite: move /proc/fs/lustre/llite/stats_track* to
 sysfs

Move stats_track_pid, stats_track_ppid and stats_track_gid files
from /proc/fs/lustre/llite/* to /sys/fs/lustre/llite/*/

This is a modified version of

Linux-commit: 62cf4b1315e86a1f373f77acf32a23d76428ba27

due to the large amount of changes to the OpenSFS/Intel branch.

Test-Parameters: trivial

Change-Id: I29dc2dd269f66e40a7d016ba366353ae2ca7fb51
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32503
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10124 lnet: Correctly add peer MR value while importing

while adding peer using lnetctl import, the MR value of the
peer is not correctly imported.

Checks for MR value other than True/False in
handle_yaml_config_peer() -
1. No value provided - Use default as True
2. Value other than True/False - Error out

Change-Id: I02a21e35086f1c6f29081b464dd1a63aba692cbc
Test-Parameters: trivial
Signed-off-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-on: https://review.whamcloud.com/31138
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-4684 migrate: shrink dir layout after migration

Use setxattr(XATTR_NAME_LMV) to shrink dir layout after migration,
this may change dir fid because it may shrink dir to 1-stripe, if so
replace this dir with its stripe.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I4fd3e83e188ac0c6c4845183c17701276fae94a8
Reviewed-on: https://review.whamcloud.com/31626
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-4684 migrate: migrate striped directory

Migrate striped directory in below steps:
1. create target object if needed: if source is directory, a
   target object is always created, otherwise if source is
   already located on the target MDT, or source still has
   link on source MDT, then skip creating.
a) if source is directory, detach source stripes and
   attach them to target.
b) migrate source xattrs to target.
c) if source is regular file, update PFID to target
   fid.
d) update fid to target for all links of source
2. update namespace
a) migrate dirent from source parent to target parent.
b) update linkea parent fid to target parent.
c) destroy source object.

This implementation improves following fields:
1. all involved objects are locked to avoid race.
2. directory migration doesn't migrate its dir entries, instead
   it's done in each sub file migration, this avoids timeout in
   migrating dir entries for large directory, and also avoids
   touching dir entries without lock.
3. file/dir is migrated in one transaction, so migrate recovery
   is the same as others.
4. migrating directory can be accessed (modifiable) like normal
   directory.
5. if migration of sub files under a directory fails, user can
   redo migrate to finish migration of this directory.

Disable migrate in racer.sh.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I7906e50a0bf55375eafdf2cf5500979dd2d94d48
Reviewed-on: https://review.whamcloud.com/31427
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11016 obdclass: change JobID error into warning

Change the "jobid value too large" message from a console error into a
warning, since it doesn't really indicate incorrect Lustre behaviour.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ibde3b686061fc7b4daa48dae5623e60ad157e5c8
Reviewed-on: https://review.whamcloud.com/33128
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11349 tests: remove insanity test 15

Test 15 of the insanity test suite has not been used
in testing since 2003 and tries to run a script that no
longer exists. Thus, this test needs to be removed.

At the same time, clean up the script in converting
spaces to tabs.

Test-Parameters: trivial testlist=insanity
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: Ie3b2f88dde925d7a1c53f288215b3643d20fd5ff
Reviewed-on: https://review.whamcloud.com/33121
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11281 ptlrpc: race in AT early reply

In ptlrpc_at_check_timed, the refcount of the request could
be already dropped to zero, the ptlrpc_server_drop_request
could continue without the "scp_at_lock" and free the request
by writing 0x5a5a5a5a5a5a5a5a to the memory, but the following
"atomic_inc_not_zero(&rq->rq_refcount)" will return nonzero and
cause freed request to be used in ptlrpc_at_send_early_reply.

Change-Id: I5d884be86de007f49b044e022ad90663b08078d7
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33071
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11010 tests: remove calls to return after skip()

The skip routine now contains a call to exit. All calls
to skip() and skip_env() should be reviewed and calls to
return() that follow skip() should be removed.

This is the third patch in a series that removes calls
to return() after skip() in the Lustre test suites.

Calls to return after skip() are removed for:
large-lun.sh
large-scale.sh
lfsck-performance.sh
lnet-selftest.sh
lustre-rsync-test.sh
mds-survey.sh
metadata-updates.sh

Test-Parameters: trivial testlist=large-lun,large-scale,lfsck-performance,lnet-selftest,lustre-rsync-test,mds-survey,metadata-updates

Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: Id11dc4bbde5824976a5a4d8f94ea59efd45e46f4
Reviewed-on: https://review.whamcloud.com/32732
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11165 llite: set iflags and catch error from md_setattr() properly

As John Hammond pointed out:
In ll_ioctl_fssetxattr() we may discard errors from md_setattr().
And we only set the local i_flags when lli_clob is not NULL.

This patch tried to address above problems.

Test-Parameters: trivial testlist=sanity-quota
Change-Id: Ib65ec61932b00f71367a8da6be67e3f94ebd5005
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/32860
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9268 tests: check obj distribution for PFL striping

Add test to sanity-pfl to verfy that new components are
distributed evenly among OSTs for composite files.

Test-Parameters: trivial osscount=3 ostcount=4 testlist=sanity-pfl
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I0335fd376f6008529885236faad19a29eb01d5ba
Reviewed-on: https://review.whamcloud.com/26863
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10660 mdt: revoke lease lock for truncate

Lustre lease lock is usually used to protect file data
against concurrent access. Open lock used on MDT side
is for this purpose. However, truncate will change
file data but it doesn't revoke lease lock.

This patch fixes the issue by acquiring open sem,
checking lease count and revoking lease if there exists
any pending lease on the file.

Change-Id: Ia55457c6d7e1e76f98b41feef39577822304513b
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33093
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

New tag 2.11.55

Change-Id: I08a290fd7cd909cf24405daf412fe9a0dd4a8757
Signed-off-by: Oleg Drokin <green@whamcloud.com>

Revert "LU-11057 tests: exercise 'network' client mount option"

This is causing 100% failures in review-dne-zfs-part-2
which are tracked in LU-11363

This reverts commit 76af9f62481b1ac910cd6a0f7237fe32b0657c29.

Change-Id: Ie82f848ce1f170b2c93f106c5886deadf7b4df75
Reviewed-on: https://review.whamcloud.com/33139
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Oleg Drokin <green@whamcloud.com>

LU-10447 build: fix checkpatch.pl to not mangle case

Fix the checkpatch.pl script to not mangle the case of the lines
in the spelling.txt file, so that the replacement text can be
found, and is shown as intended.

Add checks for "GETSTRIPE" and "SETSTRIPE" to spelling.txt.

Sort the spelling.txt file so that it is easier to see what lines
are present in the file.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Id3a3319bc62898fbdc94573512cd6a665b13cd11
Reviewed-on: https://review.whamcloud.com/32865
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 llite: move /proc/fs/lustre/llite/checksum_pages to sysfs

Move checksum_pages file from /proc/fs/lustre/llite/*
to /sys/fs/lustre/llite/*/

This is a modified version of

Linux-commit: 40cc864a9fcd48d7f3a67421a8e578e8d4cbe308

due to the large amount of changes to the OpenSFS/Intel branch.

Change-Id: Ice48c69503680ac995b60844a8d0d7ae2bddd102
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32502
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-11012 doc: improve the lfs-setstripe.1 man page

Restructure the lfs-setstripe.1 man page to be more like a standard
man page.  The DESCRIPTION section provides a high-level overview of
the command, and describes the various different ways that it can be
used.  The various options are described in the STRIPE_OPTIONS and
COMPONENT_OPTIONS sections.

Describe how default layouts are inherited for plain and composite
files, and that different pools can be specified for each component.

Add description of the --copy option.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I7ba3f3f4dba6825f283dd6874ae1988914c774fb
Reviewed-on: https://review.whamcloud.com/32463
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Joseph Gmitter <jgmitter@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11309 lnd: Clean up logging

No need to output error in ksocknal_tx_done() as this error is
tracked in lnet.
No need to keep a cookie in the connection. It's always set to
the message. This will allow us to set the msg's health status
properly before calling lnet_finalize()

Test-Parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I168c8ca32192ba4f8f4480e8e7384d45f0a062a7
Reviewed-on: https://review.whamcloud.com/33096
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11283 lnet: fix setting health value manually

Changed the health value option letter to 't' instead
of 'h' to avoid conflict with help option.
Don't increment optind unnecessarily.
In the handling code in api-ni.c there was a missing return.

Test-Parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: Ib56f19d1781e44297158c64c5b26004220a1db21
Reviewed-on: https://review.whamcloud.com/33074
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11245 flr: lfs mirror dump command

Adds "lfs mirror dump" command to dump a mirror's content of a
mirrored file.

Usage:

lfs mirror dump <-N|--mirror-id <mirror_id>>
[-o|--outfile <output_file> ] <mirrored_file>

Options:

--mirror-id <mirror_id>
  Specify the mirror by mirror_id to dump its content.
--outfile <output_file>
  Dump the content into file output_file, if not specified, the
  content will be dump to the standard output stream.

Test-Parameters: testlist=sanity-flr
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I2846b1c1968d9cc302c07b57a05687af50530d53
Reviewed-on: https://review.whamcloud.com/32997
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11057 tests: exercise 'network' client mount option

Test 'network' client mount option that enables to limit connections
from the client to be on the network NID specified.

This option is currently not compatible with Dynamic Peer Discovery.
test_31 will have to be updated once Dynamic Peer Discovery is
modified so that it does not conflict with 'network' client mount
option.

Test-Parameters: trivial testlist=sanity-sec
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I4631a034137a346d0bb4efed20983e2a8bd7b0ec
Reviewed-on: https://review.whamcloud.com/32590
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10961 ldlm: don't cancel DoM locks before replay

Weigh a DOM locks before lock replay like that is done
for OSC EXTENT locks and don't cancel locks with data.

Add DoM replay tests for file creation and write cases.

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: If706835455c2b25c01cfcde033fccc4f3c67ee24
Reviewed-on: https://review.whamcloud.com/32791
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11270 mdt: better DOM data discard

Discard DOM data when all object are put and unlocked.
So current thread will wait for discard but resources are
not blocked by it. Also cancel discard lock immediately.

Patch also add missing part for lock prolongation for DoM
locks and fix for tgt_blocking_ast() to don't miss a call to
ldlm_server_blocking_ast() in case of lu_env_init() failure.

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I09aaa08e28d2a280ca1f9939315311c45d0334b5
Reviewed-on: https://review.whamcloud.com/33026
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9855 include: replace OBD_CHECK_DEV by obd_check_dev

Static inline functions are preferred over macros. Hence, the function
obd_check_dev was introduced. obd_check_dev replaces the macro
OBD_CHECK_DEV. All functions that call obd_check_dev store the return
values and return them if they represent an error code.

Some of the changes were carried out manually while others were done
using coccinelle.

Linux-commit: 4ca1b8fbe609b7c52aeaa78a5f4678efcc9b8d73

Test-Parameters: trivial

Signed-off-by: Aya Mahfouz <mahfouz.saif.elyazal@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: Iad96cc151a711892d950bd2ccaa93b13d6b23ae5
Reviewed-on: https://review.whamcloud.com/33101
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-11328 ldlm: fix compile problem with --disable-lru-size

add the missing type change for ldlm_pool_recalc.

Test-Parameters: trivial
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I5b96dbe79e9b202888cf0f23a947c2d6c0f0d8d8
Reviewed-on: https://review.whamcloud.com/33103
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-10740 tests: disable tests for replay-dne-zfs-part-4

Disable a few tests, by adding them to the ALWAYS_EXCEPT list,
that are constantly failing for replay-single so that
review-dne-zfs-part-4 might normally pass.

A potential fix for replay-ost-single test_6 and test_7 to calculate
the log size for DNE+ZFS systems more accurately for multiple MDTs.

Test-Parameters: trivial testgroup=review-dne-zfs-part-4
Test-Parameters: trivial testgroup=review-dne-zfs-part-3
Test-Parameters: testlist=sanity-sec
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I2316baeacbb18686d2958265f7128e95033ebbe5
Reviewed-on: https://review.whamcloud.com/33106
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11304 misc: update all url links to whamcloud

Even old links could redirect to whamcloud automatically,
We'd better update to use new whamcloud links to
avoid any further confusions.

Test-parameters: trivial
Change-Id: Ida7161a062d822141bf0c1fdf20b2098a21ea9e7
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/33094
Reviewed-by: Peter Jones <pjones@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-11279 lod: reset ostlist properly in lod_get_default_lov_striping

Ostlist might be allocated previously, and we should
reset them properly, otherwise, it will pollute new
default setting and cause unexpected behavior.

Test-Parameters: trivial testlist=sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity
Change-Id: I9b7acb5f05ec4b371da99f68b9647f0b75cd7021
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/33069
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11006 lnet: fix show peer yaml tree with no peer

When no peer exists then the root created for the peer
yaml tree should be deleted. And lnetctl show peer
should not display anything.

Currently lnetctl peer show shows the root string "peer"
even when there is no peer. This create issues when
starting lnet using /etc/lnet.conf derived from the
existing configuration.

Change-Id: Ie310a49e60386b579b48898b032467b1bc112da9
Test-Parameters: trivial
Signed-off-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-on: https://review.whamcloud.com/32320
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8130 libcfs: prepare rhashtable support

Linux has a resizeable hashtable implementation in lib,
so we should use that instead of having one in libcfs.
In the process we gain lockless lookup which should be
a performance boost. All modern distributions Lustre
support has rhashtable support but a few pieces are
missing for systems running a 4.4 kernel. The other
target platforms have the full implementation we need.

Test-Parameters: trivial

Change-Id: I63d5b7dae9d52eed12dbefed8ca6062af33efd30
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32102
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-10654 lnet: lnetctl doesn't error out on mistyped options

Running lnetctl command to add/delete peer/net/route should
error out with mistyped options.

This patch add the changes in lnetctl.c to make lnetctl
error out withmistyped options.

Change-Id: Ib8ae54bea919d6ff235b4ca3a23807a809f8962c
Test-Parameters: trivial
Signed-off-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-on: https://review.whamcloud.com/31265
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11273 lnet: update logging

Add the retry count when logging message sending/resending.
Make timed out responses visible on net error.
Log cases when a message is not resent

Test-Parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I0908d495c8ba54754fa77b0fc3b5df59317bb2e8
Reviewed-on: https://review.whamcloud.com/33044
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11195 lod: Mark comps cached on replay of layout change

Replay of a layout change request on a PFL file leaves the object
in an unexpected state: Some components can have llc_stripe set
but ldo_comp_cached is not set in the object. The next layout
change request on the same object will LBUG when it tries to free
the comp entries.

The fix is to set ldo_comp_cached on replay so subsequent layout
change requests will use the in memory components rather than
fetching them from disk.

Signed-off-by: Ann Koehler <amk@cray.com>
Change-Id: I8eaee5614c7f2f6e6a3f2c51de93a65422a3122b
Reviewed-on: https://review.whamcloud.com/32904
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10818 obdecho: don't set ma_need in echo_attr_get_complex()

echo_attr_get_complex() copies ma_need to a local variable, masks
MA_* values other than MA_INODE if MA_INODE is set in ma_need,
and restores the saved value of ma_need before the function exits.
This does not seem to be useful, and triggers an assertion in
echo_big_lmm_get() when MA_LOV and/or MA_LMV is set in ma_need.

Signed-off-by: Nikitas Angelinas <nangelinas@cray.com>
Cray-bug-id: LUS-6252
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Change-Id: I3f5a01b57bdd83937f19fd1fa392b53f7b316455
Reviewed-on: https://review.whamcloud.com/33097
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11057 obd: check '-o network' and peer discovery conflict

"-o network=net" client mount option is not taken into account
when LNet dynamic peer discovery is active.
Check if LNet dynamic peer discovery is active on local node. If it
is, return error if "-o network=net" option is specified.

This patch will have to be reverted when the incompatibility between
"-o network=net" client mount option and LNet dynamic peer discovery
is resolved.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I0520e58b22b7adecf797fbd351506c2f8712dc85
Reviewed-on: https://review.whamcloud.com/32562
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11272 lnet: router handling

Re-create the md and mdh if the router checker ping times out.
When re-transmitting a message do so even if the peer is marked down
to fulfill the message's retry quota.

Test-Parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I7b2a1ec6602dac9a112f4d318b0512f68f923969
Reviewed-on: https://review.whamcloud.com/33043
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11271 lnd: conditionally set health status

For specific error scenarios a more accurate health status is set
per transmit. These shouldn't be overwritten in
kiblnd_txlist_done()

Test-Parameters: trivial
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I6c3ad6660aa654d32e823b29ebe3aedb9fc5508e
Reviewed-on: https://review.whamcloud.com/33042
Tested-by: Jenkins
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11275 llite: check truncate race for DOM pages

In ll_dom_finish_open() check vmpage mapping still
exists after locking and exit otherwise. This can
happen if page has been truncated concurrently.

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ib6ef551673a40ad99baaa9bd620225c65ce34454
Reviewed-on: https://review.whamcloud.com/33087
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-11282 osd-zfs: drop cache immediately

if this is requested via:
  lctl set_param osd-zfs.*.readcache_max_filesize=<bytes>

dropping cache at read is almost free, but may take few
cycles at write as we have to find corresponding dbufs.

Change-Id: I107fc1bf5a8d7655da4054048ff07d3dffa9d4d8
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33072
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9855 obdclass: simplify md_stats code

The md_stats code is layered in many levels of macros that make
the code difficult to read as well as introduce undetected
errors. This peels away the macro wrappers by replacing it with
the function lprocfs_exp_count_increment() which doesn't care
about the order of the function pointers in struct md_ops. The
other change is macros used for initializing the counters. This
is done by replacing lprocfs_init_mps_stats() with very simple
handling in lprocfs_alloc_md_stats().

Change-Id: I036ce4518ffb08d53e2d27bcdea564a4c799181d
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32822
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11110 ofd: remove obdfilter.*.* symlinks in few releases

Add a #if LUSTRE_VERSION_CODE < OBD_OCD_VERSION(2, 14, 53, 0) check
around the obdfilter.*.* symlinks creation code to keep them in
place for another few releases, so that old test scripts that use
them will not break, then remove them.

Change-Id: I703c7ec3af8434b0de8b7cbed19c2c32611f6b18
Signed-off-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/32752
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11075 ldlm: correct logic in ldlm_prepare_lru_list()

In ldlm_prepare_lru_list() fix an (x != a || x != b) type error and
correct a use after free.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I4e34e531260295805c4461e7d8d98675400f1148
Reviewed-on: https://review.whamcloud.com/32660
Tested-by: Jenkins
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10365 tests: set proper paths headers for sanity test 400a

For the case when /usr/include/lustre doesn't exist sanity test
400a attempts to use the lustre user land headers located in the
source tree. Some of the lustre user land headers are wrappers
around the UAPI headers so we need to include those paths as well.

A test move was done in the linux kernel that moved the UAPI headers
to their proper place. Errors were reported mainly due to
linux/types.h being missing. This could be the reason Ubuntu18 fails
the sanity 400a test.

Test-Parameters: trivial clientdistro=ubuntu1604 testlist=sanity envdefinitions=ONLY=400a

Change-Id: If17da7d9fc4cedb3b9c18feaafbee47d1f94d49b
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/31737
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 obdclass: Fix style issues for lu_ref.c

This patch fixes issues reported by checkpatch
for file lustre/obdclass/lu_ref.c

Change-Id: I8733fcac454685704b327219ba4afb096d3943c3
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33081
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-6142 obdclass: Fix style issues for lustre_handles.c

This patch fixes issues reported by checkpatch
for file lustre/obdclass/lustre_handles.c

Change-Id: I6e6ad8c56e225dcdd3707bf5f3b233eda3f90320
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33080
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-6142 obdclass: Fix style issues for lustre_peer.c

This patch fixes issues reported by checkpatch
for file lustre/obdclass/lustre_peer.c

Change-Id: I6cf95dfdd709974cae62626ac50a3507588f425d
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33079
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-11255 kernel: kernel update [SLES12 SP3 4.4.143-94.47]

Update SLES12 SP3 kernel to 4.4.143-94.47.

Test-Parameters: mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs \
clientdistro=sles12sp3 ossdistro=sles12sp3 mdsdistro=sles12sp3 \
testgroup=review-ldiskfs

Change-Id: I8b2c99c9a65149f1b149fa91351970034d6f7a47
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33065
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11125 ofd: decrease message level

The "destroys_in_progress already cleared" message
in ofd_create_hdl() may be result of high load on OST
server prior failover. It is not an error, so decrease
its level to D_HA from D_ERROR.

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Id5142672a61244a6362be3778d0769baafc87b86
Reviewed-on: https://review.whamcloud.com/32985
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>

LU-10509 mdd: don't set size attr for DOM file

When client does truncate it calls ll_md_setattr() followed by
ll_setattr_ost() to set size on OSTs. With DOM file that causes
setattr on MDT first including size then PUNCH RPC on the same
object. That was considered as non-optimized situation and
LU-11033 is intended to improve it, but with ZFS there is
check in OSD which does no truncate if size is the same already.
Therefore real file blocks are not truncated actually so sparse
write beyond the end of file will get old data in hole instead of
zeroes.

Quick patch checks if mdd_attr_set() is going to set SIZE attr for
DOM file and clear LA_SIZE bit, assuming there will be truncate.

Complete solution for this will be implemented under LU-11033

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I47873dccf4270e5f0338f7b6696aa5969cfb9444
Reviewed-on: https://review.whamcloud.com/33008
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10018 protocol: MDT as a statfs proxy

MDT can act as a proxy for statfs data. this should
make df faster (RTT vs RTT*(#MDTs+1)) and enable
idling connections so that clients don't connect to
each OST just to report statfs data. the protocol
has been changing slightly to let MDT differentiate
self and aggregated statfs.

also, obd_statfs has got a new field "granted" where
OST reports how much space has been granted to the
requesting MDT so that space can be added to available
space.

client's NID is used to distribute MDS_STATFS among
MDTS.

Change-Id: I59e03cb5abf809ae8820f874ec51dd2b74e1806c
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/29136
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8708 osc: enable/disable OSC grant shrink

Add an OSC proc interface to enable/disable client's grant shrink
feature.

lctl get_param osc.*.grant_shrink
lctl set_param osc.*.grant_shrink={0,1}

Change-Id: I7974b3bf1c4f9c294dd0d4871d09b1a2e45a8d78
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/23203
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>