LU-11820 tests: skip conf-sanity test_106 for SLOW=no

This test currently takes 40 minutes to run, so it should be
skipped for SLOW=no test runs.

Test-Parameters: trivial testlist=conf-sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I120747e21d046bd28667e2ff26013ae51f9cab07
Reviewed-on: https://review.whamcloud.com/33902
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Artem Blagodarenko <c17828@cray.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11606 utils: Call llapi_error() for errors in liblustreapi files

Change calls to remaining printf() and fprintf() to use
llapi_printf() and llapi_error() so that it is possible
to override these calls when liblustreapi is linked into
some other application.

Change-Id: I6531d29b05b34766a68dabf17299cd00babe2517
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33877
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-8066 lnet: properly isolate kernel_param_ops

Don't reference kernel_param_ops if not available.  There is
already a HAVE_KERNEL_PARAM_OPS configure check for this, but
it is just misplaced.

Fixes: 7092309f32516cbfb95a964c87b8030129edeb27

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I56f86257c6bc8a9c53c7901bc2765e10587cab07
Reviewed-on: https://review.whamcloud.com/33874
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11771 osd: avoid use of HZ in brw_stats

Don't use HZ when computing the brw_stats IO time.  This brw_stats
field was formerly in jiffies, but since the conversion to ktime_t
(now always in msec) the output is incorrect when HZ != 1000.

Fixes: 4c2f028a95063aa30006a071a039e8a3eefe52f2

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ia01705b94ddedec2e375621c6e01a957f53ebbe5
Reviewed-on: https://review.whamcloud.com/33857
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11757 lod: use calculated stripe count

lod_prep_md_striped_create() tries to allocat big
chunk of memory because
lum->lum_stripe_count == -1 and is converted to __u32.

ldo_dir_stripe_count was calculated already in lod_ah_init()

Change-Id: Id99d9e024638dfb1b34262840d2e543c808a9cdc
Cray-bug-id: LUS-6694
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Tested-by: Elena Gryaznova <c17455@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-on: https://review.whamcloud.com/33829
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11736 utils: don't set max_sectors_kb on MDT/MGT

The max_sectors_kb tunable should not be applied to MDT and MGT
devices. This tuning is needed for efficiency of large IOs for
spinning disks, but is not needed for SSDs or regular IO. It can
cause problems with DM Multipath configurations for minimal
benefits, so should be limited to OST devices.

This only applies to ldiskfs backend filesystems, no such tuning
is currently done for any ZFS devices.

Test-Parameters: trivial testlist=conf-sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I496603da13aae042f63cc37c0dea221a393ebbe5
Reviewed-on: https://review.whamcloud.com/33796
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10070 tests: Fix replay-single test_85b

test_85b of replay single sets a default striping on $DIR
and does not remove it.  This makes it impossible to
correctly test self-extending layouts, so fix this first.

cray-bug-id: LUS-2528
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I0057c8403e3dae2437cf0c8810af8086e2971c35
Reviewed-on: https://review.whamcloud.com/33777
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 sysfs: make ping sysfs file read and writable

Starting with 4.15 kernels any sysfs read only is limited to
root access only. To retain the ability for non root users
to detect if a remote server is alive using the 'ping' sysfs
file we need to change it to writable. Retain the read ability
so older tools will work.

Change-Id: I6560c119328d723a20a2b32e1fa8c68dce5d407a
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33776
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>

LU-11719 ldlm: Adjust search_* functions

The search_itree and search_queue functions should both
return either a pointer to a found lock or NULL.

Currently, search_itree just returns the contents of
data->lmd_lock, whether or not a lock was found.

search_queue will do the same under certain cirumstances.

Zero lmd_lock in both search_* functions, and also stop
searching in search_itree once a lock is found.

cray-bug-id: LUS-6783
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: Ie231166756e60c228370f8f1a019ccfe14dfda6a
Reviewed-on: https://review.whamcloud.com/33754
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11717 tests: exit_status () defect

exist_status() should search a word FAIL, otherwise
all the tests skipped due to reason which contains
FAILOVER_MODE have status = 1 and exited with the error.

Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-6711
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Change-Id: I5a69e0a4aed6cc8d75d97fe83c6f304b666701ea
Reviewed-on: https://review.whamcloud.com/33751
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11415 socklnd: improve scheduling algorithm

Modified the scheduling algorithm to use all scheduler threads
available. Previously a connection is assigned a single thread
and can only use that one. With this patch any scheduler thread
available on the assigned CPT can pick up and work on requests
queued on the connection.

Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I7e8e5220e1914155fd793994485e3a443fe1ba44
Reviewed-on: https://review.whamcloud.com/33740
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11700 osp: GPF in osp_reserved_mb_low_seq_write()

opd_pre pointer may be uninitialized on statrup

Change-Id: If6ff4289545503e07719c7e9a282be1bd393465b
Cray-bug-id: LUS-6118
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-on: https://review.whamcloud.com/33717
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11699 lfsck: Umount while running LFSCK

lod_device_free() ASSERTION( atomic_read(&lu->ld_ref) == 0)failed

Wake up master thread after releasing all refernces.

Change-Id: I2933f7c58c12f4e3cd2510b01babbf02814c8768
Cray-bug-id: LUS-6510
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-on: https://review.whamcloud.com/33715
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11696 utils: "lfs getsom" returns "24" to userspace

The "lfs getsom" command always returns "24" to userspace because
"rc = 24" (sizeof(struct lustre_som_attrs)) after fetching the
xattr from the kernel.
In this patch, rc is set to 0 if the lfs_getsom()->lgetxattr()
call returns a positive value.

Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Ie3151f67b5ce2b5b2bc35a4b6528ba9a20a5db9f
Reviewed-on: https://review.whamcloud.com/33714
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11689 lfs: make sure project proceed all dirs

Leftover fix since LU-10986 lfs: make lfs project tolerant errors
We should proceed other dirs if we hit errors, otherwise,
some dirtree like following will fail if aaaa not exists.

testdir/
├── subdir
│   └── 1
├── bbbb -> aaaa
└── cccc

Also remove extra error output since we have output failing
messages inside every action function.

Test-Parameters: trivial testlist=sanity-quota,sanity-quota
Change-Id: I0062dbc3f4d1925c9e9e1a509ee35ac569bd9b74
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/33707
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11669 tests: add project in yml_test_group()

This patch fixes yml_test_group() in yaml.sh to
add test project name, which is required in
results.yml for Maloo to parse.

Test-Parameters: trivial

Change-Id: I0ae563d855dc2d28eaea85e86b1cb23d2428988b
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33658
Tested-by: Jenkins
Reviewed-by: Leonel Ochoa <lochoa@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11658 lov: cl_cache could miss initialize

The cl_cache may be missed initialize when we mount
a client with deactivate osc and then active it.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I92cd44375d70624fb55ef7a0218e7178211a8687
Reviewed-on: https://review.whamcloud.com/33650
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11620 lfsck: change llsd_rb_lock to rwsemaphore

llsd_rb_lock is taken in ->init, and released in ->fini, but during
this period it may getxattr which can sleep. Change it to rwsemaphore.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Idc68eb886e60dc45ccfb7ac9bf5bf06db42d690d
Reviewed-on: https://review.whamcloud.com/33603
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11579 lov: quiet lov_dump_lmm_ console messages

Limit messages in lov_dump_lmm_objects() and lov_dump_lmm_common()
printing to the console repeatedly when D_ERROR is used.  Change
CDEBUG() to CDEBUG_LIMIT() so that rate-limiting is applied.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I32fd70cdf6422222ab0a8f599aa60bc2f6da229e
Reviewed-on: https://review.whamcloud.com/33513
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11575 build: install systemd stuff only for debian with systemd enabled

Add a precheck for systemd, to avoid to try to package systemd
stuff intolustre-client/server-utils deb when building on debian
series without systemd supported.

Test-Parameters: clientdistro=ubuntu1604 trivial

Change-Id: If58b64acc035e621594ab420a8b900b18a34a211
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Reviewed-on: https://review.whamcloud.com/33492
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11504 tests: trigger lfsck after/during failover/failback

Patch adds the possibility to trigger lfsck
during and/or after failover/failback tests on
specified node with spectified device if set and
with lfsck custom parameters if required.

Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-6123,LUS-6591
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Change-Id: I4bdafdb5e6840f13e23a41abcda55de78db9d8b4
Reviewed-on: https://review.whamcloud.com/33351
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11503 tests: fix "division by 0" defect

if stop file is created before ha_repeat_mpi_load() or
ha_repeat_nonmpi_load() are started nr_loops is equal to 0
and test fails with
   (1476438773 - start_time) / nr_loops: division by 0
Patch fixes this case.

Patch correctly processes ha_dump_logs()->ha_on() rc
to avoid the test failure when nodes are not accessible for
logs dumping.

Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-2205
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Change-Id: I97957875685492ddec66daa7c353ff6ef930933c
Reviewed-on: https://review.whamcloud.com/33350
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11398 ptlrpc: clean up rq_interpret_reply callbacks

Clean up the function prototypes of several rq_interpret_reply
callback functions to match the function pointer type instead
of using typecasting to avoid the risk of bad function pointers.

Clean up related code to match code style.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I2df73304a8884c1cc838b9c9344d3fd9443ebbe5
Reviewed-on: https://review.whamcloud.com/33203
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8066 mdd: migrate from proc to sysfs

Move the ofd module from using proc for most single value files
to sysfs. The more complex proc entries are moved to debugfs.

Change-Id: I01eebf1c58f1a13c2f5e8c599a1363c80468b0bd
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33632
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11078 tests: customise run_ior()

Sometimes it is required to run ior with parameters
missing in run_ior() cmd, custom mpi threads per
cient and custom striping.

Now these parameters can be specified by:
  ior_custom_params
  mpi_ior_custom_threads
  ior_stripe_params

Test-Parameters:trivial testlist=parallel-scale
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Change-Id: I785d7aece107b2b53d0c422fbaec5c469b5bbc97
Reviewed-on: https://review.whamcloud.com/32688
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10862 ldlm: remove trace from ldlm_pool_{count,skrink}()

The trace in ldlm_pool_{count,shrink}() is too noisy given its
information value so remove it.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Iefed8000368c66b678aeae203e3afed06faa75c4
Test-Parameters: trivial
Reviewed-on: https://review.whamcloud.com/31820
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-8066 obdclass: report all obd states for OBD_IOC_GETDEVICE

The wrong state '--' which is reported when the obd device is
inactive. Reporting the "IN" state cover all the information that
is provided by 'devices' debugfs file. Now all the information
from 'devices' can be collected from the lustre sysfs tree.

Change-Id: I2479b15578babeb6f45da8129ffc0de99044730a
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33774
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-10171 headers: define pct(a,b) once

pct is defined 6 times in different places.  Define it in one.
Also change it to a static inline to do a better job of
enforcing types.

Signed-off-by: Ben Evans <bevans@cray.com>
Change-Id: If61132a1096c351a9bcb7debb868351206267535
Reviewed-on: https://review.whamcloud.com/29852
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

LU-6142 llite: Fix style issues for symlink.c

This patch fixes issues reported by checkpatch
for file lustre/llite/symlink.c

Change-Id: I053c5fe8ca1354e49fc8f1c4d54ba620253e6888
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33811
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-6142 llite: Fix style issues for lcommon_misc.c

This patch fixes issues reported by checkpatch
for file lustre/llite/lcommon_misc.c

Change-Id: I06410af949183bc054444b8189fc39aecc48a4ce
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33810
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-6142 llite: Fix style issues for llite_nfs.c

This patch fixes issues reported by checkpatch
for file lustre/llite/llite_nfs.c

Change-Id: I6457af5be663985f2e978115d1a1986a4ee15b08
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33809
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-6142 llite: Fix style issues for glimpse.c

This patch fixes issues reported by checkpatch for file
lustre/llite/glimpse.c

Change-Id: Idf5b811fd5def8976d97e928b451d0b0956ea9af
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33808
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>

LU-11712 osd-ldiskfs: Wrap blk integrity with config check

Build is currently broken for kernels without
CONFIG_BLK_DEV_INTEGRITY. Build failure introduced by LU-11096
commit c8505c2e70d03ba20edf9fcbf431888e87a21147
https://review.whamcloud.com/#/c/32725/
Use of blk integrity should be wrapped in the config check for
HAVE_BLK_INTEGRITY_ENABLED

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: Iac9e4a2572024c026132c87c11042cf353b14d48
Reviewed-on: https://review.whamcloud.com/33745
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 mgs: Fix style issues for mgs_fs.c

This patch fixes issues reported by checkpatch
for file lustre/mgs/mgs_fs.c

Change-Id: I2865df1a6663e288935f65353a408e3676eaa015
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33712
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-6142 lod: Fix style issues for lod_dev.c

This patch fixes issues reported by checkpatch for file
lustre/lod/lod_dev.c

Change-Id: I72eaa79a12769e61889e567e5f28fdf3e8045c94
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33594
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-6142 lmv: Fix style issues for lmv_fld.c

This patch fixes issues reported by checkpatch for file
lustre/lmv/lmv_fld.c

Change-Id: I98a5bdca61eff990370a7b2518b54e7f4137d120
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/33566
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11537 osp: avoid nested transaction

Don't create and start new transaction in
osp_precreate_reserve (osp_sync_force)
because it has been already started in mdd_create.
New transaction rewrites oti_declare_ops_cred
resulting in assert in osd_trans_exec_op:

osd_trans_exec_op()) lustre-MDT0000: opcode 3: credits = 0, rollback = 4
osd_trans_exec_op()) ASSERTION( !ldiskfs_track_declares_assert ) failed:
...
 #2 [ffff88008983f600] panic at ffffffff816a863f
 #3 [ffff88008983f680] lbug_with_loc at ffffffffc0513854 [libcfs]
 #4 [ffff88008983f6a0] osd_create at ffffffffc0dfac32 [osd_ldiskfs]
 #5 [ffff88008983f718] lod_sub_create at ffffffffc101b585 [lod]
 #6 [ffff88008983f7c0] lod_create at ffffffffc100d6e9 [lod]
 #7 [ffff88008983f800] mdd_create_object_internal at ffffffffc0ed8888 [mdd]
 #8 [ffff88008983f838] mdd_create_object at ffffffffc0ec3e05 [mdd]
 #9 [ffff88008983f8b0] mdd_create at ffffffffc0ecc673 [mdd]

Change-Id: Ic2c4589a9a1f640c7a0aa989fc62d81ca08f917f
Signed-off-by: Sergey Cheremencev <c17829@cray.com>
Cray-bug-id: LUS-6098
Reviewed-on: https://es-gerrit.dev.cray.com/153461
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Tested-by: Alexander Lezhoev <c17454@cray.com>
Reviewed-on: https://review.whamcloud.com/33391
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11296 osc: speed up page cache cleanup during blocking ASTs

While we are cleaning a write lock, we don't need to check if
page cache pages under this lock are covered by another lock.

If a client needs to give up its lock, cleaning gigabytes of
page cache can take quite a long time.

Signed-off-by: Andrew Perepechko <c17827@cray.com>
Cray-bug-id: LUS-6352
Change-Id: I576130216ed4de4e352ea697bddb5ff83046443a
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-on: https://review.whamcloud.com/33090
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10883 llog: fix backup file name comparison

Last 3 bytes of key are wrongly compared with ".bak" extention.
4 bytes need to be compared.

Signed-off-by: Artem Blagodarenko <artem.blagodarenko@gmail.com>
Change-Id: I90c1af2e41569780bf81df3975603acdc828edd6
Test-Parameters: testlist=conf-sanity
Reviewed-on: https://review.whamcloud.com/31886
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>

LU-11453 misc: add compat for INIT_LIST_HEAD_RCU

Add a compat version of INIT_LIST_HEAD_RCU() if unavailable.

Fixes: 68bc3984975bb72f730d8a8ab7aa2d836e50abe5

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I9c1296206e002b895bb6bdf55ddd0d8ec70cab07
Reviewed-on: https://review.whamcloud.com/33875
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11750 krb5: krb5int_derive_key has 'hash' extra parameter

From Kerberos 5 release 1.15, and introduction of support of
aes-sha2, krb5int_derive_key() groks an additional 'hash' parameter.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I7c6ea5ac2d6844371b254b7361d28c462afe5afa
Reviewed-on: https://review.whamcloud.com/33817
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-7004 mgs: remove using obdname2fsname() from mgs layer.

Further testing shows that with the complex of obd device names
as well as wild card handling makes covering every case in
obdname2fsname() very difficult. Instead of mining the file
system name out of obd device name take advantage of the fact
the MGS device will know all the file system names and test if
the file system name is present in the lctl set_param -P
cmd.

Fix up the PERM_CMD test in the test suite which was wrong.
Make config-sanity test 29 wait until the tunables are set
after mounting the client.

Test-Parameters: envdefinitions="PERM_CMD=$LCTL set_param -P"

Change-Id: Ie3f5c8e1e1ff5ec309a31ba598f7cdf2f19c791f
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32784
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10239 osc: limit chunk number of write submit

Don't queue too many pages in an extent for a write RPC, we need
to take care of the chunk limit in write submit as well (refers to
LU-8135 for more details).

Change-Id: Idaca6f365a6f8bb467fea43c811f775b37f49818
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/30627
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10384 mgs: replace_nids large string and failover support

Replace_nids uses nids list as new UUID. UUID string
length is limited by 38 symbols. So new nids list need
to be less then 38 symbols.

With this patch first nid in list string representation
is used for UUID as this done for failover nids.

Replace nids finds records for given device and regenerates
lines that contain old nids. add_uuid and add_conn lines for
failover used to be deleted during replace_nids which breaks
failover configuration.

This patch adds failover support to replace_nids command.
For example:

lctl replace_nids lustre-MDT0000 nid1,nid2:nid3,nid4:nid5,nid6

nid3,nid4 - nids from first failover node
nid5,nid6 - nids from second failover node

Signed-off-by: Artem Blagodarenko <c17828@cray.com>
Cray-bug-id: MRP-4505
Change-Id: I4e9a35e8fa8781909ecbaa74785700f4ca04cf92
Reviewed-on: https://review.whamcloud.com/30624
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

New tag 2.12.50

Start of 2.13 development effort.

Change-Id: Id378f5ad264aca50d31d6fdf0e2364904b76a4e2
Signed-off-by: Oleg Drokin <green@whamcloud.com>

New release 2.12.0

Change-Id: Icc5da4f5d1d032982a144ee5d13f214b04389d76
Signed-off-by: Oleg Drokin <green@whamcloud.com>

New release candidate 2.12.0-RC4

Change-Id: I5aa92f8a4232f5293e3253223d9447d23b3b0337
Signed-off-by: Oleg Drokin <green@whamcloud.com>

LU-11809 llite: don't use %p to generate cfg_instance

In kernel 4.15 and later, using "%p" in a string format to
print a kernel pointer will result in the pointer being
hashed with a random value, and the high bytes will be masked
on 64-bit CPUs to prevent leaking kernel address-space info
to userspace to defeat ASLR.  In early boot, the "%p" pointer
may resolve to "        (ptrval)", if there is not enough
entropy in the system to generate a random hash value.

The superblock pointer is used on the client to uniquely
identify all of the OBD devices connected to it, and to
find the configuration llog that was used to mount the
filesystem, so that it can also be used at unmount time.
The sb pointer is also used in the OBD device names, and
the "        (ptrval)" expansion breaks /sys filenames,
and also breaks the uniqueness of the config instance.

On the server, there is also a pointer value used for the
FLDB SEQ servers of the OSTs.

For the short term, bypass the "%p" hashing, so that mount
continues to work properly, and this can be resolved in a
later patch to change ll_get_cfg_instance() to provide a
unique value that is not directly a kernel pointer.

In llapi_getname() don't depend on the cfg_instance being
exactly 16 characters long, if this changes in the future.

Test-Parameters: clientdistro=ubuntu1804 testlist=sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I166de0248af8fe57535628a64bb770a4e03ebbe5
Reviewed-on: https://review.whamcloud.com/33900
Tested-by: Jenkins
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11783 build: remove lustre_user.h deprecation warning

The "lustre/lustre_user.h" header has been in use for many years.
The patch https://review.whamcloud.com/25246 "LU-6401 uapi: migrate
remaining uapi headers to uapi directory" moved the header to
"linux/lustre/lustre_user.h" and left a stub "lustre/lustre_user.h"
behind that generates a compiler warning that this header is
deprecated.

However, no window was given between the introduction of the new
header and the deprecation of the old header, which makes it harder
for applications to smoothly transition to the new header location.
Also, installing Lustre headers into the "linux/" directory before
Lustre is actually part of the kernel may potentially cause problems.

Disable the deprecation warning in the old header for several
releases, until the new header location has been available for a good
time and it is safe for applications to assume that it is available.

Test-Parameters: trivial
Fixes: 6712478e79588e73e28c7ccac3afc7ac2368a4f3
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If5a62587e2d3627178a0f7a09c3a4c10801cab07
Reviewed-on: https://review.whamcloud.com/33872
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

New release candidate 2.12.0-RC3

Change-Id: I841a274fd9ab3b35c8c3a5b93e33469a9e230af5
Signed-off-by: Oleg Drokin <green@whamcloud.com>

LU-11753 obdclass: lu_dirent record length missing '0'

In lu_dirent packing, a '0' is appended after name, but it's not
counted in size calcuation, which may cause crash.

Add sanity test_230l.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Iab4947dea8e26ea798d5f64e218268200a5fabe8
Reviewed-on: https://review.whamcloud.com/33865
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11753 obdclass: index_page support variable length rec

mdd_dir_is_empty() may readdir from other MDT if directory
is striped or remote, in this case, it will issue OBD_IDX_READ
RPC to fetch dir page, and on remote MDT dt_index_page_build()
is called to build page, but this function doesn't support
variable length record, so it may miscalculate offset in
reading, which may cause crash.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ia25a0aca52fb1323ea64a7ff72bf6022754af32c
Reviewed-on: https://review.whamcloud.com/33837
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11729 tests: skip sanity test 810 for ARM

Skip sanity.sh test_810 for ARM clients as it always failing.

Test-Parameters: trivial clientarch=aarch64 testlist=sanity
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I84117aebb277d4ddcb7787b715587e330f3ebbe5
Reviewed-on: https://review.whamcloud.com/33864
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11770 osd-ldiskfs: preserve bio_integrity API

Preserve the existing kernel API for bio_integrity when the T10-PI
patches are applied, so that any other code that may be using this
interface do not break.

In particular, keep the EXPORT_SYMBOL(bio_integrity_alloc) and
EXPORT_SYMBOL(bio_integrity_prep) in place to avoid module breakage.

In struct bio_integrity_payload put the *bip_generate_fn and
*bip_verify_fn pointers after *bip_vec, since bip_vec is the last
field directly accessed by callers.

In struct blk_integrity_exchg the bi_idx field only needs to be an
unsigned short since the bio->bi_idx and bio->bi_vcnt values used
with it are also unsigned short.  This saves 8 bytes of padding in
the struct and puts the added bi_bio field at the end to preserve
the structure field alignment for external callers.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I9e29723d5d581a65b1c2ca2611d012c05b953514
Reviewed-on: https://review.whamcloud.com/33840
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11753 utils: print out DNE2 directory hash flags

There may be flags stored in the lmv_hash_type field, such as
"LMV_HASH_FLAG_MIGRATION" that is set while the directory is
being migrated.  Print out the flag from "lfs getdirstripe".

This is still missing support for "lfs find" to find directories
that have incomplete migration.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib6b362f9eb993b5fa0562b3a51b54eaee1ccab07
Reviewed-on: https://review.whamcloud.com/33843
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11783 utils: fix warnings when lustre_user.h included

Checking for lustre/lustre_user.h in a configure script
generates a warning because of the included <sys/quota.h>

  checking lustre/lustre_user.h usability... no
  checking lustre/lustre_user.h presence... yes
  WARNING: present but cannot be compiled
  WARNING: check for missing prerequisite headers?
  WARNING: see the Autoconf documentation
  WARNING: section "Present But Cannot Be Compiled"
  WARNING: proceeding with the preprocessor's result
  WARNING: in the future, the compiler will take precedence

Looking into config.log it shows:

  In file included from /usr/include/lustre/lustre_user.h:59,
                   from conftest.c:91:
  /usr/include/sys/quota.h:221: error: expected declaration
    specifiers or '...' before 'caddr_t'

Since we don't really need much from the <sys/quota.h> header,
add conditional #defines for the few needed fields.

The FASYNC constant is not declared everywhere in userspace,
provide a compat declaration if unavailable.

Fix an unused variable warning in ll_dir_ioctl().

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I9cd2b0fcbaf16fe8a5a4a7a0309aada3a72cab07
Reviewed-on: https://review.whamcloud.com/33876
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

New release candidate 2.12.0-RC2

Change-Id: I84a1bcb460331928bd4987f33232c22a40b3d58c
Signed-off-by: Oleg Drokin <green@whamcloud.com>

LU-11740 kernel: new kernel [RHEL7.6 4.14.0-115.2.2.el7a]

This patch support new RHEL 7.6 release on ARM

Test-Parameters: trivial

Change-Id: I79424f0759b79e0a2f45ea5337c3577f832dccb1
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33804
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11684 config: fix conf-sanity test_123

conf_param parameters go into FSNAME-MDT/OST/client files
set_parm -P parameters go into "params"

Change the test to set the conf_param parameter files first
followed by the set_param -P parameters, since there may be
overlap.  In this case, the test infrastructure is using
conf_param to set jobid_var, and test_123 is using set_param.
This causes collision and occasional failure.

Test-Parameters: trivial testlist=conf-sanity

Signed-off-by: Ben Evans <bevans@cray.com>
Change-Id: I5cbdbaf6cc0c1c55a870bd587e89b2cbdaf77c29
Reviewed-on: https://review.whamcloud.com/33720
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

Revert "LU-11152 lnd: test fpo_fmr_poool pointer instead of special bool"

This reverts commit 9b790ba0f5606c0a91563828fa43f5e4ae210425.

Change-Id: Ibca8e813ec7372510709578e33309140e8fc7b5f
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33802
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11734 lnet: handle multi-md usage

The MD can be used multiple times. The response tracker needs to have
the same lifespan as the MD. If we re-use the MD and a response
tracker has already been attached to it, then we'll update the
deadline for the response tracker. This means the deadline on the MD
is for its last user.

Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I681630c3d599f66c007926525708e3004b343455
Reviewed-on: https://review.whamcloud.com/33794
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11440 misc: require ldiskfsprogs-1.44.3.wc1 or later

Require a current version of ldiskfsprogs to include support for
project quotas and large_dir.  Upstream now also includes ea_inode
support and many bug fixes in xattr verification and repair.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I24eeb1c30d5c7b1daa1ad7d5d2f603d273054035
Reviewed-on: https://review.whamcloud.com/33766
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

New Release candidate 2.12.0-RC1

Change-Id: I5464c8f19a06e3dd35b7704bf4359726514e62ad
Signed-off-by: Oleg Drokin <green@whamcloud.com>

LU-11389 lnet: increase lnet transaction timeout

Increase the new LNet Health transaction timeout to the original
50s value, to avoid spurious lnet-selftest failures and expected
false timeouts under load.

Fix the lnet_transaction_timeout module parameter description.

Test-Parameters: trivial clientarch=aarch64 testlist=lnet-selftest,lnet-selftest,lnet-selftest,lnet-selftest
Test-Parameters: clientarch=aarch64 testlist=lnet-selftest,lnet-selftest,lnet-selftest,lnet-selftest

Signed-off-by: Sonia Sharma <sharmaso@whamcloud.com>
Change-Id: Ic9df69ef8c7a4085815b54dc6741f37a73d36a75
Reviewed-on: https://review.whamcloud.com/33231
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>

LU-10576 tests: allow log files to be created/removed

Allow an llog file to be created or removed during the course of
the test, as this can happen due to internal housekeeping activity.

Also ensure that background cleanup has finished with ZFS before
fetching the number of objects from the MDT.

Test-Parameters: trivial
Test-Parameters: testlist=sanity mdscount=2 mdtcount=4 mdtfilesystemtype=zfs ostfilesystemtype=zfs
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I0a0968cfcd90c7493c67b54ba8a7f326163ebbe5
Reviewed-on: https://review.whamcloud.com/33677
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11697 ost: do not reuse T10PI guards of unaligned page write

If the write is partial page, the guards of RPC checksum should not
be reused for bio submission since the data might not be full-sector.
The bio guards will be generated later based on the full sectors. If
the sector size is 512B rather than 4 KB, or the page size on OST is
larger than 4KB, this might drop some useful guards for partial page
write, but it will only add minimal extra time of checksum calculation.

Change-Id: I868342df87c28ea91f5f8364fe377277595ecf6d
Signed-off-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/33752
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11697 osc: wrong page offset for T10PI checksum

The page offset might could be non-zero value. Thus, when
calculating T10PI checksum, the offset should be correct value.

Change-Id: Ib32584eb47ea55ec3804e531ac02ffd252411886
Signed-off-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/33727
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11663 osd-zfs: write partial pages with correct offset

otherwise non-aligned writes send wrong data to ZFS.

Change-Id: I1ae1f361981d548307d74344a5694f3ef39c0609
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33726
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11652 kernel: kernel update [SLES12 SP3 4.4.162-94.69]

Update SLES12 SP3 kernel to 4.4.162-94.69.

Test-Parameters: clientdistro=sles12sp3 ossdistro=sles12sp3 mdsdistro=sles12sp3

Change-Id: Iea1ec6def609059d67053d25360e8e986f2adbd9
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33637
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11668 mdt: check parent type in rename/migrate

Check parent existence and type in rename and migrate to avoid
potential race.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I583e2f5a6f47073601e36c06890a6b22dfc734ad
Reviewed-on: https://review.whamcloud.com/33709
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-11668 debug: print object type in mdd_parent_fid

mdd_parent_fid() get parent fid for directory, but racer shows
the passed in object is not directory, print its type to help
debug.

Test-Parameters: trivial
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Signed-off-by: Peter Jones <pjones@whamcloud.com>
Change-Id: I4b3eedb159fc0efccc15e35cb59010fe02fa9e01
Reviewed-on: https://review.whamcloud.com/33700
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>

LU-10114 hsm: increase upper limit of maximum HSM backends registered with MDT

Lustre only supports at most 32 HSM backends, which limits HSM to be applied
to other features, such as LPCC. This patch breaks the limitation by allowing
the system take any interger number as a valid archive-id.

Test-Parameters: clientjob=lustre-b2_10 clientbuildno=136 testlist=sanity-hsm
Test-Parameters: mdsjob=lustre-b2_10 ossjob=lustre-b2_10 serverbuildno=136 testlist=sanity-hsm

Change-Id: I9523b92500b962db3e45a2bd6a67dba54eef5335
Signed-off-by: Teddy Zheng <teddy@ddn.com>
Signed-off-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/32197
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-5152 quota: disable sync chgrp to OSTs

The syschronous chgrp to OSTs introduced by the previous patch
(the commit is 8a71fd5061bd073e055e6cbba1d238305e6827bb) causes
deadlock between MDT and OST, this patch disable it for now and
leave the updated patch to fix it.

Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Change-Id: I5ce48424f4d2011ce62e69047ace7f0b7c3ebbe5
Reviewed-on: https://review.whamcloud.com/33705
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11653 hsm: copytool registration wakes the coordinator

When a copytool registers to the MDS, it is possible there are
pending requests in the coordinator's llog that previously could not
be sent (either because there were not any copytools, or not any
compatible copytools).

With this patch, the coordinator will process those requests on its
next wake up (which happens every second).

Test-Parameters: trivial
Test-Parameters: mdscount=2 mdtcount=4 mdtfilesystemtype=zfs testlist=sanity-hsm
Test-Parameters: mdscount=2 mdtcount=4 mdtfilesystemtype=ldiskfs testlist=sanity-hsm
Test-Parameters: mdscount=2 mdtcount=4 mdtfilesystemtype=zfs testlist=sanity-hsm
Test-Parameters: mdscount=2 mdtcount=4 mdtfilesystemtype=ldiskfs testlist=sanity-hsm

Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: Ie49b40d312f2f3e0d9c85dee27bb8813dc4dde40
Reviewed-on: https://review.whamcloud.com/33649
Tested-by: Jenkins
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11541 build: Use correct kernel version for DKMS MLNX OFED.

Check for $O2IBPATHS/${LINUXRELEASE} before $O2IBPATHS/default.
The "default" link is only created on the first dkms build of OFED.
So if doing dkms build for another kernel it may not be pointing
to the correct kernel.

Test-Parameters: trivial

Signed-off-by: Ake Sandgren <ake.sandgren@hpc2n.umu.se>
Change-Id: If8054ee64cf8795ed0e3ee50a8ef9ced067059d7
Reviewed-on: https://review.whamcloud.com/33702
Tested-by: Jenkins
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11567 utils: llog_reader print changelog index

When processing changelog type llogs, print the changelog index number
with each changelog record.  This allows one to compare the records on
disk with the output of lfs changelog or relatives.

Test-Parameters: trivial
Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Change-Id: I0059cc34b39161462b3eadbb2512dc811c38705a
Reviewed-on: https://review.whamcloud.com/33473
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11329 utils: Add maintainer entries

Add clio reviewer entry.

Add Grant subsystem section.

Test-Parameters: trivial
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I461b39fe0b65f8d48b3912086927c945b2da3db7
Reviewed-on: https://review.whamcloud.com/33684
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>

LU-11645 tests: fix sanity-sec test 31

In sanity-sec test 31, command to add new LNet network ${NETTYPE}999
may fail if servers have interface names different from the one used
on the client.
So fix the command so that it is run directly on each node.

Test-Parameters: trivial envdefinitions=ONLY=31 testlist=sanity-sec
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ibf9101524d94188b3beae3debe45e2ba151999ca
Reviewed-on: https://review.whamcloud.com/33622
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11572 tests: make sanity-hsm test_260c reliable

When the coordinator restarts, its first run does housekeeping.
For the following `loop_period' seconds, it will not run in
housekeeping mode.

This patch uses this to improve the reliability of sanity-hsm
test_260c.

Test-Parameters: trivial testlist=sanity-hsm,sanity-hsm,sanity-hsm
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I8eeb8b6856d91b69495d592cdd1cb5f091b1cc2b
Reviewed-on: https://review.whamcloud.com/33478
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11519 hsm: improve the testing of hsm.max_requests

Modify sanity-hsm::test_250() to send more diverse HSM request.

This brings better code coverage (it would have caught the bug
reported in LU-11519).

Test-Parameters: trivial testlist=sanity-hsm,sanity-hsm,sanity-hsm
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I0142b46dc804f649c33deb81efea7f68d1e29afa
Reviewed-on: https://review.whamcloud.com/33590
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11662 llite: handle -ENODATA in ll_layout_fetch()

In ll_layout_fetch() handle -ENODATA returns from mdc_getxattr(). This
is needed for interop and restores the behavior from before commit
0f42b388432c4b898857660197ef13a40a82cd9d (LU-11380 mdc: move empty
xattr to mdc layer) landed.

Test-Parameters: clientjob=lustre-b2_10 clientbuildno=136 testlist=sanity-hsm
Test-Parameters: mdsjob=lustre-b2_10 ossjob=lustre-b2_10 serverbuildno=136 testlist=sanity-hsm
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I1fb85faf35c7d2303a1f61a5a9c5922988739817
Reviewed-on: https://review.whamcloud.com/33665
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11595 mdt: fix read-on-open for big PAGE_SIZE

Client PAGE_SIZE can be larger than server one so data returned
from server along with OPEN can be misaligned on client.

Patch replaces assertion on client with check and graceful exit,
changes MDC_DOM_DEF_INLINE_REPSIZE to be PAGE_SIZE at least and
updates mdt_dom_read_on_open() to return file tail for maximum
possible page size that can fit into reply.

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ic2c54b95c814d3b6df3b527527cac08488060651
Reviewed-on: https://review.whamcloud.com/33606
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11582 llite: protect reading inode->i_data.nrpages

truncate_inode_pages() looks up pages in the radix tree without
lock, and could miss finding pages removed from the radix tree
by __remove_mapping(), so that after calling truncate_inode_pages()
we need to read the nrpages of the inode->i_data with the protection
of tree_lock.

Since it could still be in the race window of __remove_mapping()->
__delete_from_page_cache()->page_cache_tree_delte(), before the
nrpages being decreased.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I44ba6bea3dec4f0a110d1ae2a749514ec7dd0d12
Reviewed-on: https://review.whamcloud.com/33639
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Maloo <maloo@whamcloud.com>

LU-11466 mdt: Skip SOM xattr update for DoM-only files

When scan the MDT image, DoM-only file can be specialized handled,
the size and blocks can be got directly from the inode on MDT, no
need SOM xattr anymore.
Thus, there is no need to store the SOM xattr for DoM-only files.

Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: I0f871cde38fc846460dd1b6f92509dee9ea90bfc
Reviewed-on: https://review.whamcloud.com/33331
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9793 ptlrpc: Do not map unrecognized ELDLM errnos to EIO

The lustre_errno_hton and lustre_errno_ntoh functions map between
host and network error numbers before they are sent over the network.
If an errno is unrecognized then it is mapped to EIO.

However an optimization for x86 and i386 architectures replaced the
functions with macros that simply return the original errno. The
result is that x86 and i386 return the original values for ELDLM
errnos and all other architectures return EIO. This difference is
known to break glimpse lock callback handling which depends on clients
responding with ELDLM_NO_LOCK_DATA. The difference in errnos may
result in other as yet unidentified bugs.

The fix defines mappings for the ELDLM errors that leaves the values
unchanged. Error numbers not found in the mapping tables are still
mapped to EIO.

Cray-bug-id: LUS-6057
Signed-off-by: Ann Koehler <amk@cray.com>
Change-Id: I0b4e1e0dc6de065729e18f2381ec9cfc58fe31db
Reviewed-on: https://review.whamcloud.com/33471
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11519 hsm: handle hsd_request_count == 0 properly

In mdt_cdt_waiting_cb() it may be that the coordinator has already
reached the limit of active requests and hsd contains no requests to
be started. Handle this properly when trying to prioritize a restore.

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ic843b7672ae6a4509ac127c2d2f90bf3681f84fc
Reviewed-on: https://review.whamcloud.com/33580
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11071 build: use --with-linux-obj for ubuntu

We can use --with-linux-obj instead of --with-linux-config
the config will be found just under LINUX_OBJ.
Otherwise the configure could fail if LINUX and LINUX_OBJ
are different paths. e.g. ubuntu18

Test-Parameters: trivial
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: Ibaa159e5611c4054b3987da0ce3a8bca2992e057
Reviewed-on: https://review.whamcloud.com/33145
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11642 lmv: allocate fid on parent MDT in migrate

During directory migration, if the migrated file is not directory,
the target should be allocated on its parent MDT, not user specified
MDT. Because if it's parent is striped, this file should be migrated
to the MDT by its name hash, not the starting MDT of its parent.

Add sanity 230k to check file data not changed after migration.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Ic7d3de8ea982b7cf4da758e4d3ab8d8ee15ecfdb
Reviewed-on: https://review.whamcloud.com/33641
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11642 mdt: revoke remote LOOKUP lock in dir layout shrink

mdt_dir_layout_shrink() should revoke remote LOOKUP lock if parent
is remote, because it will alter dir layout, which is refreshed
upon lookup.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I26ae1af5da6142b44005e5d9ea11293af65ed7b5
Reviewed-on: https://review.whamcloud.com/33640
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11497 tests: improve ha.sh to set striped dirs

For DNE II testing we need the possibility to create
the striped directories, like it is done in t-f:test_mkdir().
This patch covers the following settings:
-c stripe_count -i mdt_index
-c stripe_count -i <random mdt_index>,
which allows to have the clients work directories with
the same stripe_count but different mdt indexes.

Test-Parameters:trivial
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-5974
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Change-Id: I6117601e741a95059750149d1c38b402fccb29b7
Reviewed-on: https://review.whamcloud.com/33340
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11494 tests: sanity-quota/22 syntax error fix

Patch fixes test_22() trivial syntax error.

Test-Parameters:trivial testlist=sanity-quota envdefinitions=ONLY=22
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-6011
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Change-Id: Icb3474c7166d6b74b3092617bb314a561687df06
Reviewed-on: https://review.whamcloud.com/33337
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>

LU-11492 tests: fix thread_sanity() defect

tmin, tmax, tstarted are not set if do_facet fails by some reason.
This leads to the following failure:
Assertion 28 failed: (($tstarted >= $tmin && $tstarted <= $tmax ))
   (expanded: ((16 >=  && 16 <= 16 )))
Patch sets variables equal to 0 for cases of failed do_facet, like
it is done for other similar cases.

Test-Parameters:trivial testlist=sanity envdefinitions=ONLY="53a 53b"
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: LUS-5638
Reviewed-by: Alexander Boyko <c17825@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Change-Id: If713ba20a0f71cb17208f776a9a4edd359c07c43
Reviewed-on: https://review.whamcloud.com/33335
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11541 build: Adjust OFED check for DKMS enabled MLNX OFED.

The DKMS packaging of MLNX OFED, i.e. mlnx-ofed-kernel-dkms, (at least
v4.4 and newer), doesn't have /usr/src/ofa_kernel/default as part of
the package, it is created as a link during package installation.

Restore LB_USES_DPKG autoconf test that was removed since it is needed
in lustre-lnet.m4 so Debian / Ubuntu can build properly with external
infiniband stacks.

Test-Parameters: trivial
Signed-off-by: Ake Sandgren <ake.sandgren@hpc2n.umu.se>
Change-Id: I1739a67e16bc0dab8cf551b460b424441d81fc91
Reviewed-on: https://review.whamcloud.com/33396
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-9906 clio: use pagevec_release for many pages

When Lustre releases cached pages, it always uses
page_release, even when releasing many pages.

When clearing OST ldlm lock lrus in parallel with lots of
cached data, the ldlm_bl threads spend most of their time
contending for the zone lock taken by page_release.
Also, when osc_lru_reclaim kicks in when there's not enough
LRU slots during I/O, the contention on zone lock kills
I/O performance.

Switching to pagevec when we expect to actually release the
pages (discard_pages, truncate, lru reclaim) brings
significant performance benefits as shown below.

This patch introduces cl_pagevec_put() to release the pages
in batches using pagevec, which is essentially calling
release_pages().

  mpirun -np 48 ior -w -r -t 16m -b 16g -F -e -vv -o ... -i 1 [-B]

                mode         write (GB/s)    read (GB/s)
  master        O_DIRECT     20.8            21.8
  master+patch  O_DIRECT     20.7            22.2
  master        Buffered     11.6            12.3
  master+patch  Buffered     15.3            19.6

Also clean up the dead lovsub_page related code.

Signed-off-by: Patrick Farrell <paf@cray.com>
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I71447528db12858defb627c9c03b7193d116c935
Reviewed-on: https://review.whamcloud.com/28667
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-8777 mdt: add parameter to disable remote/striped dir

Restore mdt_enable_remote_dir to enable/disable remote directory
feature, and also add mdt_enable_striped_dir and
mdt_enable_dir_migration for striped directory and directory
migration.

Enable remote directory, striped directory and directory migration
by default.

Add sanity.sh 417.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I583dbc5a5eb54de02c8a4459092d4a3f3b24b262
Reviewed-on: https://review.whamcloud.com/24498
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11596 tests: disable several sanity sub-tests for ARM

The following sanity sub-tests are consistently failing
on ARM clients:
- 42d, 42e, 63a, 63b, 64a, 64b and 64c (LU-11596)
- 45 (LU-11671)
- 101c (LU-11665)
- 103a (LU-11594)
- 317 (LU-11667)

This patch adds them to ALWAYS_EXCEPT list to exclude
the failures, so as to make ARM client test session pass.

They will be removed from the list and re-enabled in the
patches that fix the failures.

Test-Parameters: trivial

Change-Id: I1728dc665c8a52f0fd88de8173f88ccee0c8772d
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33652
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>

LU-11635 gss: proper gss build for client-only

Fix gss build system for both rpms and debs in case of client-only
build.
Fix consists in passing along '--{enable,disable}-gss' option, and
including lgssc.conf file in case of client-only build.

Test-Parameters: trivial
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Id884d282e312177c262b1f4725b8f19e2c14fd37
Reviewed-on: https://review.whamcloud.com/33608
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-11102 ldlm: don't skip bl_ast for local lock

The previous commit 954cc675 skips bl_ast for local lock but
there are cases on MDT when local lock can become a client lock,
see mdt_intent_lock_replace(). In that case the client should be
notified if lock is a blocking lock.

Patch reverts commit 954cc675 and provides alternative solution.
During downgrade to COS the lock renews own blocking AST states
and start reprocessing. Any new lock conflict will cause new
blocking AST and related async commit as needed.

Test-Parameters: mdssizegb=20 testlist=racer,racer,racer
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I41adab5c805a59fdbeade8ae3556556b779dc3c0
Reviewed-on: https://review.whamcloud.com/33458
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10626 test: create custom udev rule

Some lustre developer do all their testing from their lustre
source tree instead of installing it into an image. Since lustre
is not installed into the image the udev rule for lctl set_param
doesn't work since its not in /etc/udev/rules.d. Additonally udev
rules require using the absolute paths for executables. To handle
this setup create a special udev rule on the fly using $LCTL to
be placed into /etc/udev/rules.d.

Change-Id: I84c55450c866977650b3c6d4190e233a24eb03d8
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33143
Reviewed-by: Nathaniel Clark <nclark@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

LU-10335 test: enable sanity 130 tests for Ubuntu

The default e2fsprogs for Ubuntu 16 lacked the changes for fiemap.
This caused sanity test 130 to fail. Now with lustre using newer
e2fsprogs with the new Ubuntu server support this is no longer the
case. Lets re-enable sanity 130 test again.

Test-Parameters: clientdistro=ubuntu1604 trivial testlist=sanity envdefinitions=ONLY=130

Change-Id: Iad42e1f55855da904515b3fe00c8d047fa13b45c
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/33459
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>