LU-9560 tests: sanity test_801a sleeps more time

If the barrier_freeze thread sleep time is too short, then
it is possible that before the barrier_stat thread scheduled,
the barrier_freeze thread has gone through phase "freezing_p1",
that will cause test failure.

This patch makes the barrier_freeze thread to sleep more long
time, then the barrier_stat thread can check its status properly.

Test-Parameters: trivial
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iebbc05a1d86553b484bd7093f147e5ac4b58e199
Reviewed-on: https://review.whamcloud.com/27288
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9556 tests: wait recovery complete in replay-single test 57

This patch fixes replay-single test 57 to wait recovery complete
for MDS and also wait for sync between MDS and OSS instead of
just sleeping for 1 second.

Test-Parameters: trivial envdefinitions=ONLY=57 \
testlist=replay-single,replay-single,replay-single

Change-Id: I8ca133104116383eccec324ab2ef56666659600e
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: https://review.whamcloud.com/27279
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-9550 adio: delete the old Lustre ADIO driver patches

Since Lustre ADIO driver was released by MPICH2 about 8 years ago,
those old Lustre ADIO driver patches (lustre/contrib/*mpich*.patch)
can be deleted.

Test-Parameters: trivial
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Ibbd4c67b36065d913c6e3b71895f2413481c923c
Reviewed-on: https://review.whamcloud.com/27264
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-9558 llite: generic_readlink is no longer exported

Starting in linux kernel 4.10 if inode_operation's
readlink field is NULL then it is assumed to be using
generic_readlink. Since this is the case the function
generic_readlink is no longer visible to individual
file systems.

Linux-commit: dfeef68862edd7d4bafe68ef7aeb5f658ef24bb5

Change-Id: I847844cc53f7ae80d9d212590f5026a92da0107e
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27280
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-9463 utils: print lcme_flags in text format

Print lcme_flags in comma-separated text format(e.g. init)
instead of hex numbers.

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Iae8b679aef8f7c0b86af3e94d5bb996d4d98757b
Reviewed-on: https://review.whamcloud.com/27058
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-8576 test: Fix hard-coded NETTYPE in test-framework.sh

In test-frameowrk.sh, h2o2ib() and h2tcp() hard-codes
the network type value to o2ib or tcp respectively.

With this patch, h2nettype() is defined which replaces
h2o2ib(), h2tcp(), h2ptl() and h2elan(). Function
h2nettype() uses value of NETTYPE parameter passed
from user instead of any hard-coded value. With this patch,
NETTYPE value is used as network name instead of network type.

Test-Parameters: trivial testlist=conf-sanity, sanity, sanity-sec
Change-Id: If0fe16a1f6aade4d7f784243c75f70764280ea88
Signed-off-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-on: https://review.whamcloud.com/27027
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9439 scripts: Provide a sample lnet.conf file

Create a sample lnet.conf file that is packaged and
installed.

Move /etc/sysconfig/lnet.conf to /etc/lnet.conf. Configuration
files should not be in /etc/sysconfig.

Cleaned up the init.d lnet script.

Test-Parameters: trivial
Signed-off-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Change-Id: Ie94d0c2680c0f1fd65cbf2e33b08194beee5d8be
Reviewed-on: https://review.whamcloud.com/26971
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9439 scripts: Change behavior of lustre_rmmod

The lustre_rmmod script was modified to take an arbitrary list of
modules and try to remove them and any modules dependent on them.
Previously its behavior was to always remove the libcfs module,
along with either the ldiskfs or another module passed as a
parameter.

The old interface was roughly maintained. Any of the following
commands will remove the ldiskfs, libcfs, and all dependent modules:

    $ lustre_rmmod
    $ lustre_rmmod ldiskfs
    $ lustre_rmmod ldiskfs libcfs

The benefit now, is that any other list of modules can be specified
without removing libcfs. For example, the following command will only
remove ptlrpc and its dependent modules (leaving libcfs intact):

    $ lustre_rmmod ptlrpc

The lnet init script was modified to perform a lustre_rmmod ptlrpc
before performing an lctl network down. By removing the ptlrpc
module, we can ensure that lnet is not in use. This will help
systems running lustre to shut down cleanly.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Signed-off-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Change-Id: I3fbb33ccb00eb96c255c702b238af9d46768954b
Reviewed-on: https://review.whamcloud.com/27213
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-9119 lnet: increment per NI stats

Increment the per NI stats for messages being routed.
This will give a better view of the traffic distribution
over multiple peer interfaces.

Added extra trace messages to track the messages
sent and received.

Test-Parameters: trivial
Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I213b1b36e9787d25705d91d091ad9e9c6a5b2ae8
Reviewed-on: https://review.whamcloud.com/26907
Tested-by: Jenkins
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9288 tests: t-f to check if ZFS module is loaded

it's possible that ZFS modules are missing in /lib/modules/ in
a development environment. if so, then modprobe fails preventing
regular testing. t-f can check whether ZFS module is loaded already.

Change-Id: Ie871be7a9af3c15968de8b5d8a8c99b3e1cb4274
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/26334
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9010 ldlm: use static initializer macros where possible

In lustre/ldlm/ replace module load time initialization of several
lists, locks, mutexes, and wait queues with static initialization
using the kernel provided macros.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I145daf6355c568c422ab8c8b02ca284637b75320
Reviewed-on: https://review.whamcloud.com/24824
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-8797 lbuild: Fail build if zfs or spl git checkout fails

This forces lbuild to fail if zfs or spl checkout fails, for instance
if the tag isn't present.  This turns a subtle "wrong zfs version
checked out" into a more obvious error.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I43b424a755c41cb32a7a07fa848c582bdf279e62
Reviewed-on: https://review.whamcloud.com/23563
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7088 tests: merge sanityn test_1[a-d] into test_1

The sanity.sh test_1a, test_1b, test_1c, and test_1d cases could
not be run independently. Merge these test cases to form a single
test_1 that is self contained.

Test-Parameters: trivial envdefinitions=ONLY=1 testlist=sanityn
Seagate-bug-id: MRP-335
Signed-off-by: Kyrylo Shatskyy <kyrylo.shatskyy@seagate.com>
Signed-off-by: Elena Gryaznova <elena.gryaznova@seagate.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@seagate.com>
Change-Id: I7e550f4c34ef57ac8a2c6484f95731f61d17f86d
Reviewed-on: https://review.whamcloud.com/16226
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-9137 utils: replace readdir_r with readdir

Originally readdir_r was created to handle the case of readdir
not being reentrant. The readdir_r implementation has several
flaws and modern platforms have a glibc version were readdir
is reentrant. Because of this platforms are now marking
readdir_r as obsolete which breaks our lustre build. This
patch migrates us to readdir instead.

Change-Id: I939d5a402e6c73c56615972b2b3c6f0a21b59b55
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27276
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9536 lnet: set LND tunalbes properly

Make sure to set all NIs to the proper LND tunables
specified.

Add ntx tunable to dynamic configuration. This way all
tunables required to tune OPA performance can be configured
via lnetctl, allowing the ability to tune OPA network and IB
network differently

Test-Parameters: trivial
Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I015f3f959bd46784d4607bd4259b4640303dc362
Reviewed-on: https://review.whamcloud.com/27263
Tested-by: Jenkins
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9487 tests: stop primary and failover services in mmp test 1

In mmp.sh test_1(), there is a race of starting the primary and
failover services at the same time. Since it's likely the failover
service is started before the primary one, we need check and stop
the failover service.

Test-Parameters: trivial \
envdefinitions=MMP_EXCEPT=5 \
clientcount=4 osscount=2 mdscount=2 mdtcount=1 \
austeroptions=-R failover=true iscsi=1 \
testlist=mmp

Change-Id: I945f4eb7839400e75b372fc0cc9ad9455ccb5668
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: https://review.whamcloud.com/27253
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9526 spec: Improve systemd compat in spec file

Few minor edits that make the systemd compatibility
in the lustre.spec file cleaner.

Signed-off-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Change-Id: I02076c7806f2c7cb318a505943dca0ca5a030093
Reviewed-on: https://review.whamcloud.com/27215
Reviewed-by: Christopher J. Morrone <morrone2@llnl.gov>
Tested-by: Jenkins
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9289 nodemap: fix fileset string length issue

In nodemap_fileset_seq_write(), it is necessary to allocate count+1
for nm_fileset in order to have a '0' at the end of the string got
with copy_from_user().

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Id170e51648a8483c8d343d9f99f115bc04f798b3
Reviewed-on: https://review.whamcloud.com/26335
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9229 osc: osc page lru list race

In osc_lru_use() the osc page's ops_lru access is not protected, which
could race with osc_lru_del() and ensuing that
client_obd::cl_lru_in_list counter decreased twice for a single page.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I41179112693f77113fdf43d3f859bf778d7bbc44
Reviewed-on: https://review.whamcloud.com/26086
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9384 ldiskfs: extra patch for changing extra isize

Port a patch from upstream for changing extra size:

commit  887a9730614727c4fff7cb756711b190593fc1df
"ext4: keep existing extra fields when inode expands"

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I9555cb4efd1cbca257536ec699627e8fdb5a5b46
Reviewed-on: https://review.whamcloud.com/27285
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9549 lnet: prevent assert on ln_state

lnet_peer_primary_nid() is called from lnet_parse. It checks
ln_state outside the net lock, causing a race condition
during shutdown where the code expects the state to be
running, but it's stopping or shutdown.

Fixed the issue by renaming lnet_peer_primary_nid() to
lnet_peer_primary_nid_locked(). This function is now called
when lnet_net_lock is held in lnet_parse().

In lnet_create_reply_msg() we already have access to the
msg_txpeer, so we lookup the primary_nid directly

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I0518cdbec95b38bd8690517320b601676ae259f0
Reviewed-on: https://review.whamcloud.com/27262
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9384 ldiskfs: port extra isize patches to sles12sp2

Port 3 extra isize patches for sles12sp2 as below:

commit e3014d14a81edde488d9a6758eea8afc41752d2d
       "ext4: fixup free space calculations when expanding inodes"
commit 94405713889d4a9d341b4ad92956e4e2ec8ec2c2
       "ext4: replace bogus assertion in ext4_xattr_shift_entries()"
commit 887a9730614727c4fff7cb756711b190593fc1df
       "ext4: keep existing extra fields when inode expands"

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Iab0914d0010436aa26d99b78b2ae4acea783fb7d
Reviewed-on: https://review.whamcloud.com/27244
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-9490 llite: return v1/v3 layout for legacy app

Legacy app such as ADIO fetches LOVEA by ioctl LL_IOC_LOV_GETSTRIPE
and treats file layout as v1/v3 blindly, we'd return a reasonable
v1/v3 in this case.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I95694c3e32013c75da097f2f6ae4e8915503596e
Reviewed-on: https://review.whamcloud.com/27183
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9449 tools: component add to no stripe info file

When add component by 'lfs setstripe --component-add' to a
file without LOVEA, it'll be turned into a 'lfs setstripe' operation
internally.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I4ead7fae5ba2d0c4427b820e0506b507ef17a43b
Reviewed-on: https://review.whamcloud.com/26944
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>

LU-9424 tools: v1/v3 as composite layout

This patch changed 'lfs getstripe' & 'lfs find' to treat v1/v3
layout as single component composite layout, the component
attributes of the v1/v3 are:

component-count=0, (to distinguish with real composite file)
component-id=0,
component-flags=init/0 (file: init, dir: 0),
component-start=0,
component-end=eof.

When there is no specific component options provided,
'lfs getstripe' will keep the old output format of v1/v3.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ica5c958a964b16d26a7833ec2676c408f7f2e46c
Reviewed-on: https://review.whamcloud.com/27060
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9338 hsm: cache agent record locations

Create a cookie indexed hash of record locations for the agent actions
log. Use this cache to avoid linear scans of the log in HSM RPC
handlers that must update given entries of the log.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I1b9bb0746e88fb0bed81fe1174b5d3c7abac1211
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/26901
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9397 ksocklnd: move remaining time handling to 64 bits

Examination of the ksocklnd time handle revealed that the
code only requires second level precision. Since this is
the case we can move away from using jiffies to time64_t.
This allows us to be independent of the HZ settings in
addition to making it clear what is time handling, using
time64_t verses unsigned long. In the process we remove
many of the various libcfs time wrappers as well.

Change-Id: I968630ef94febd4bff703fb633e677996939f95b
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/26813
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9371 tools: handle component options properly

'lfs find' should match the file when any individual component of
the file matches all criteria. The component options should be
combined checking against individual component are:
--component-start, --component-end and --component-flags.

Fixed man page and usage message for 'lfs find':
'+N' actually means more than N, but not 'AT LEAST'
'-N' actually means less than N, but not 'AT MOST'

Test-Parameters: testlist=sanity-pfl
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ibdcd3167eb4b2de6d1754b9de563d3ec46828379
Reviewed-on: https://review.whamcloud.com/26766
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9448 lnet: handle empty CPTs

Make the LND code handle empty CPTs.

If a scheduler is associated with an empty CPT it will have no
threads created. If a NID hashes to that CPT, then pick the
next scheduler which does have at least 1 started thread.
Associate the connection with the CPT of the selected scheduler

Test-Parameters: trivial
Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I8ed83f295fe9852537d4bb063a4d8271c6a45c2c
Reviewed-on: https://review.whamcloud.com/27145
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2049 tests: Add GRANT_CHECK_LIST to sanity

Test GRANT_CHECK_LIST across a selection of sanity.

Fix check_grant() to account for grant precreate and grant pending and
subtract it from total grant on OSTs.

Test-Parameters: trivial osscount=2 mdscount=2 mdtcount=1 ostcount=2 mdtfilesystemtype=zfs ostfilesystemtype=zfs testlist=sanity
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I05b9f994033e8edf92bf30a17f74f2a0f6bee462
Reviewed-on: https://review.whamcloud.com/21619
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>

LU-9485 test: revert a change from LU-5361

Revert the changes from patch 5c315184152 in test-framework.sh
that caused wait_osc_import_state() not working.

Test-Parameters: trivial \
mdtfilesystemtype=zfs ostfilesystemtype=zfs \
testlist=conf-sanity,conf-sanity,conf-sanity

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ic20d5d7fa7344383d5999aa2684b736e64309ebb
Reviewed-on: https://review.whamcloud.com/27148
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8964 clio: Parallelize generic I/O

Add parallel version of cl_io_loop() function which use information
about stripes from LOV layer and process them in parallel.
This feature is disabled by default. To enable it you should run
"lctl set_param llite.*.pio=1" command.

IOR results on KNL for:
access             = file-per-process
ordering in a file = sequential offsets
ordering inter file= no tasks offsets
clients            = 1 (1 per node)
repetitions        = 1
blocksize          = 128 GiB
aggregate filesize = 128 GiB

xfsize pio Write Read
16 none 170.46 372.12
16 off 370.46 926.53
16 on 668.49 899.55
32 off 368.75 908.95
32 on 469.54 987.64

IOR results on Broadwell Xeon for:
access             = file-per-process
ordering in a file = sequential offsets
ordering inter file= no tasks offsets
clients            = 1 (1 per node)
repetitions        = 1
xfersize           = 16 MiB
blocksize          = 128 GiB
aggregate filesize = 128 GiB

pio Write Read
none 1419.80 1277.88
off 1348.98 2245.84
on  990.76 2320.08

The scalability IOR results on other Broadwell Xeon for:
access             = file-per-process
ordering in a file = sequential offsets
ordering inter file= no tasks offsets
repetitions        = 1
xfersize           = 4 MiB
blocksize          = 8 GiB

Threads pio Write Read
 32 off 9358.38 2649.28
 32 on 9147.14 2677.44
 64 off 8538.65 2811.05
 64 on 8944.19 2908.44
128 off 7978.61 2937.03
128 on 8613.91 2928.44

The numbers are in ‘MB/s’

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Id028faba1726fb377d0e903e8b8095d5ea9d1ee2
Reviewed-on: https://review.whamcloud.com/26468
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8964 libcfs: Introduce parallel tasks framework

In this patch new API for parallel tasks execution is introduced.
This API based on Linux kernel padata API which is used to perform
encryption and decryption on large numbers of packets without
reordering those packets.

It was adopted for general use in Lustre for parallelization of
various functionality. The first place of its usage is parallel I/O
implementation.

The first step in using it is to set up a cfs_ptask structure to
control of how this task are to be run:

    #include <libcfs/libcfs_ptask.h>

    int cfs_ptask_init(struct cfs_ptask *ptask, cfs_ptask_cb_t cbfunc,
                       void *cbdata, unsigned int flags, int cpu);

The cbfunc function with cbdata argument will be called in the process
of getting the task done. The cpu specifies which CPU will be used for
the final callback when the task is done.

The submission of task is done with:

    int cfs_ptask_submit(struct cfs_ptask *ptask,
                         struct cfs_ptask_engine *engine);

The task is submitted to the engine for execution.

In order to wait for result of task execution you should call:

    int cfs_ptask_wait_for(struct cfs_ptask *ptask);

The tasks with flag PTF_ORDERED are executed in parallel but complete
into submission order. So, waiting for last ordered task you can be sure
that all previous tasks were done before this task complete.

Test-Parameters: trivial
Change-Id: I36d9802226f9b6fae627594a86ad1a83f7bc6b0c
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/24474
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9513 liblustre: always check return value

Always check if the return value of __llapi_layout_cur_comp() is
NULL.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ibc803de5f27b981b0daa5e7be1a18adde9a46a18
Reviewed-on: https://review.whamcloud.com/27154
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9503  kernel: kernel update [SLES11 SP4 3.0.101-100]

Update SLES11 SP4 kernel to 3.0.101-100
Refresh one ldiskfs patch for ext4 context changes.

Test-Parameters: mdsdistro=sles11sp4 ossdistro=sles11sp4 \
  clientdistro=sles11sp4 mdtfilesystemtype=ldiskfs \
  ostfilesystemtype=ldiskfs testgroup=review-ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ia392004c12dda9d49c0f596b166f333e7fcae590
Reviewed-on: https://review.whamcloud.com/27132
Tested-by: Jenkins
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9459 kernel: kernel update [SLES12 SP2 4.4.59-92.17]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
  mdsdistro=sles12sp2 ossdistro=sles12sp2 \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Change-Id: I3b47a8c3a6f0f6c6f63dba97be769ab653fc6cb6
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-on: https://review.whamcloud.com/27095
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9482 hsm: release restore lock without object (part2)

In the cleanup phase of mdt_hsm_cdt_stop(), there is no need to
recursively unlock the mdt_objects that were to be restored.

Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I4d9212966722d4a2fcecc26210308ea51a200dcd
Reviewed-on: https://review.whamcloud.com/27033
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 libcfs: remove CFS_TIME_T definition

The CFS_TIME_T macro serves no real purpose as we stopped using time_t
and changed over to time64_t, so we can remove the last remaining uses
of this.

Two uses of this macro are incorrect and refer to jiffies values
rather than time_t, and one refers to an inode timespec that gets
changed separately.

Linux-commit: 93d3a405a168fba4450bdda793149e3cd4174736

Change-Id: I548ec8fffc9c46b8b2025b094f1e5d9cd469e3b7
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27025
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-9333 osd: few cleanups to support gcc6

few cleanups to support gcc6

Change-Id: Ibe664e87ff67eb3d34ce09c9a1892e55a63ab2cb
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/26584
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8956 llite: set sec ctx on client's inode at create time

On client side, security context must be set on the inode of
every new dir or file that is being created.

With LL_SBI_FILE_SECCTX, security context is obtained from the
dentry thanks to a call to ll_dentry_init_security(). And it is
saved to security.xxx xattr directly on MDS side when processing
the create request. So it is only necessary to call
security_inode_notifysecctx() to set the sec ctx on the client's
inode.

Without LL_SBI_FILE_SECCTX, security context can only be obtained
from the inode, ie after the file has been created on MDS side.
So use ll_inode_init_security() that will set the sec ctx on the
client's inode, and at the same time save it on disk to
security.xxx xattr, generating an additional request to the MDS.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I08b7828db6a4106cca0e78e57bed2967b86a922c
Reviewed-on: https://review.whamcloud.com/24426
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Jenkins
Reviewed-by: Jean-Baptiste Riaux <riaux.jb@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9090 ofd: increase default OST BRW size to 4MB

Increase the default maximum bulk RPC size reported by OSTs to the
clients to 4MB.  This has shown to improve performance for tested
workloads under a variety of conditions, and is already the defacto
standard for new installations.

Clients will still negotiate their preferred and supported BRW size
with the OSTs at connection time.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Id213e598bde14ed53b78ec230c7ae9aa5f3ebbe5
Reviewed-on: https://review.whamcloud.com/25336
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9507 lnd: Don't Assert On Reconnect with MultiQP

LU-8943 activated the ability to have multiple connections
between peers.  If any of those connections need to be
reconnected (i.e. parameter renegotiation), we were getting
an assert from kiblnd_reconnect_peer() which was not changed
to allow for having multiple connections ongoing at the
same time.

This patch gets rid of the assert from kiblnd_reconnect_peer()
which is no longer valid after LU-8943.

Signed-off-by: Doug Oucharek <doug.s.oucharek@intel.com>
Change-Id: I9cc7fae8836f2648603018fac38a88e3f90ec190
Reviewed-on: https://review.whamcloud.com/27139
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9312 hsm: convert cdt_llog_lock to a rw semaphore

Convert struct coordinator:cdt_llog_lock from a mutex to a rw
semaphore. Add a flag to cdt_llog_process() to indicate whether a read
or write lock is needed.

Change-Id: Ie5c14d8eccda22731ad0ad9594bee7db99fe9cfd
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Reviewed-on: https://review.whamcloud.com/26803
Tested-by: Jenkins
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8998 tools: parse optional argument properly

Use standard way '-S[opt_arg]' or '--long[=opt_arg]' to specify
optional arguments for 'lfs getstripe', the former way of parsing
optional argument is flawed.

Reverted use of '+' (at least), '-' (at most) for --component-start
and --component-end options.

Support "EOF/eof/-1" for the --component-end of 'lfs getstripe' &
'lfs find'.

Updated man pages and test scripts accordingly.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I51980e9186675e3668a4b40a1edc5da6d2a1801c
Reviewed-on: https://review.whamcloud.com/26491
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Revert "LU-8359 ldlm: Wrong evict during failover"

Mike points out that this patch actaully does not fix
anything and just adds another problem instead

This reverts commit c60e949e3b9f7ff19e1a644210cc764ee150ad8b.

Change-Id: I727eaee8dace8dd429451e4b6fcefbf66b940c30
Reviewed-on: https://review.whamcloud.com/27300
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9183 ptlrpc: handle posix_acl: xattr representation cleanups

In commit 2211d5ba5c6c4e972ba6dbc912b2897425ea6621 typedef of
posix_acl_xattr_{header,entry} was removed and replaced with
struct posix_acl_xattr_{header,entry} declaration. Also member
a_entries was removed from struct posix_acl_xattr_header.

Since we have to support older kernels this patch does the
opposite of what was done upstream in that we treat struct
posix_acl_xattr_{header,entry} of newer kernels as the typedef
used by older kernels.

Change-Id: I880f73d495dc6c35a1671b49e509350118f6e154
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/25892
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9521 tests: Avoid cleanup() being called twice.

/usr/bin/*-survey when ran,
- calls cleanup() twice, resulting in non-zero exit status,
- and attempts to cleanup already cleaned/detached echo client,
  resulting in invalid argument.

This patch fixes the same.

Test-Parameters: trivial testlist=obdfilter-survey
Signed-off-by: Parinay Kondekar <Parinay.Kondekar@seagate.com>
Change-Id: I5bb1fe5d7e6adf15e81d88501d6edbda1c5ef3c3
Reviewed-on: https://review.whamcloud.com/27160
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-8364 ldiskfs: fixes for failover mode for RHEL7.3

When ldiskfs runs in failover mode with read-only disk, it
may lose part of allocation updates and fail while mounting
fs due to group descriptor checks before journal replay.
Don't produce panics with on-disk checks in read-only mode.

Include ext4-dont-check-before-replay and ext4-dont-check-in-ro
patches in the RHEL7.3 series.

The check-ro patch should be ported in to RHEL7.3.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Icd4b899f9c48040c2453c4d759149e323fa33e18
Reviewed-on: https://review.whamcloud.com/27077
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9413 llite: llite.stat_blocksize param for fixed st_blksize

llite.stat_blocksize is added to allow configurable st_blksize
for stat(2). The latter is treated incorrectly by some
applications. For example, glibc pre-2.25 uses this value for
stdio buffering which completely ruins performance with random
reads.

The patch changes the behaviour of getattr rather than inode
initialization so that change of the setting causes immediate
effect without the need of reclaiming existing inodes.

The patch is similar to the patch from bz # 12739 by Aurelien
Degremont.

Change-Id: Ic6ab3fea40940892b740b8e87347dbb361619e8b
Signed-off-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Reviewed-on: https://review.whamcloud.com/26869
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@seagate.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9140 nrs: calculate the measured rate according to dd runtime

This patch help to calculate the measured IOPS by dd according
to dd runtime, instead of obtaining the result from the
performance data of dd output which value changes according to
its unit: 'MB/s', 'GB/s' or 'kB/s'.

Change-Id: Iace926d5c001f20b1c1d89ad6124d14a80316e86
Test-Parameters: trivial testlist=sanityn,sanityn,sanity
Signed-off-by: Qian Yingjin <qian@ddn.com>
Reviewed-on: https://review.whamcloud.com/27110
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>

LU-6707 tests: Add ability to skip tests in POSIX

Although the POSIX test suite only has a single sub test,
there are some times when you may need to exclude this
test from being run.

We need to add the ability to skip subtests in the POSIX
test suite.

Test-Parameters: trivial
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Idb051bc7ced02c9f09190874b5e7dacb3c6ad6ce
Reviewed-on: https://review.whamcloud.com/27012
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2049 grant: Fix grant interop with pre-GRANT_PARAM clients

Fix a performance regression with pre-GRANT_PARAM clients.
This patch brings the grant allocation logic back into line with how
servers prior to GRANT_PARAM support allocated grant space.

Test-Parameters: clientdistro=el6.6 clientjob=lustre-b2_7 clientbuildno=29 testlist=sanity-quota mdscount=1 mdtcount=1 ostcount=2 envdefinitions=SANITY_QUOTA_EXCEPT="36"
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I63fd5b91b499936b74d2dd21c1215e0b006af085
Reviewed-on: https://review.whamcloud.com/25853
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7988 hsm: change the cdt event flags to a simple boolean

Change the coordinator flag into a boolean event since only one bit is
now used. Do not include lustre_net.h as it's not necessary anymore.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: frank zago <fzago@cray.com>
Change-Id: I15dc10125af97f6244c98b421a262f320b79158b
Reviewed-on: https://review.whamcloud.com/22789
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

New tag 2.9.58

Change-Id: Ifb7a33a06cb953de0472693cd7d892a7685d8983
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9303 doc: man pages for snapshot components

Including the separated lctl-barrier and lctl-snapshot man
pages, the lctl-fork-lcfg and lctl-erase-lcfg sections in
the lctl man page. Also some other small cleanup.

Test-parameters: trivial

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ied60b824afcfab5d2d030f12d39c9aa1bab0969a
Reviewed-on: https://review.whamcloud.com/26453
Tested-by: Jenkins
Reviewed-by: Joseph Gmitter <joseph.gmitter@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 selftest: use jiffies_to_*() instead of cfs_duration_usec

The cfs_duration_usec() function has a timeval as its output, which we
want to avoid in general because of the y2038 problem.

There are only two locations remaining in lustre, so we can for now
eplace one with jiffies_to_timeval(), which is a generic kernel function
that does the same thing, the other can just use jiffies_to_usecs()
and completely avoid the timeval.

This is not a full solution yet, but it's a small step that lets us
build a larger portion of lustre without this reference to timeval in
a header file, and avoid triggering automated checking tools that wants
to warn about timeval.

Linux-commit: 70513c5d17b9812cc218e8b4c7826ebb5f375d9a

Test-Parameters: trivial testlist=lnet-selftest

Change-Id: If39f4d4857a2b3210bb0dc634b8bb42530df83dc
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/27019
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9412 lfs: A invalid memory write in llapi_layout_to_lum

When lum is realloc(), the comp_v1 needs to be updated, otherwise
it could point to the old invalid memory area.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I4493e5f13bce22dae07200bada14ba2349635890
Reviewed-on: https://review.whamcloud.com/26858
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9330 osp: make variables match proc tunables

Make the osp_syn_* and osp_max_rpcs_* function names consistent with
the tunable names in proc so that they are easier to find.

osp_max_rpcs_in_prog_seq_show()->osp_max_rpcs_in_progress_seq_show()
osp_max_rpcs_in_prog_seq_write()->osp_max_rpcs_in_progress_seq_write()
osp_syn_changes_seq_show()->osp_sync_changes_seq_show()
osp_syn_changes_seq_write()->osp_sync_changes_seq_write()
osp_syn_in_flight_seq_show()->osp_sync_rpcs_in_flight_seq_show()
osp_syn_in_prog_seq_show()->osp_sync_rpcs_in_progress_seq_show()

This entails renaming the osp_syn_* function names consistent with the
function names, so change _syn_ -> _sync_, _prog_ -> _progress_,
and _rpc_ -> _rpcs_.

Make osp_sync_check_for_work() a proper function rather than having
both a macro and __osp_sync_check_for_work() as a function that just
calls the macro.

Remove unused field opd_syn_sync_in_progress.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I537e04fdce7eea194f1a9567b6e5c3ccee2cab07
Reviewed-on: https://review.whamcloud.com/26571
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6401 uapi: turn lustre_param.h into a proper UAPI header

Move all the kernel specific function prototypes from
lustre_param.h into obd_config.h which is a kernel only
header. The inline functions lustre_is_*_valid are used
only by user land so we can remove them. Remove the user
land error checking with lustre_is_*_valid() since its
the job of the kernel to validate the date passed in.
The libcfs.h header shouldn't be exposed to user land
so remove it.

Change-Id: I6b6b4fe8f4d6799608c0e74318afecb85168ad54
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/24325
Tested-by: Jenkins
Reviewed-by: Ben Evans <bevans@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9019 libcfs: migrate watchdog to 64 bit time

The watchdog computes timestamps from timeval, which overflows
in 2038 on 32-bit systems. This changes the results output to
timespec64 type to avoid the overflow. The lcw_last_touched
field is changed to ktime_t since better than seconds resolution
is wanted. For lcw_last_watchdog it is changed to time64_t since
we only care about seconds percision. Both changes will avoid
the 2038 overflow issue and the HZ variablity across platforms
for jiffies.

Change-Id: If81f60a408db956540c563fc695f729cd67cdd9e
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/26994
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8659 tests: use runcon in sanity-selinux

In order to switch to other SELinux context, use runcon
instead of 'ssh user@localhost'.
This requires the SELinux policy to allow transitions from
unconfined_t to user_t and guest_t:
allow unconfined_r guest_r;
allow unconfined_r user_r;

Test-Parameters: trivial clientselinux testlist=sanity-selinux
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I8f45dfe71d95d03af0c3577f46c91b47232d958a
Reviewed-on: https://review.whamcloud.com/23962
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9491 llite: Handle multi-vec append write correctly

http://review.whamcloud.com/20256 cleaned up LLITE code by using
iov_iter for old and new kernels. However, it introduced a bug to
append write with multiple iovec buffers where each buffer was written
separately therefore each of them saw the same file size.

Append write with multiple buffers has to be performed atomically.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I89d59e924c7029fedc096d724a946763c7f7006d
Reviewed-on: https://review.whamcloud.com/27086
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9384 ldiskfs: port upstream patches for changing extra isize

Port upstream 5 patches for changing extra isize as below:

commit d0141191a20289f8955c1e03dad08e42e6f71ca9
"ext4: fix xattr shifting when expanding inodes"
commit 418c12d08dc64a45107c467ec1ba29b5e69b0715
"ext4: fix xattr shifting when expanding inodes part 2"
commit 443a8c41cd49de66a3fda45b32b9860ea0292b84
"ext4: properly align shifted xattrs when expanding inodes"
commit e3014d14a81edde488d9a6758eea8afc41752d2d
"ext4: fixup free space calculations when expanding inodes"
commit 94405713889d4a9d341b4ad92956e4e2ec8ec2c2
"ext4: replace bogus assertion in ext4_xattr_shift_entries()"

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I01414bcc91d8f57ca72281916d35536d3926e570
Reviewed-on: https://review.whamcloud.com/27045
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9472 lnd: Fix FastReg map/unmap for MLX5

The FastReg support in ko2iblnd was not unmapping pool items
causing the items to leak.  In addition, the mapping code
is not growing the pool like we do with FMR.

This patch makes sure we are unmapping FastReg pool elements
when we are done with them.  It also makes sure the pool
will grow when we depleat the pool.

Signed-off-by: Doug Oucharek <doug.s.oucharek@intel.com>
Change-Id: I4b4ba4de72941b38c4115a00a992cfd1e78e9e49
Reviewed-on: https://review.whamcloud.com/27015
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9340 lov: Initialize component extents unconditionally

LOV should initialize extent of components even they are not
instantiated because lov_io_iter_init() relies on this to issue
write intent RPC.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Iaa452e3a0f3186cfea77866b7a1fabbc9feeec53
Reviewed-on: https://review.whamcloud.com/27097
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9522 test: sanity 27z fix update local variable

sanity test_27z is failing for servers running ZFS. The
problem is that filter_fid is not being updated for each
stripe of the file being reviewed in check_seq_oid().

For each stripe, the variable that holds the filter_fid,
ff, needs to be reinitialized.

Test-Parameters: trivial testgroup=review-zfs-part-1

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I550720a0a2bad22c4ffa98ca84b5f085eb220051
Reviewed-on: https://review.whamcloud.com/27163
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-9514 test: sanity 51f add to ALWAYS_EXCEPT

sanity test 51f is failing many times for servers with
ZFS file systems. This test sould be skipped for ZFS
testing until a solution is found.

Test-Parameters: trivial testgroup=review-zfs-part-1
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ia4b035d158cdf8e2179f74c7f1f6b0b382a93663
Reviewed-on: https://review.whamcloud.com/27189
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Revert "LU-9439 scripts: Change behavior of lustre_rmmod"

This is causing widespread testing problems partially documented in LU-9524 and also in LU-9439

This reverts commit 645153be3eb1fd8c634717507f73d85625d1b84a.

Change-Id: Ie20b9c7a52aec5153566ad0712689852545a0948
Reviewed-on: https://review.whamcloud.com/27181
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7988 hsm: change coordinator start/stop mechanisms

Instead of using cdt_state and cdt_thread.t_flags to keep the state of
the coordinator, only use cdt_state. cdt_thread.t_flags is now only
used to signal the coordinator, not to stop it. cdt_state is used to
control the coordinator behaviour.

Split mdt_hsm_cdt_stop() into 2 functions. One to actually signal the
coordinator to stop, and another one to cleanup its ressources. The
coordinator is now responsible to clean its own ressources instead of
having two different paths depending on whether Lustre is shutting
down, or a user is stopping it through /proc.

Protect cdt_state with a spin lock, and add a transition table to
catch invalid transitions.

Removed mdt_opts.mo_coordinator as it is just a subset of cdt_state.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: frank zago <fzago@cray.com>
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I7b0a878792d287b781578a7afe4e2c2cf7dec5cc
Reviewed-on: https://review.whamcloud.com/22667
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Jean-Baptiste Riaux <riaux.jb@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9439 scripts: lnet systemd service

Create an lnet systemd service which properly
brings lnet up and down.

Test-Parameters: trivial
Signed-off-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Change-Id: I584827dbb1fc4e0999b7f107bb4250678b7b68e8
Reviewed-on: https://review.whamcloud.com/26925
Tested-by: Jenkins
Reviewed-by: Ned Bass <bass6@llnl.gov>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9415 lfsck: quiet noisy console message

Mark LFSCK console startup message D_LFSCK unless it is an error.

Test-Parameters: trivial testlist=sanity-lfsck
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ia3e8bc1cfbfa8a19c56626feb7e88e4c544ebbe5
Reviewed-on: https://review.whamcloud.com/27059
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9477 tests: check correct handling of dead object

Since original fix to allow for correct handling of dead/removed
object has been finally implemented LU-9312
This patch now only introduces related tests of correct handling
in sanity-hsm test suite.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I1e3ce88909e8820657ee5753fea5d6757c354574
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/27018
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9464 hsm: use OBD_ALLOC_LARGE() for hsm_scan_data array

In mdt_coordinator() use OBD_ALLOC_LARGE() rather than OBD_ALLOC() for
the hsm_scan_data request array.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ieb648dfb92e6019ab316c7643aa0c0b5cf1d86f7
Reviewed-on: https://review.whamcloud.com/27014
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9119 lnet: Fix deleting peers from YAML

Deleting peers with lnetctl command
"import --del < config.yaml" throws error

The above command deletes prim_nid first
and then tries deleting the other nids which
results in error, since deleting the primary_nid
deletes the entire peer and then after that we try
to delete non-existent NIDs.

The behavior should be if the primary_nid is
present in the list of NIDs then delete the
entire peer, otherwise delete only the NIDs
specified within the peer

Test-Parameters: trivial
Signed-off-by: Sonia Sharma <sonia.sharma@intel.com>
Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I55114fca4d332c950872bd446e02e4f0904ee716
Reviewed-on: https://review.whamcloud.com/27001
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9315 pfl: static analysis issues

1. Buffer Overflow - Non-null Terminated String
 * lustre/utils/liblustreapi_layout.c: in llapi_layout_expected,
   Buffer overflow of 'donor_path' due to non null terminated string
   'donor_path'
2. Use of Freed Memory by Pointer
 * lustre/utils/liblustreapi_layout.c: in llapi_layout_comp_del,
   Object 'comp' was dereferenced at line 1770 after being freed by
   calling '__llapi_comp_free' at line 1769
3. Result of function that may return NULL will be dereferenced
 * lustre/lov/lov_pack.c: in lov_unpackmd, Pointer
   'lsm_op_find(magic)' returned from call to function 'lsm_op_find'
   at line 334 may be NULL and will be dereferenced at line 334.
4. Uninitialized Variable - possible
 * lustre/utils/liblustreapi.c: in find_check_comp_options, 'ret'
   might be used uninitialized in this function. Also there are 2
   similar errors on lines 3243, 3264.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I397737affeaa409e97b0ed859efcd7ff2840cc89
Reviewed-on: https://review.whamcloud.com/26503
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

LU-9241 llite: ASSERTION( de->d_op == &ll_d_ops ) failed

The assertion can be checked while dentry isn't fully
initialized in HAVE_DCACHE_LOCK case.

Reorder assignments to make assertion always true.
Later ll_dentry_data bitfields modification should be
protected by a spinlock.

Change-Id: I4a1ea42b9fe1c9398539a1a241b8a191dba5903c
Seagate-bug-id: MRP-4257
Signed-off-by: Andriy Skulysh <andriy.skulysh@seagate.com>
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-on: https://review.whamcloud.com/26109
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9202 lfsck: skip unavailable targets when sync failures

It is normal that some target (MDT or OST) may become unavailable
when the LFSCK engine tries to sync failures with related target.
Under such case, just skip such target without LBUG.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ib677e6fec121e946caafb34ebc71b8b3068bd6f5
Reviewed-on: https://review.whamcloud.com/25931
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9183 llite: remove struct file on stack in ll_setxattr()

Backport upstream commit c139f3cef36291902d7a29a334acd25a1251cd9d

Change-Id: I3a7ddf9a2abed8692a203c4d88b9a8ecdaeb9a1a
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/25893
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9183 ptlrpc: handle changes in struct group_info

In commit 81243eacfa400f5f7b89f4c2323d0de9982bb0fb few members of
struct group_info where changed.

Change-Id: Id8e5b483cf389eff41c63942cfd78303fff67a8c
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/25838
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8730 test: Remove duplicate $dev from conf-sanity test_83

Remove duplicate $dev variable passed to add function, which
passes it as argument to mkfs.lustre. mkfs.lustre utility in
Lustre 2.1 throws error when multiple device names passed to it.
Skip test if OSTs are not ldiskfs-based.

Test-Parameters: trivial envdefinitions=ONLY=83 testlist=conf-sanity
Change-Id: I56535aac27e25bbf359bda835b24a56a4101c8ed
Seagate-bug-id: MRP-2239
Signed-off-by: Ashish Maurya <ashish.maurya@seagate.com>
Signed-off-by: Sushant Mane <sushant.mane@seagate.com>
Reviewed-on: http://es-gerrit.xyus.xyratex.com:8080/9425
Tested-by: Jenkins
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@seagate.com>
Reviewed-by: Ujjwal Lanjewar <ujjwal.lanjewar@seagate.com>
Tested-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Signed-off-by: Ashish Maurya <ashish.maurya@seagate.com>
Reviewed-on: https://review.whamcloud.com/23249
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8589 osd: remove "object" from method function names

Remove "object_" from various OSD API dt_object_operations
method functions (create, destroy, {read,write}_{lock,unlock},
write_locked, ref_add, ref_del, and declare_ variants of same)
to match the actual method names so the code is easier to trace.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I18860bc965f958d6ba2e595882e7e56ca00cab07
Reviewed-on: https://review.whamcloud.com/22542
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8359 ldlm: Wrong evict during failover

There is a race between setting obd_fail & OBD_OPT_FAILOVER.
tgt_client_del() checks only OBD_OPT_FAILOVER,
class_disconnect_export_list() is called with flags copied
from obd, and umount can start while disconnect
is in progress.

It is better to rely only on obd_fail.
We shouldn't evict during failover at all, it should
be handled on a new server.
Such wrong evict can happen when server can't send CP AST
to the client because failover has started already.

Change-Id: I649d35d180b2239fe558b375872d3805629968a9
Seagate-bug-id: MRP-3604
Signed-off-by: Andriy Skulysh <andriy.skulysh@seagate.com>
Reviewed-on: https://review.whamcloud.com/21114
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@seagate.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9305 osd-zfs: arc_buf could be non-pagesize aligned

ZFS only guarantees PAGE_SIZE alignment to arc_buf_t only when
the block size is not less than (PAGE_SIZE << 2).

The patch for ZFS https://github.com/zfsonlinux/zfs/pull/6084 fixes
the alignment problem, buf Lustre still needs a fix to handle
the problem in case it's running old ZFS release.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I6fd17d7b20499ec0406a3e10cebf6882b92a730f
Reviewed-on: https://review.whamcloud.com/26895
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-8670 tests: test_115 Fixes & Improvements

This patch address below points in sanity test_115.
- Do not skip the test when OST is remote.
- Fail Test if ll_ost_io thread has not started.
- Fail Test if ll_os_io  post count is less then
equal to ll_ost_io pre count.
- Fail Test if the count of started thread more
than thread_max.

Test-Parameters: trivial envdefinitions=SLOW=yes testlist=sanity
Signed-off-by: Arshad Hussain <arshad.hussain@seagate.com>
Signed-off-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Seagate-bug-id: MRP-981
Change-Id: I51e434a15ee78db59ea5c091b3ec7d7fcf0bab64
Reviewed-by: Ashish Purkar <ashish.purkar@seagate.com>
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@seagate.com>
Reviewed-on: https://review.whamcloud.com/22950
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6952 mount: Mount options parsing problem

The code which parse the mount option expect that the default
mount options come first and then the user given options. But
actually the default options are appended after the user given
option. Because of this user given options are cleared.
Following code fix this bug.

Added a conf-sanity test 104 to verify this fix.

Seagate-bug-id: MRP-2193
Signed-off-by: Pratik Shinde <pratik.shinde@seagate.com>
Signed-off-by: Rahul Deshmukh <rahul.deshmukh@seagate.com>
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Change-Id: I1a22e5d559e01d8bcba0ceebba5544e92d39bb19
Reviewed-on: https://review.whamcloud.com/15906
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Grégoire Pichon <gregoire.pichon@bull.net>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9425 lnd: Turn on 2 sges by default

Currently, the fix in LU-5718 which allows for multiple sges
to deal with RDMA fragmentation is turned off by deafult
(set to 1).  This patch changes the default to 2 so
RDMA fragmentation is fixed by default.

Test-Parameters: trivial
Signed-off-by: Doug Oucharek <doug.s.oucharek@intel.com>
Change-Id: I8a29a7b32ababd37cbc471664083362bc7253d97
Reviewed-on: https://review.whamcloud.com/26911
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ned Bass <bass6@llnl.gov>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7584 tests: clean up sanity test_129

Clean up sanity test_129 to follow Lustre script coding style.
There is no need to create so many files to verify that the
directory size limit has been removed.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ib80bd8a70386b3e8881f8ca3d417a8be1becab07
Reviewed-on: https://review.whamcloud.com/26874
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9362 lfs: getstripe print last init-ed component info

If getstripe asks for part of stripe information of a PFL file,
lfs should print the last instantiated component stripe info.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Iebdc4d44e47a25ef8445fda240b25975f8dbcbbe
Reviewed-on: https://review.whamcloud.com/26947
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-8998 tests: improve sanity.sh::check_seq_oid()

Enable the use of debugfs to decode the OST "fid" xattr to
avoid an unmount/remount cycle of the OST if possible.

Improve sanity.sh::check_seq_oid() in test_27z to be agnostic
about field position, and improve overall code style to avoid
external command usage.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ia971e33cc3d8a5e4ca6f821116f12c0a72bcab07
Reviewed-on: https://review.whamcloud.com/26569
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8998 tools: support negative flags

- Make 'lfs setstripe --component-flags' support negative flags
  "^init".
- Change llapi_layout_file_comp_del() to accept both 'id' and
  'flags', 'id' must be real ID, 'flags' can be negative flags.
- Fix swab defect in lod_declare_layout_del().
- Update man pages and test scripts accordingly.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I9b66492133f56eabc928dbbb41bb8eb5627be095
Reviewed-on: https://review.whamcloud.com/26490
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>

LU-7473 acl: increase ACL entries limitation

Originally, the limitation of ACL entries is 32, that is not
enough for some use cases. In fact, restricting ACL entries
count is mainly for preparing the RPC reply buffer to receive
the ACL data. So we cannot make the ACL entries count to be
unlimited. But we can enlarge the RPC reply buffer to hold
more ACL entries. On the other hand, MDT backend filesystem
has its own EA size limitation. For example, for ldiskfs case,
if large EA enable, then the max ACL size is 1048492 bytes;
otherwise, it is 4012 bytes. For ZFS backend, such value is
32768 bytes. With such hard limitation, we can calculate how
many ACL entries we can have at most. This patch increases
the RPC reply buffer to match such hard limitation. For old
client, to avoid buffer overflow because of large ACL data
(more than 32 ACL entries), the MDT will forbid the old client
to access the file with large ACL data. As for how to know
whether it is old client or new, a new connection flag
OBD_CONNECT_LARGE_ACL is used for that.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I557d2696a8df84bf83fcd3955292227d7aa0284c
Reviewed-on: https://review.whamcloud.com/19790
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9096 test: add sanity 253 to ALWAYS_EXCEPT

Discussion in the LU-9096 ticket questions if sanity
test 253 is a valid/correct test. Since tes 253 is failing
intermittently for ZFS servers add the test to the
ALWAYS_EXCEPT list until the test can be improved.

Since the correctness of the test is being questioned,
skip the test for all server file systems, not just ZFS.

Test-Parameters: trivial
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I14cffb288b4d338ff009f4b57a19d5a5f4c516ff
Reviewed-on: https://review.whamcloud.com/27013
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9458 ptlrpc: handle case of epp_free_pages <= PTLRPC_MAX_BRW_PAGES

Current code where page_pools.epp_free_pages is too small isn't
handled correctly.  This mod fixes those instances.

Change-Id: I9242cbdeba9f5ea4836189f0049ae3617cd665f7
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-on: https://review.whamcloud.com/27016
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-8294 gss: quiet cache_check return ENOENT warning

gss_svc_upcall_handle_init spews a lot of error messages if GSS
support has been compiled, but GSS is not being used. The current
error message is not very useful, so this patch moves it to CDEBUG.

Test-Parameters: trivial
Signed-off-by: Kit Westneat <kit.westneat@gmail.com>
Change-Id: Id58e3a9b5c702f8669f6659c6ff8e577391484de
Reviewed-on: https://review.whamcloud.com/26976
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9456 lnd: Change sock_create() to sock_create_kern()

Changing all calls in the ksocklnd from sock_create() to
sock_create_kern().

Signed-off-by: Doug Oucharek <doug.s.oucharek@intel.com>
Change-Id: Ib8b175e73478b1edfb5e8cd3491e589e8267f52a
Reviewed-on: https://review.whamcloud.com/26958
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Olaf Weber <olaf.weber@hpe.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9439 scripts: Change behavior of lustre_rmmod

The lustre_rmmod script was modified to take an arbitrary list of
modules and try to remove them and any modules dependent on them.
Previously its behavior was to always remove the libcfs module,
along with either the ldiskfs or another module passed as a
parameter.

The old interface was roughly maintained. Any of the following
commands will remove the ldiskfs, libcfs, and all dependent modules:

    $ lustre_rmmod
    $ lustre_rmmod ldiskfs
    $ lustre_rmmod ldiskfs libcfs

The benefit now, is that any other list of modules can be specified
without removing libcfs. For example, the following command will only
remove ptlrpc and its dependent modules (leaving libcfs intact):

    $ lustre_rmmod ptlrpc

The lnet init script was modified to perform a lustre_rmmod ptlrpc
before performing an lctl network down. By removing the ptlrpc
module, we can ensure that lnet is not in use. This will help
systems running lustre to shut down cleanly.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Signed-off-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Change-Id: If9681ebd785b6b5920e7d1553333ad8d6120de56
Reviewed-on: https://review.whamcloud.com/26959
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Ned Bass <bass6@llnl.gov>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-8346 obdclass: guarantee all keys filled

In keys_fill, the key_set_version could be changed after
the keys are filled, then the keys in this context won't
be refilled by the following lu_context_refill for its
verion is equal to the current key_set_verion.

Change-Id: Ibaa49ec6e95ffee902cfa98f18ac9e66f2127bf1
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: https://review.whamcloud.com/26099
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7567 utils: fix timestamp printing in lfs_changelog()

lfs_changelog() should use ".09d" when printing the fractional part
instead of ".06d" for accurate timestamps.

Test-Parameters: trivial
Signed-off-by: akam kumar bharathi <azurelustre@gmail.com>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I2b219f588de688ba06b3c8cf3ccd255ed845c45e
Reviewed-on: https://review.whamcloud.com/18006
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7108 test: Remove sanityn tests from ALWAYS_EXCEPT list

Removing sanityn tests from Always_Except list. All the
tests on sanityn's ALWAYS_EXCEPT list ran about 40 times
with no errors except for test_28 and test_29.

Since tests 14b, 19 and 35 ran with no failures, therefore
removing them from ALWAYS_EXCEPT list.

Test-Parameters: trivial testlist=sanityn,sanityn,sanityn
Test-Parameters: testgroup=review-zfs-part-1
Test-Parameters: testgroup=review-dne-part-1
Signed-off-by: Saurabh Tandan <saurabh.tandan@intel.com>
Change-Id: Icac5201ba27f010707f536287fa4478ff5afcfd6
Reviewed-on: https://review.whamcloud.com/16383
Tested-by: Jenkins
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9324 lnet: move cyaml.h under lnet/include/

So that it can be included in other projects.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I980578742fd194e2464870f1ab8d6a9ae8deb9e2
Reviewed-on: https://review.whamcloud.com/26859
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>