LU-8900 doc: Lustre snapshot man page

To describe how to use barrier and snapshot commands.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ia1833ddd2949aaf5b63411214310c7792b41384d
Reviewed-on: https://review.whamcloud.com/24270
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8900 snapshot: user space snapshot tools

The user space snapshot tools are implemented as part of the
lctl tools set. They use other user-space tools, such as lctl
commands for write barrier on all MDTs, fork/erase Lustre
configuration, zfs/zpool commands for backend zfs snapshot.

1. Create the snapshot
lctl snapshot_create [-b | --barrier [on | off]]
[-c | --comment comment] <-F | --fsname fsname> [-h | --help]
<-n | --name ssname> [-r | --rsh remote_shell]
[-t | --timeout timeout]

2. Destroy the snapshot
lctl snapshot_destroy [-f | --force] <-F | --fsname fsname>
[-h | --help] <-n | --name ssname> [-r | --rsh remote_shell]

3. Modify the snapshot
lctl snapshot_modify [-c | --comment comment]
<-F | --fsname fsname> [-h | --help] <-n | --name ssname>
[-N | --new new_ssname] [-r | --rsh remote_shell]

4. Query the snapshot(s)
lctl snapshot_list [-d | --detail] <-F | --fsname fsname>
[-h | --help] [-n | --name ssname] [-r | --rsh remote_shell]

5. Mount the snapshot
lctl snapshot_mount <-F | --fsname fsname> [-h | --help]
<-n | --name ssname> [-r | --rsh remote_shell]

6. Umount the snapshot
lctl snapshot_umount <-F | --fsname fsname> [-h | --help]
<-n | --name ssname> [-r | --rsh remote_shell]

The system config information, such as each target's hostname,
pool name, local filesystem name, role, index, and so on, will
be written in the snapshot config file "/etc/ldev.conf". To be
compatible with old snapshot usage, it will firstly check whether
the "/etc/ldev.conf" exists or not, it yes, it will load the
configuration from such file preferentially, otherwise, the file
"/etc/lsnapshot/${fsname}.conf" will be used.

Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs mdscount=2 mdtcount=4 testlist=sanity-lsnapshot
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I1191c2ea35c355e8554bd2bd022c1b9d32741392
Reviewed-on: https://review.whamcloud.com/24269
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9231 tests: cleanup obdecho before iokit exit

Otherwise under some failure cases, it may leave the
FID_SEQ_LOCAL_NAME local object which is used by the
obdecho in the system, then the subsequent tests can
NOT exit cleanly.

Lustre-commit: c5675497b411d8b3f0f08fec3271fd528b4a52cf
Lustre-change: http://review.whamcloud.com/23738

Test-Parameters: alwaysuploadlogs envdefinitions=PTLDEBUG=-1,DEBUG_SIZE=150 mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs clientdistro=el7 ossdistro=el7 mdsdistro=el7 mdscount=2 mdtcount=4 testlist=mds-survey,mmp,mds-survey,mmp,mds-survey,mmp
Signed-off-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I741cf20875df679ecdf72d5461cc14eec166e3ae
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-on: https://review.whamcloud.com/26092
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9223: Server on Linux 4.4: XATTR_NAME_POSIX_ACL_ACCESS

Add linux/xattr.h to Lustre_compat.h to allow the server
code to Build in Linux 4.4.

Change-Id: Iccf87acffc04d7c1084a384ae146d1077eed53fd
Signed-off-by: Keith <keith.mannthey@intel.com>
Reviewed-on: https://review.whamcloud.com/26076
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9197 llog: return 0 if llog destroy fails

In llog_cancel_rec(), let's return 0 if only destroying the
final plain llog fails, because the bitmap has been cleared
successfully, and the orphan supposed to be handled by LFSCK
anyway. So let's not try to destroy again, which will pretty
much complicate the code here.

And also in this case, we do not need restore the llh_count and
bitmap, because the record has been deleted for llog anyway.

Signed-off-by: Di Wang <di.wang@intel.com>
Change-Id: I762def0f0de08a2960997ab9463f4e72fbad531d
Reviewed-on: https://review.whamcloud.com/26069
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9213 scripts: check MGT status in lustre init script

This patch fixes health_check() in lustre init script to
account for the status of MGT on a dedicated node.

Test-Parameters: trivial

Change-Id: Ifdf2e4a1002e870c25925b6e7fce82b132cc24f2
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: https://review.whamcloud.com/25995
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6051 utils: improve efficiency of lfs_migrate cmp

The file comparison used by lfs_migrate could be more efficient by
using the "cmp -s" option to avoid printing byte-by-byte differences.
This allows faster comparisons via memcmp() and comparing the file
size.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I424538388d7e31609fa1f875128f03fef33ebbe5
Reviewed-on: https://review.whamcloud.com/25878
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9170 test: fix for hostlist_expand

Fix string substitution in hostlist_expand. It
may produce a wrong result in some case.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Iba33833ec4955dafeeefe5042508a0162c2d477e
Reviewed-on: https://review.whamcloud.com/25698
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1095 obdclass: improve missing operation message

Some tests in the past reported missing OBD operations, so improve
the error message to include the device name to make it easier to
debug where this is happening.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Id49ab801158690510f9ac191c4e8f7a253bcab07
Reviewed-on: https://review.whamcloud.com/25586
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9085 llstat: don't use a hash as a reference

This patch is to fix "Using a hash as a reference is deprecated"
in llstat file. In the newer perl version, we should use "$xxx"
instead of "%xxx" for a hash reference.

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I67a1d1e365076b412550de775b115ac373589c60
Reviewed-on: https://review.whamcloud.com/25320
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8907 llite: handle client racy case during create

Some very infrequent situations exists on client side
able to cause a race during create when concurrent access
by fid occurs. The result of the race can allow a d_alias
to be already present when it was not expected when original
code/LBUG has been written.

One of the identified scenario is when a concurrent access of
inode thru the .lustre/fid/<[FID]> method occurs.

New sanity/test_161d has been added to reproduce this scenario.

Final fix is to remove inaccurate
LASSERT(ll_d_hlist_empty(&inode->i_dentry));
in ll_create_node().

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I1ff19883cc5b53831f1d5c577f4152225fa9e0fb
Reviewed-on: https://review.whamcloud.com/25296
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8930 lfsck: sync failure with others no check LF_INCOMPLETE

During the first stage scanning, if the MDT has ever failed to
verfiy some OST-objects (related OSTs inaccessable temporarily
because of network trouble or OSTs restart), it needs to sync
failures with other MDTs at the end of the first stage scanning
to avoid improper repairing in the second stage scanning. Such
sync failure should be done even if the "LF_INCOMPLETE" is set
locally, otherwise, other MDTs may have no chance to know that.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I8466d8560b5d5070873f10746189fd82bf6f0172
Reviewed-on: https://review.whamcloud.com/24900
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8403 obd: remove unused data parameter from obd_notify()

Remove the unused data parameter from obd_notify() and related
functions.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I1ae72de176ae71441cf486ad6a10c45d57b32aad
Reviewed-on: https://review.whamcloud.com/24428
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8808 mdt: avoid out of order ChangeLog during restore

In hsm_cdt_request_completed(), upon restore completion, layout
lock is returned before associated ChangeLog entry has been created.
This allows for a timing window where concurrent file updater can
take control and create its own ChangeLog out of order (before
restore ChangeLog).
This can puzzle, ChangeLog readers (like RobinHood), and have them
think of a wrong file state.
This patch fixes this hole by posting restore ChangeLog before
giving back layout lock.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I958cb61aab34a6a0edc97f4d1390eed623631c21
Reviewed-on: https://review.whamcloud.com/23638
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8701 tests: scrub-performance with separated MGS and MDS

Support to run scrub-performance under the environment of MGS and
MDS on separated devices.

Test-Parameters: trivial envdefinitions=SLOW=yes combinedmdsmgs=false testlist=scrub-performance
Test-Parameters: trivial envdefinitions=SLOW=yes testlist=scrub-performance
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I5b01097f3e73c2e0eca6071ecb597ac84893c77d
Reviewed-on: https://review.whamcloud.com/23124
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-137 obdclass: add dt_object_put() and use it

Introduce dt_object_put() and dt_object_put_nocache() and use them
instead of lu_object_put() and lu_object_put_nocache(), to balance
the use of dt_object_locate() to make a symmetrical interface.

Move the existing ofd_object_put() and osd_object_put() wrappers
into common header functions to use instead of dt_object_put()
or lu_object_put() calls in the respective code, when possible.

Functionally, this patch is a no-op.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I7ef657438441de4a91ee08edb4bc097ac97b7418
Reviewed-on: https://review.whamcloud.com/23092
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5969 tests: allow "version" without "lustre:" prefix

Fix lustre_build_version() to handle if "lctl get_param version"
returns a bare version number like "2.9.54" as done with the
upstream kernel client and once https://review.whamcloud.com/25632
lands, instead of the old prefixed "lustre: 2.8.53" version string.

This was already fixed in "lctl lustre_build_version" in patch
https://review.whamcloud.com/25324 but due to the order of checks
in lustre_build_version() the fallback of using "lctl --version"
took precedence, which returns only the build version of the lctl
tool itself and not the Lustre module version.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I1b0d87d186d234b62aa59bcadde42452323ebbe5
Reviewed-on: https://review.whamcloud.com/26047
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8872 quota: incorrect LASSERT

Remove an incorrect LASSERT in qsd_add_deferred().

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I7ea0274032ac67e3a039a0a0982d5945498fd434
Reviewed-on: https://review.whamcloud.com/24024
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9040 scrub: handle group boundary properly (2)

In the osd_inode_iteration() logic, if the next/exec method
sets the iteration @offset variable just as group boundary:
the LDISKFS_INODES_PER_GROUP(sb), the original logic cannot
incrase the group#, as to the iteration repeats in the same
groupc for ever.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ie55d8ffaed2257baed9761a30c09c62e23105434
Reviewed-on: https://review.whamcloud.com/26059
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>

LU-9216 config: move config types into lustre_idl.h

Move config type values CONFIG_T_XXX into lustre_idl.h since they
will be put on wire when reading config log for IR and nodemap.

Add missing wire checks for mgs_nidtbl_entry, mgs_config_body and
mgs_config_res.

Redefine CONFIG_SUB_XXX for the sub clds attached on config log.

Test-Parameters: clientjob=lustre-b2_7_fe clientbuildno=133 testlist=runtests

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I9eb7a89c0fee0c11666bc8f587d3fd5e2ad4fc8d
Reviewed-on: https://review.whamcloud.com/26022
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7497 tests: Fix test failure in conf-sanity 32b

It turned out that the MGS svc was wrongly turned on in disk image
'disk2_4-zfs.tar.bz2', which caused the OST mount failure with
EALREADY because the MGS service has already been started by the MDS.

This is just a temporary solution and the final fix would be to clean
the disk image in question.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I912d39c08ba1437078fe7126d4498950ebeadaf1
Reviewed-on: https://review.whamcloud.com/25940
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9102 test: Include specific headers needed by Posix suite

Change prefix to the missing headers on el7

Change-Id: I79d59104a224ccf51795072e2a12da1a113cdaa0
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-on: https://review.whamcloud.com/25606
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9026 o2iblnd: Adapt to the removal of ib_get_dma_mr()

In Linux kernel 4.9-rc1, the function ib_get_dma_mr()
was removed and a second parameter was added to ib_alloc_pd().

This patch fixes this breakage by:

- Detect if ib_get_dma_mr() has been removed, if so, do the
  following:
- Make it so the module parameter map_on_demand can no longer be
  zero (we have to configure FMR/FastReg pools; it can no longer be
  off).
- No longer try to use the global DMA memory region, but make use
  of the FMR/FastReg pool for all RDMA Tx operations.
- Everywhere we are using the device DMA mr to derive the
  L-key for non-registered memory regions, use the
  pd->local_dma_lkey value instead.
- Make the default map_on_demand = 256.  This will allow nodes with
  this patch to still connected to older nodes without this patch
  and FMR/FastReg turned off.  When FMR/FastReg is turned off, we
  use 256 as the max frags so the two sides will still be able to
  communicate and work.

Signed-off-by: Doug Oucharek <doug.s.oucharek@intel.com>
Change-Id: Iab967d1fdff760c2b06aed395152772fe71bc26d
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/25277
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8366 test: Remove PIOS tests

PIOS is an old benchmark and is not open source. PIOS should
be removed from sanity-benchmark.

Test-Parameters: trivial testlist=sanity-benchmark
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Id89ab1107e4d7e444f6149d04bd90b4ea8c7828b
Reviewed-on: https://review.whamcloud.com/26014
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Cliff White <cliff.white@intel.com>

LU-9169 kernel: kernel update [SLES12 SP2 4.4.49-92.11]

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
  mdsdistro=sles12sp2 ossdistro=sles12sp2 mdsfilesystemtype=ldiskfs \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I210b548c1f27a9e444626a29dfd61953850d60ff
Reviewed-on: https://review.whamcloud.com/25688
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 llite: get rid of lustre_dump_dentry()

it's not only badly racy, it's actually dead code - the call in
ll_invalidate_aliases() is unreachable.  For an alias of our inode
to be root dentry, the inode would have to be its ->d_inode, aka
the root inode, and we never call ll_invalidate_aliases() for that.

Linux-commit: 5e9e2040247ac19a46d6627736ece726c46f1fdf

Change-Id: Ic5e475102793424396bb8b173eb8d9a3b020bd82
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25639
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9153 utils: improve llog name parsing

Current llog name specification is totally non-standard for
lctl llog_print, llog_info, and related commands.  Change this
to accept llog names without the leading '$', since adding '$'
is non-intuitive and the shell will think this is a variable
and "$fsname-client" becomes "-client" unless it is escaped.

Similarly, llog identifiers were formatted like "#0x4:10#0000000"
instead of the standard FID format [0x4:0x10:0x0].  Update the
code to accept and print FIDs in normal format.

Add lctl-llog_print.8, lctl-llog_info.8, lctl-llog_catlist.8 man
pages.

Add a test case for parsing "lctl llog_print" with both old
and new input formats.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Idc00308162eb40b121b49bae3a6bc1d9326927fb
Reviewed-on: https://review.whamcloud.com/25468
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9117 tests: SKIP sanity/129 on older MDS versions

LU-3531 esp the change,
"LU-3531 mdt: delete striped directory" is not present
on 2.5.x esp on tags less 2.5.56.

Thus sanity/129 test needs to be skipped in interop
with such MDS versions.

Test-Parameters: testlist=sanity
Signed-off-by: Parinay Kondekar <Parinay.Kondekar@seagate.com>
Change-Id: I3f1b92ed5ef5b8011bee8b2ef77c7c5b0db197b2
Reviewed-on: https://review.whamcloud.com/25440
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1095 mdt: quiet mdt_mfd_set_mode() debugging

Quiet overly verbose debugging in mdt_mfd_set_mode().

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ibdd77b70bfdd430fb9e3f976564780f5d1877613
Reviewed-on: https://review.whamcloud.com/25399
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7092 tests: generate sanity proc_dirs on facet

For sanity test_133f and test_133g generate the list of available
/proc and /sys directories for lnet and lustre on the remote facet
instead of only the client, so that the tests work during interop.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I0125e20656a712eb620ba4f07d1eaeeb0b9cab07
Reviewed-on: https://review.whamcloud.com/25363
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9014 test: handle separate MGS in conf-sanity test_72()

In conf-sanity test_72() when using a separate MGS, add --replace to
the mkfs options to prevent failures from already registered targets.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I06a266a8ca954c4b7c62fee3957145854d48dd38
Reviewed-on: https://review.whamcloud.com/24852
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-9010 obdclass: use static initializer macros where possible

In lustre/ldlm/ replace module load time initialization of several
atomics, lists, locks, mutexes, wait queues, etc with static
initialization using the kernel provided macros.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I5514992762b95fb23a154970abebd34fb65d9be2
Reviewed-on: https://review.whamcloud.com/24827
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9066 ldlm: don't evict client on umount if AST fails

When server is being stopped the AST send is denied and
that AST failure will lead to the unexpected client
eviction.
Patch adds extra checks in ldlm_handle_ast_error() to
distinguish that type of error and proceed without
eviction.

Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: I57c7f417376ac64c8df4c6d5f68bea5083c0d22f
Reviewed-on: https://review.whamcloud.com/23921
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8735 llite: Return -ERESTARTSYS in range_lock()

If we return -ERESTARTSYS rather than -EINTR then the syscall can be
retried rather than failing with -EINTR.

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I20d1a552448e567f0400a14a71a00d0d25a9164d
Reviewed-on: https://review.whamcloud.com/23259
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7790 test: skip test_130a and test_130b if server is less than 2.7.2

Skip test_130a and test_130b if server version is less than 2.7.2

Test-Parameters: trivial testlist=recovery-small

Change-Id: I4377792111e973cad793a56221a7bc8fe8ddfe68
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-on: https://review.whamcloud.com/19682
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5964 tests: open a large number of files at once

Modify createmany to allow keeping files open after creation to test
the total number of files that can be open at one time.  Also add a
sanity.sh test for this functionality.

Fix handling of last "-seconds" argument, but deprecate positional
parameters in favor of proper named argument parsing, since that is
more flexible and sustainable in the long term.

Order the include files alphabetically to simplify coding.
Order the argument parsing to be alphabetical to simplify coding.

Use booleans for do_* flags, and don't treat multiple of the same
argument as an error, that doesn't really improve usability at all.

Rename the "-r" option to "-u", to better match "unlinkmany", but
allow both to be accepted for the time being.  Allow "-u" (or "-r")
to be used in isolation to only delete files created in a prior run.
Print stats properly if only "-u" is specified, instead of assuming
that files are always being created in the same pass as deletion.

Print only the short program name instead of all of argv[0].  This
will also simplify multiplexing "createmany" as "unlinkmany" by name
at a later time.

Test-Parameters: trivial testlist=sanityn
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I0e8c1af65dcf0b25065f26731c694f4beaab75d7
Reviewed-on: https://review.whamcloud.com/12885
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9019 socklnd: use 64-bit incarnation time stamp

ksnn_incarnation is a 64-bit value, but using timeval to compute
it will cause an overflow in 2038. This changes it to use
ktime_get_real_ns() instead.

Change-Id: I9a0b1f2c35d04d8dc2c8fd5e17e19583c8af5ff0
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25690
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Olaf Weber <olaf@sgi.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9019 msg : migrate IR stats to 64 bit time

Replace the struct timeval usage due to the overflow
issues in 2038 on 32-bit systems. Use of struct
timeval upstream is deprecated as well. Replace its
use with timespec64 and use ktime to store the
fsdb_notify_* fields.

Change-Id: I7771710723d6717cbff93e64abdf794957c42be5
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/24919
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8900 snapshot: rename filesysetem fsname

Introuduce new tunefs options "--rename" for renaming the existing
Lustre filesystem fsname with configuration/parameters preserved.
Usage: tunefs.lustre --fsname $NEW_FSNAME --rename $OLD_FSNAME ...

NEW_FSNAME is the new fsname, must be specified.
OLD_FSNAME is the old fsname, it is optional if the MGS and the MDT
are combined together.

It shares the functionality of fork/erase Lustre configuration logs
in kernel.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ic9aaf6ed6209c7580bfabd4b660a6a7667576484
Reviewed-on: https://review.whamcloud.com/24268
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8900 snapshot: simulate readonly device

Introduce new server-side mount option: rdonly_dev. Under the
device readonly mode, the Lustre kernel threads that may cause
system modification, such as lfsck, scrub, osp_precreate_thread,
osp_sync_thread, and so on, will be disabled when mount. And the
system modification for recovery and client connect/disconnect,
will be ignored also.

On the other hand, if the server is mounted as "rdonly_dev", then
the client must specify "-o ro" option when mount; otherwise, it
will get -EACCES.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iafc02bf53c41b9d7f08120ba7639b91ae527bbbc
Reviewed-on: https://review.whamcloud.com/24267
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8900 snapshot: fork/erase configuration

Introuduce new lctl command to fork/erase the configuration for
the specified system. It is used for mount the snapshot together
with the snapshot's original system MGS. Usage:

lctl fork_lcfg <fsname> <newname>
lctl erase_lcfg <fsname>

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I55d420b54215b09d42b2496b5c1271211c1fb5d7
Reviewed-on: https://review.whamcloud.com/24266
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8900 snapshot: user interface for write barrier on MDT

The user can control the write barrier on MDTs via lctl commands:

Freeze barrier:
lctl barrier_freeze <fsname> [timeout (in second)]
NOTE: the default timeout value is 30 (seconds).

Thaw barrier:
lctl barrier_thaw <fsname>

Query barrier:
lctl barrier_stat <fsname>

Rescan barrier bitmap:
lctl barrier_rescan <fsname> [timeout (in second)]
NOTE: the default timeout value is 30 (seconds).

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Id953203fc3ce6ebbce9f1ae0511fbe2b3813bb9f
Reviewed-on: https://review.whamcloud.com/24265
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8900 snapshot: check write barrier before modification

For client sponsored modifications, the RPC service thread on the
MDT will check whether there is write barrier or not when creates
transaction handler, if yes, return -EINPROGRESS to the caller.

Generally, the check is done inside mdd_trans_create(). For those
cases that bypass mdd_trans_create(), such as some mdd_iocontrol,
quota, will be checked individually.

For open-unlink case, when close the last open handler, it will
try to destroy the orphan. But if the close-destroy is blocked
by the barrier, the orphan MDT-object will be kept in the MDT's
orphan list and will be destroyed automatically when remount the
MDT next time. Since barrier-snapshot is relative rare operation,
and race with close-destroy will be more rare, such solution is
reasonable and will not cause serious trouble.

When client get the -EINPROGRESS error, the expected behaviour is
to re-try the RPC some time later. That is the standard action on
the client-side, not only for blocked by barrier but also for some
cases, such as the case of related OI mapping is invaid and the OI
scrub is rebuilding the OI files. So there is no interoperability
trouble caused by MDT-side write barrier.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I8c3571f7f89fc9ff7397457955ffc75543eb2164
Reviewed-on: https://review.whamcloud.com/24264
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9003 mdc: support posix_acl_valid with user namespace

Starting with Linux 4.8, posix_acl_valid takes an additional struct
user_namespace argument. Use init_user_ns since no other namespace is
available at this point (based on Linux commit
0d4d717f25834134bb6f43284f84c8ccee5bbf2a).

Change-Id: I656cb32f632c7ee00bdc36b6f6d4b442fcb03fcc
Signed-off-by: Michael Kuhn <michael.kuhn@informatik.uni-hamburg.de>
Reviewed-on: https://review.whamcloud.com/25307
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9201 libcfs: reduce libcfs checksum speed test time

Loading the libcfs module is getting increasingly slow due to
multiple checksum types being speed tested at startup (8 different
checksums * 1s per checksum).

Reduce the number of checksum algorithms checked at module load
time to the ones that are actually need the speed (i.e. the bulk
data checksums), and reduce the amount of time taken to compute the
checksum.  The other checksum types typically do not need the speed,
but rather are selected by the configuration.

Precompute the checksum speeds and supported types for the OST so
they are not recomputed for each new client that connects.

This reduces the module load time from 8.0s to 0.76s in my testing.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I4b4ea109633585f61201a661c54ce4229c3ebbe5
Reviewed-on: https://review.whamcloud.com/25923
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9182 tests: SKIP sanity/205 for older version of MDS

LU-5946 is not present on 2.5.x esp changes from
"LU-5946 lprocfs: free expired jobstats after /proc read"

The changes in sanity/205 rmdir w.r.t DNE are to be SKIPPED
where we are seeing the failure.

Signed-off-by: Parinay kondekar <Parinay.Kondekar@seagate.com>
Change-Id: I02d24eddaac3fc65e801eb86f2e878e117d6cdb1
Reviewed-on: https://review.whamcloud.com/25813
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7537 tests: clean up sanity test_133c

Fix up code style for sanity.sh test_133c.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Id8d20e14cbf825f81eff40afd44a1d8998738819
Reviewed-on: https://review.whamcloud.com/25583
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8952 tests: handling test specific cleanup of ost pools

Using destroy_test_pools instead of cleanup_test as trap
function which will call destroy_pools to make sure pools
get removed from the list. Some testcase pool cleanup trap
is missing so incase of test failure pool will not get
destroyed, used trap to remove pool inside create_pool
instead of using in each testcase.

Seagate-bug-id: MRP-3767
Signed-off-by: Jadhav Vikram <jadhav.vikram@seagate.com>
Reviewed-by: Ujjwal Lanjewar <ujjwal.lanjewar@seagate.com>
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@seagate.com>
Tested-by: Vitaly Fertman <vitaly.fertman@seagate.com>
Change-Id: I2bab8514962f3ec5312decd2ded29893e16c5cef
Reviewed-on: https://review.whamcloud.com/24416
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-8918 obdclass: hoist locking in lu_context_exit()

Hoist lu_keys_guard locking out of the for loop in lu_context_exit().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I2f79a1321a48d8a459b4f83a7996ca2ae9a8e4bb
Reviewed-on: https://review.whamcloud.com/24217
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8820 hsm: skip LL_IOC_HSM_COPY_START for HSM removes

In llapi_hsm_action_begin(), if the action is a remove then skip
calling the LL_IOC_HSM_COPY_START ioctl.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Id84f1877a4f05c91e063797b51bd2046f2283ec4
Reviewed-on: https://review.whamcloud.com/23700
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9176 osd-zfs: improve statfs estimate for ZFS

When a ZFS filesystem is newly formatted, there are 128 OI ZAPs and
96 object directories, but only a handful of regular files.  This
skews the average MDT file size to be much larger than regular files,
which in turn makes the estimated total number of files on the MDT
much too small.

Improve the inode size estimate when the filesystem has few files to
be closer to the expected long term average so that the reported total
inode count is reasonable for a new or otherwise empty filesystem.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ic40af2057d8b21e0ee4d9bb3b13b7f6786d0d406
Reviewed-on: https://review.whamcloud.com/25743
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Ned Bass <bass6@llnl.gov>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8624 osc: hung in osc_destroy()

cl_destroy_in_flight becomes < 0 because the
osc_can_send_destroy() won't increment
cl_destroy_in_flight if l_wait_event() gets
interrupted by a signal, but the request will
still be sent and the request's interpret
function will decrease the counter.

Don't send OST_DESTROY request on signal
and return -EINTR.

Seagate-bug-id: MRP-3834
Change-Id: Ie725e501909095b8cd853287b9a43feb1f89ddb5
Signed-off-by: Andriy Skulysh <andriy.skulysh@seagate.com>
Reviewed-on: https://review.whamcloud.com/22588
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8813 gss: limit the number of error messages in logs

Acrually LGSS_MECH_SK processing if SSK is not enabled, rather than
just printing an error message.  Distinguish between SSK being
disabled or not included in the service at all.

Rate limit the number of error messages submitted to the logs.

Test-Parameters: trivial testlist=sanity-sec
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I0190c46b94ecf668ef629c5b864c43658f3560f3
Reviewed-on: https://review.whamcloud.com/25584
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Chris Hanna <hannac@iu.edu>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6707 test: load loop module to have loop devices

Since there is no default loop device on EL7, load
loop module with option max_loop=8 to have loop devices
ready. posix.sh is not going to pass until LU-9102 get
resolved, but first fix the loop device issue.

Change-Id: I9fb9eba15099e5569dbc04fcd4454656a59fdd52
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-on: https://review.whamcloud.com/15130
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Jenkins
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9174 kernel: kernel update RHEL7.3 [3.10.0-514.10.2.el7]

update RHEL 7.3 kernel to 3.10.0-514.10.2.el7

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I06e11f6a8e53c2c1bcf86f37edcabb1f9f178c66
Reviewed-on: https://review.whamcloud.com/25747
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9138 kernel: kernel update [SLES12 SP1 3.12.69-60.64.32]

Update target and kernel_config files for new version
Add an extra ldiskfs patch to adapt to changes from recent landing
of LU-9031.  These changes were required due to new features
in upstream ext4 code seen in both linux 4.4 and linux 3.12.

Test-Parameters: clientdistro=sles12 testgroup=review-ldiskfs \
  mdsdistro=sles12 ossdistro=sles12 mdsfilesystemtype=ldiskfs \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I0c7f01dfaa087e0799a300531aff45841ba87c68
Reviewed-on: https://review.whamcloud.com/25556
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9167 hsm: use a new environment during HSM purge

In hsm_cancel_all_actions() use a newly allocated environment instead
of borrowing it from the coordinator.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Idd5f05d1d5629f4e4e627e137e3addc1a51f8a96
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/25730
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9161 utils: don't load bitmaps when checking features

Change is_feature_enabled() to avoid loading the bitmaps from disk
when checking the filesystem features.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I7017e3ffce4ce745b1746229a5c219886c479bf9
Reviewed-on: https://review.whamcloud.com/25671
Tested-by: Jenkins
Reviewed-by: Artem Blagodarenko <artem.blagodarenko@seagate.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 ptlrpc: use 64-bit time for debugfs import state output

This time is only printed in debugfs, and can be easily converted
to 64-bit to avoid overflowing on 32-bit systems in 2038.

Linux-commit: 74e489aa9cce1ec8f8a44ecc1651cf06b61db204

Change-Id: I7ee171367e90ef32b29bce31abddd4e17eb8daf2
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25409
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9107 build: Update SPL/ZFS to 0.6.5.9

Update SPL and ZFS version that is built against

https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.6.5.9

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ife6f63455fea57251c0ec32c68e15ea40b8bda97
Reviewed-on: https://review.whamcloud.com/25390
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8306 ldlm: send blocking ASTs after lock replay

If blocking AST wasn't received by client before recovery,
we need to scan the whole waiting lock list to send blocking
ASTs after lock replay done, otherwise, client could be
evicted unpurposely like following:

- cl1 has a granted lock;
- cl2 has a waiting lock, BL AST is sent but lost on a way;
- failover, locks are replayed and applied on the server in
  the correct order;
- waiting lock is just put to the resource, no new BL AST
  is re-sent, no timeout can happen for the granted lock on
  server, no timeout for the waiting lock on client;
- cl2 will be hanging for a long time until cl1 will cancel
  its aged lock; may lead to cl2 eviction.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I2a3fecf3b7fa79f96874d5ae21c599725334d9a5
Reviewed-on: https://review.whamcloud.com/24716
Reviewed-by: Vitaly Fertman <vitaly.fertman@seagate.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-8880 gss: fix GSS support for DNE

With DNE, an OSP can be used on MDT for connections to other MDTs.
So to determine exactly every connection's purpose, use client_obd's
cl_sp_to in import_to_gss_svc() and gss_svc_upcall_install_rvs_ctx().

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I84934010102bdf760d07796866272b359bf0465e
Reviewed-on: https://review.whamcloud.com/24236
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>

LU-8857 config: refactor sptlrpc config process

MDT/OFD may access config llog directly, e.g. accessing sptlrpc
local copy, but this config context is not initialized, these code
existed before, but maybe got lost in server code refactor before.
But after reviewing code this is not necessary, target doesn't
need to access sptlrpc config directly, but leave it the same as
other configs, that is, let MGC to handle.

Besides, ofd_set_info_hdl() doesn't need to handle KEY_SPTLRPC_CONF
because this key is handled in o_set_info_async only.

As a side effect, remove obsolete function lustre_rename().

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: If0937cf6e5e03bc84f883131bbf63326e847e701
Reviewed-on: https://review.whamcloud.com/24119
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-5361 llite: Remove OBD_FAIL_OSC_CONNECT_CKSUM

Remove OBD_FAIL_OSC_CONNECT_CKSUM as all clients and servers
since 1.8 support OBD_CONNECT_CKSUM. No reason to check
interoperability with older servers anymore. Remove sanity
test_77i(), update test_77j().
Fix _wait_osc_import_state() to fetch OST state when facet
is client.

Seagate-bug-id: MRP-1421
Signed-off-by: Abrarahmed Momin <abrar.habib@seagate.com>
Change-Id: I664f50bc8dbd3a169bcb7e42706717416d6e1006
Reviewed-on: https://review.whamcloud.com/23644
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8662 osd-ldiskfs: check OI mapping update

Originally, we assumed that the OI mapping will not be modified
during the osd_fid_lookup(). But it is not always true, because
OI scrub may update the OI mapping for repairing inconsistency.
This patch checks the OI mapping update when osd_fid_lookup(),
once happened, then trust the latest OI mapping. It also removes
some incorrect LASSERT() checks.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ic2617400d56003ec67982ae5135cedc884f09e3a
Reviewed-on: https://review.whamcloud.com/23220
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9019 zfs: use 64-bit timestamps for brw stats

The debugfs brw stats interface contains timestamps that are
computed from timeval, which overflows in 2038 on 32-bit systems.

This changes the output to use a timespec64 type to avoid the
overflow. I also change the format to print the sub-second portion
as 9 digits (nanoseconds) for clarity, rather than printing six
digits without leading zeroes.

Change-Id: I34f3b02a0d5f80e9ae22a47f55445199332b3dc9
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25411
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4825 ofd: fix OBD_FAIL_OST_ENOINO/ENOSPC behaviour

Fix the OBD_FAIL_OST_ENOINO and OBD_FAIL_OST_ENOSPC behaviour so that
they also reduce the total number of inodes or blocks when setting the
number of free inodes/blocks to zero.  This allows "lfs df [-i]" to
still report the correct number of blocks and inodes used on the OST.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ia746273e19aed22a0926708899de009b8d5cab07
Reviewed-on: https://review.whamcloud.com/20163
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8900 snapshot: operate write barrier on MDT

Currently, the Lustre barrier is implemented as write barrier
on all MDTs. For each MDT in the system, when it starts, it
registers a barrier instance that will be used in handling
subsequent barrier requests.

The barrier_handler() processes the barrier request: freeze
or thaw the barrier on the MDT. Freezing barrier is the key
part. We use two-phases barrier to guarantee that after the
barrier setup:

1) All the MDT side pending async modifications have been flushed.
2) Any subsequent modification will be blocked.
3) All async transactions on the MDTs have been committed.

For phase1, we do the following:

Firstly, it sets barrier flag on the instance that will block
subsequent modifications from clients. (Note: server sponsored
modification will be allowed for flush pending modifications)

Secondly, it will flush all pending modification via dt_sync(),
such as async OST-object destroy, async OST-object owner changes,
and so on.

If there are some on-handling clients sponsored modifications
during the barrier freezing, then related modifications may cause
pending requests after the first dt_sync(), so call dt_sync()
again after all on-handling modifications done.

With the phase1 barrier set, all pending cross-servers modification
have been flushed to remote servers, and any new modification will
be blocked. But it does not guarantees that all the updates have been
committed to storage on remote servers. So when all the instances
have done phase1 barrier successfully, the MGS will notify all
instances to do the phase2 barrier as following:

Every barrier instance will call dt_sync() to make all async
transactions to be committed locally.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I8d209e98d175eacdadd25c385ffc1c3e4451527a
Reviewed-on: https://review.whamcloud.com/24263
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

New tag 2.9.54

Change-Id: I6283eb81dcd85d5cd6da66058e0ad1000b2322e6
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9024 tests: improve conf-sanity test_63 output

Improve the output of conf-sanity.sh test_63 to fix SLES11
test failures caused by different /proc/slabinfo results.
Replace "ldisk" with "ldiskfs".

Load ldiskfs module on the MDS if test is run with ONLY=63.
Compute number of slab entries per page even if the slab
is using multiple pages to improve object efficiency.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I7424c797a5955c06d2107c2047d3e2b64749fa56
Reviewed-on: https://review.whamcloud.com/25880
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>

LU-9146 ldiskfs: backport a few patches to resolve deadlock

The ext4_mark_inode_dirty() may trigger new block
alloction. ext4_xattr_block_set() will be invoked
and trying to grab i_data_sem. It may cause deadlock
in some time. We backport 3 patches to resolve this
issue.

Backport from upstream:
2e81a4eeedcaa66e35f58b81e0755b87057ce392
e3cf5d5d9a86df1c5e413bdd3725c25a16ff854c
a521100231f816f8cdd9c8e77da14ff1e42c2b17

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I2e1bc10bc0e03ad760d113a785d43c06f85fcfa3
Reviewed-on: https://review.whamcloud.com/25595
Tested-by: Jenkins
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8934 spec: Use correct provides in lustre-dkms

Provide kmod-lustre instead of lustre-kmod so that the lustre rpm
requires matches up correctly.

This also adds some support for suse which names thier kmod packages
differently.

Test-Parameters: trivial
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ifd3484ade6a90d4492852c7b61db629b934d5a03
Reviewed-on: https://review.whamcloud.com/25417
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 llite: use 64-bit llite debugfs timestamps

The llite debugfs interface contains timestamps that are computed
from timeval, which overflows in 2038 on 32-bit systems.

This changes the output to use a timespec64 type to avoid the
overflow. I also change the format to print the sub-second portion
as 9 digits (nanoseconds) for clarity, rather than printing six
digits without leading zeroes.

Linux-commit: 05406826aafb658d51ba7ac2fc6870992776ef6d

Change-Id: I8ea71d1f83323a82973c567a7c3442354cfdaf8f
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25403
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8582 tests: skip sanity test_255a for interop

Move the sanity.sh test_255a version check to the start of the test
instead of after some of the ladvise functionality is tested.

That said, there is still an issue here, because the client shouldn't
hang indefinitely if the server is returning -EOPNOTSUPP, in the case
where a new client/lfs is connecting to an old OSS, and that issue is
not addressed by this patch.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: If1fbc13bd60cfc1942a43aff40e3e7905cdcab07
Reviewed-on: https://review.whamcloud.com/25362
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9098 lnet: Move lnet_routes.conf to /etc

lustre_routes_config now skips lines which are blank or
have a first non-whitespace character of #. This enables
comments to be placed in /etc/lnet_routes.conf. Also,
provide a sample lnet_routes.conf.

The SysV lnet script now expects lnet_routes to be
in /etc. Updated docs to reflect the new location.

Test-Parameters: trivial
Signed-off-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Change-Id: Ie2fdc5c870aeb8165fd9896bc91120731db45c8b
Reviewed-on: https://review.whamcloud.com/25358
Tested-by: Jenkins
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8139 ofd: allow brw_size with 'M' suffix

Allow specifying the 'M' suffix for the obdfilter.*.brw_size tunable
to maintain compatibility with original version of this tunable.

Fix sanity.sh test_101g() to only specify the 'M' suffix if this is
allowed by the server version, otherwise specify only the number.
Check both read and write RPC counts in the test, and also 8MB RPCs.

Clean up test_101g() to remove duplicate code, and only change the
brw_size parameter and remount the clients if the current brw_size
is not large enough.  Calculate the number of pages correctly for
systems with different PAGE_SIZE.

Test-Parameters: trivial envdefinitions=ONLY=101 testlist=sanity,sanity
Test-Parameters: envdefinitions=ONLY=101 testlist=sanity,sanity,sanity
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: If7b268e3ec1160b41a676590a07511a2953ebbe5
Reviewed-on: https://review.whamcloud.com/25335
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8827 mdt: bypass quota enforcement for HSM release

In mdt_hsm_release() create the orphan object with root ownership to
bypass potential quota restrictions on the user and group of the file
to be released.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I2f238bf39339e67e07a48743ccd2072a40ea7f0a
Reviewed-on: https://review.whamcloud.com/23720
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8788 tests: modify create_pool to use as wrapper

Changed the required places which directly call lctl
pool_new command to use create_pool() wrapper.

Seagate-bug-id: MRP-2683
Signed-off-by: Jadhav Vikram <jadhav.vikram@seagate.com>
Change-Id: I83a867072ae289f820036338249182a3112e7fba
Reviewed-on: http://es-gerrit.xyus.xyratex.com:8080/11162
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@seagate.com>
Reviewed-by: Rahul Deshmukh <rahul.deshmukh@seagate.com>
Tested-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Reviewed-on: https://review.whamcloud.com/23541
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-8734 gnilnd: Handle dla credits exhaustion

Add delay_list so that when dla_credits are exhausted
we retry sends regardless of whether there is a CQ event
targetted at the connection that tried to send.

Remove retry limit and rely on connection timeouts to
tell us when to close a connection.

Change max_retransmits such that it is the number of
attempts before the connection enters the delay queue.

Retransmits are now expected to occur, so remove NETERROR
log and always use D_NET when logging a retransmit.

Move state changes and retransmit counter changes out of
the logging function.

Remove log_retrans_level and replace usages with D_NET
directly.

Test-parameters: trivial

Signed-off-by: James Shimek <jshimek@cray.com>
Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: Id0aeb1add8e761fc7351c28a04c20ca40c51b1e1
Reviewed-on: https://review.whamcloud.com/23258
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Chuck Fossen <chuckf@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8726 osd-ldiskfs: bypass read for benchmarking

The read operation is bypassed, so that Lustre client could get more
than 3GB/s on a single machine. This could be used when benchmarking
client site performance, such as readahead.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I518b287bf3c9b26347b43a4b82ef3114f966eb87
Reviewed-on: https://review.whamcloud.com/21648
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8403 obd: remove OBD_NOTIFY_CONFIG

None of the OBD notify handlers listen for the OBD_NOTIFY_CONFIG event
so remove it and its sole use in server_start_targets().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: If0d756755b38214574c71265b91bab6acfaf9311
Reviewed-on: https://review.whamcloud.com/21422
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-8900 snapshot: new config for MDT write barrier

We need some mechanism to set write barrier on the MDT, with that,
all the subsequent modification will be blocked, then the barrier
sponsor can operate on the frozen system, such as make snapshot.

It is implemented via new MGS/MGC plain LDLM lock on the virtual
barrier config-log. The MGS sends the barrier request to the MDTs
via glimpse callback against such barrier plain ldlm lock. The MGC
returns is local barrier status via the glimpse reply.

This patch also enhances mgs lproc interface logic.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I349249fc21e3a6394e658d7529cf87299c050f1a
Reviewed-on: https://review.whamcloud.com/24262
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7994 statahead: add smp_mb() to serialize ops

In ll_deauthorize_statahead(), it set thread stop flag, and then
wake up thread, however wakeup is called inside spinlock in case
ll_statahead_info is released, then we need to call smp_mb() to
serialize setting and wakeup.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I500bdf325aa0bd122b17bea448c3b9eec809a2a7
Reviewed-on: https://review.whamcloud.com/23040
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9118 o2iblnd: handle MOFED libcfs time api collision

Both libcfs and the MOFED 4 stack define
ktime_get_real_ns() for platforms that lack it.
The solution is to reverse the logic of testing
for ktime_get_real_ns() done by lustre. This way
we avoid the HAVE_KTIME_GET_REAL_NS collision.
Also to ensure older platforms with an older OFED
stack still will build only turn off NEED_KTIME_GET_REAL_NS
set by libcfs in o2iblnd.h when the OFED stack
has defined LINUX_3_17_COMPAT_H. The compat-3.17.h
OFED header is where ktime_get_real_ns() gets defined
when its lacking on the native platform.

Test-Parameters: trivial

Change-Id: I44966f22cfbb6138fa7bc3fa47148a6f0a94ebd4
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25564
Tested-by: Jenkins
Reviewed-by: Olaf Weber <olaf@sgi.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8642 build: suppport building various OFED

* Remove the 01-remove-mlx4-erroneous-modprobe-config-file:rhel6.ed
* differentiate each type of OFED to allow different way
of downloading, unpack and build with different options
* symlink SLES linux-obj after unpack the rpm

Test-Parameters: trivial

Change-Id: I7fcd50a6b747dbb5419bb029087967f809ef2485
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/22758
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9109 ldlm: restore missing newlines in ldlm sysfs files

Restore the missing trailing newlines in
/sys/fs/lustre/ldlm/namespaces/*/lru_{max_age,size}.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ib9acf8ea6126a16f89da86cfaeadf7685d5c802c
Reviewed-on: https://review.whamcloud.com/25522
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8686 osd: add few more credits if debugging is enabled

this can make JBD happy, prevent panic and let OSD detect
credits overuse.

Change-Id: I93fae9bd0d8208af888b75232eb9b9cde205a98f
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/23044
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-9132 utils: tuning max_sectors_kb on mount

Sometimes user doesn't want the max_sectors_kb being
overwritten on MDT/OST mount, we should provide a way for
user to have more control over this important parameter,
a new mount option 'max_sectors_kb' is introduced:

- When max_sectors_kb isn't specified on mount, change the
  max_sectors_kb to max_hw_sectors_kb, it's default behavior
  suited for most users;
- When max_sectors_kb is specified as zero, leave the old
  setting of max_sectors_kb untouched;
- When max_sectors_kb is specified as a positive number,
  change the max_sectors_kb to this number arbitrarily;

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I251d75bbdf16d8cb6d503bbbfc69fb18993f7a3e
Reviewed-on: https://review.whamcloud.com/25483
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-9127 target: tgt_cb_last_committed is too noisy

tgt_cb_last_committed() prints a D_HA message even if
last_committed was not updated. We can only print a
message when last_committed was updated, so we have
mostly the same information and save some debug log
space and cpu resource.

Change-Id: Ic2784e6a3652ca1851cde1313d6985ed2f90e36b
Signed-off-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Reviewed-on: https://review.whamcloud.com/25469
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alexander Boyko <alexander.boyko@seagate.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@seagate.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-9116 libcfs: avoid overflow of crypto bandwidth caculation

bcount and buf_len are both int, and no force convert in the caculation code:
tmp = ((bcount * buf_len / jiffies_to_msecs(end - start)) *
1000) / (1024 * 1024);
That may cause overflow in modern fast machine.

Change-Id: I1e5abccad3e4df62907317a09de02beb6d831e13
Signed-off-by: Gu Zheng <gzheng@ddn.com>
Reviewed-on: https://review.whamcloud.com/25436
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Li Xi <lixi@ddn.com>

LU-4423 llite: use 64-bit times in another debug print

The ll_setattr_raw() function prints the new inode timestamps
along with the current time using '%lu', which overflows in
2106. This changes the printing of the current time for
now, the other two will change when we migrate the VFS code
to use 64-bit timestamps.

Linux-commit: 8d7eed54a2391db16f184b18cde5c1824775ebdc

Change-Id: I96b5b1599b8af8446ee68f88fc739843291b304c
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25404
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9103 tests: SKIP recovery-small/110g for old MDS versions

LU-2430 is not present on 2.5.x esp the change
"LU-2430 mdd: add lfs mv to migrate inode."
so for MDS version less than 2.6.57 this test needs
to be SKIPPed.

Test-Parameters: testlist=recovery-small
Signed-off-by: Parinay Kondekar <Parinay.Kondekar@seagate.com>
Change-Id: I21d685062733a9d8a633ff208bcb83d3ec146ca0
Reviewed-on: https://review.whamcloud.com/25367
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>

LU-9067 utils: ensure debugfs is mounted

With the move of lustre to sysfs and tracepoint it
will become critical to have debugfs mounted. On
older platforms like RHEL6 its not mounted by default.
Also it is possible that debugfs could become umounted
by accident thus disabling needed functionality
to control lustre. Add to libcfs.a a function that
is always called to ensure debugfs is mounted.
If debugfs is not mounted then mount it if the caller
is root.

Change-Id: I21f85ba252b67bfbc22b23920e2ccaffc196074b
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25182
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-8981 test: sanity 311 check is too strict

System may be too busy to destroy unlinked objs in time, which
cause sanity fail, let's use a smaller value to not fail autotest.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Ibbe543f279c3548176c53b3fdb7b8048ea08931f
Reviewed-on: https://review.whamcloud.com/24814
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8911 tests: sanity-hsm test_24d fails on a local setup

In test_24d, do not use the default mountpoint of copytool_setup()
as it is also the one the test mounts in read-only mode.

This patch also removes:
 - the "continuing fast path" in copytool_setup (calling
   "pkill -CONT -x lhsmtool_posix" instead of launching a new
   copytool) as it is not explicitly used by any test
 - as well as the first copytool_setup located outside of any
   tests.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I86bc41213bc656d7a83d63fb8e9bc595ba6b73ca
Reviewed-on: https://review.whamcloud.com/24185
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9038 obdclass: handle early requests vs CT registering

This patch addresses cases where CDT may start to send requests
before CT has fully registered with all MDTs and thus when the KUC
pipe kernel side has still not been initialized in
lmv_hsm_ct_register().
This will avoid Oops'es due to kkuc_groups[KUC_GRP_HSM] being
uninitialized/zero'ed and we rely on CDT to later retry.
sanity-hsm/test_402b has been added to verify.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Ibccf2627aebe8da52128da5d90d24751394bf61d
Reviewed-on: https://review.whamcloud.com/25050
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6210 mdd: Change positional struct initializers to C99

This patch makes no functional changes.  Struct initializers in the
mdd directory that use C89 or GCC-only syntax are updated to C99
syntax.

The C99 syntax prevents incorrect initialization if values are
accidently placed in the wrong position, allows changes in the struct
definition, and clears any members that are not given an explicit
value.

The following struct initializer has been updated:

lustre/mdd/mdd_dir.c:
static struct lu_name lname_dotdot

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I390a67c9c7182ca1e1aee7aa926e7d2509f73cbf
Reviewed-on: https://review.whamcloud.com/23747
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8403 obd: remove OBD_NOTIFY_SYNC{,_NONBLOCK}

None of the OBD notify handlers listen for OBD_NOTIFY_SYNC{,_NONBLOCK}
events so remove them and related code in lov_notify().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I3bc2bd34b268b28777241555dae8896577150c91
Reviewed-on: https://review.whamcloud.com/21421
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9019 obd: use 64-bit timestamps for rpc stats

The debugfs rpc stats interface contains timestamps that are
computed from timeval, which overflows in 2038 on 32-bit systems.

This changes the output to use a timespec64 type to avoid the
overflow. I also change the format to print the sub-second portion
as 9 digits (nanoseconds) for clarity, rather than printing six
digits without leading zeroes.

Change-Id: I8fca45ef62672f3880a444961cb068d8c436e2c7
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25410
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Grégoire Pichon <gregoire.pichon@bull.net>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9094 o2iblnd: kill timedout txs from ibp_tx_queue

Sometimes connection can't be established for a long time
due to rejections and produces cycle of reconnections.
Peer is not removed in each iteration unlike connection.
Thus until connection becomes established txs live in
peer->ibp_tx_queue. This patch adds tx_deadline checking
for txs from peer tx_queue.

Change-Id: Id2623285c735d1dff40ec755a5c8d20e9c62e60a
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@seagate.com>
Seagate-bug-id: MRP-4056
Reviewed-on: https://review.whamcloud.com/25376
Tested-by: Jenkins
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>