LU-3951 lfsck: LWP connection from OST-x to MDT-y

When client sends object-based RPC to the OST-x, the RPC service
thread on the OST-x needs to verify whether the given parent FID
information in the client RPC matches the parent FID information
stored in the OST-object. If not match, it will query the MDT-y
according to the client given parent FID information. The query
RPC from the OST-x to the MDT-y is sent via LWP connection.

The other use case is that the LFSCK on the OST-x needs to talk
with the LFSCK on the MDT-y, such control/query RPC will be via
above LWP connection from OST-x to MDT-y.

Currently, we only support LWP connection frm OST-x to the MDT-0.
This patch enhance that to enable LWP connection from any OST to
any MDT.

Test-Parameters: allwaysuploadlogs
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ie98be82b3af90456d1838d53b6d77c12956f7bd7
Reviewed-on: http://review.whamcloud.com/7666
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4484 lbuild: add support for fresh versions of MPSS 3.x.x

* Adopt lbuild script for new version of MPSS with x.x.x notation.
* Remove dependency from MPSS package to avoid renaming issue in
  the future. The name of package which was used for dependency
  was renamed in MPSS.
* Use new server with MPSS released packages for download.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ie4407ad00177ad6d22770230a4dc6bde967d91ef
Reviewed-on: http://review.whamcloud.com/8836
Tested-by: Jenkins
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4456 osp: extra check for opd_pre

1. Add extra check for opd_pre in statfs_interpret, in case
opd_pre has been freed before the callback.

2. switch the sync_fini and pre_fini, so opd_pre will be freed
after all of the possible access has been stopped.

3. opd_pre_waitq will be accessed in several update threads,
osp_precreate, osp_statfs_timer_cb, statfs_interrupt, move
it to osp_device to make sure it is accessiable even after
osp_pre is freed.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I5c73cb52e2406ed03570fc3471111c409e6fe08f
Reviewed-on: http://review.whamcloud.com/8890
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4517 tests: get params directly in _wait_osc_import_state

In _wait_osc_import_state(), if the facet is not a client node,
go to get osc.*.ost_server_uuid params directly and quickly,
without waiting $maxtime seconds.

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Idf5989f53d050edcb69690b7f24a6e86df233bef
Reviewed-on: http://review.whamcloud.com/8989
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4515 tests: disable sanity-quota test_34 temporary

To avoid other patches to be failed for LU-4515.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I8a6949d18a1ff4f5d229ed083f4f12a667eb3329
Reviewed-on: http://review.whamcloud.com/8981
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins

LU-4413 osp: move seq allocation out of osp_import_event

Because seq allocation(osp_init_pre_fid) might be stuck
during RPC, move it out of osp_import_event, which is
inside ptlrpcd_rcv. Otherwise, some other import RPCs(like
connect req)might be blocked in ptlrpcd_rcv.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ib4014f8b0088ea3613fa4d53d3e274f5bdfe70c7
Reviewed-on: http://review.whamcloud.com/8997
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3531 mdc: release dir page cache after accessing

Release the dir page cache in llite/lmv, so the page
will be hold until entires was filled by filldir.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I8b24bec74b14ff2b65130c02294821fc16ca1421
Reviewed-on: http://review.whamcloud.com/8935
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4409 tests: disable insanity 10 for DNE

Disable insanity 10 for DNE.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I4b67cf745a18a09335e21e1e6e457134ac47f224
Reviewed-on: http://review.whamcloud.com/8650
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>

LU-1267 lfsck: framework (2) for MDT-OST consistency

The LFSCK can talk with OSP directly, then the LFSCK on the MDT can
control/monitor the specified LFSCK instance on other targets (MDTs
or/and OSTs) without breaking dt_device APIs nor making OSP to know
the LFSCK things, and simplify the handling of remote OST-object or
MDT-object. For that, each OSP will register to the LFSCK will they
are added into the system. The LFSCK maintains such target table in
RAM with the similar logic as the LOD does.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ifa14db68925a0cd2afe0c3566382dbb6176d50b2
Reviewed-on: http://review.whamcloud.com/8302
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1267 lfsck: rebuild LAST_ID

The /O/<seq>/LAST_ID records the last oid of the object allocated
within the sequence. The LAST_ID file can be crashed or missed as
the system running. The LFSCK for layout consistency verification
can detect the LAST_ID lost/crashed cases, and can rebuild it via
scanning the whole device.

This functionality is also part of LU-14 live replacement of OST.

Introduce lfsck_notify callback - the LFSCK events notification
channel from the LFSCK to the registered users (MDD/OFD).

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iee85056e2fda1ecba9424c9f0e822643e9f029a8
Reviewed-on: http://review.whamcloud.com/6997
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2818 mdt: Properly handle ENOMEM

When osd_keys_init fails in mdt_lvbo_fill, properly bail out with
error instead of asserting.

Change-Id: I832742ed49cc7740d8e709bc4b87e5d5aa100d39
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/8947
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Jenkins
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-1538 tests: sanity/101d to check available space

Fix the check which compared MBs (in size variable) with KBs
(reported by df).  Also avoid failure if the read took under
two seconds, since the timing is inaccurate at that resolution.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I43335d699ad2b7e4c5db00c36a7795683f3b04f7
Reviewed-on: http://review.whamcloud.com/4312
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4416 llite: struct kiocb ki_left removed

struct kiocb without ki_left memeber since 3.12.

Signed-off-by: yang sheng <yang.sheng@intel.com>
Change-Id: Iea1fb67ebb03430b5dc8f71ed2652967ff60b84d
Reviewed-on: http://review.whamcloud.com/8801
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 ptlrpc: fix potential NULL pointer dereference

The rest of the code seem to imply that rmf_dumper may indeed be
NULL.  Change the code so that dumping is not even considered if
rmf_dumper callback is not set.

Signed-off-by: Michal Nazarewicz <mina86@mina86.com>
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: Iaea16aaf799976d08ebb51322021cc879db1c6d8
Reviewed-on: http://review.whamcloud.com/8682
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4442 test: fix wrong usage of wait_mds_ost_sync()

Fix the wrong usage of wait_mds_ost_sync() in replay_vbr.sh
test_7_cycle(). The first parameter should be a timeout in seconds
not a facet.

Test-Parameters: testlist=replay-vbr envdefinitions=ONLY=7
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I4e6de62049b473deeaf5c75e1136d76d67a02053
Reviewed-on: http://review.whamcloud.com/8796
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Revert "LU-3319 procfs: move osp proc handling to seq_files"

This seems to be causing issues like LU-45-13 and LU-4510
This reverts commit a97e4898ad9e0b65f457b01bdfa954f7d7cd272d.

Change-Id: I6066a255ded24dbdb76b4804e82a377f1069af5f
Reviewed-on: http://review.whamcloud.com/8931
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3680 ptlrpc: Fix assertion failure of null_alloc_rs()

lustre_get_emerg_rs() set the size of the reply buffer to zero
by mistake, which will cause LBUG in null_alloc_rs() when memory
pressure is high. This patch fix this problem and adds a size
check to avoid the problem of insufficient buffer size.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I9fbd4f14e8e1263de2af564c4f2e420f5f2b43bc
Reviewed-on: http://review.whamcloud.com/8200
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4416 mem: truncate_pagecache oldsize removed

truncate_pagecache doesn't need oldsize parameter anymore.
In fact, the oldsize useless in all kernel we supported. so
just remove all things relate to it.

Signed-off-by: yang sheng <yang.sheng@intel.com>
Change-Id: Iba9ccebb73e7e8df3179ef1d68507b7403b117a7
Reviewed-on: http://review.whamcloud.com/8800
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4353 strncmp: Replace incorrect strncmp()s with strcmp()

This patch fixes a few places where strncmp() is used but strcmp()
was meant. It also corrects calls to strncmp() in functions
mdd_declare_xattr_del() and mgs_setparam().

Change-Id: I6f0fa4230bca10e8e7b310783bb89628a6eb788f
Signed-off-by: Swapnil Pimpale <spimpale@ddn.com>
Reviewed-on: http://review.whamcloud.com/8702
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4154 lfsck: skip old lfsck test in DNE mode

The old e2fsck/lfsck tool will not be allowed to run on a DNE
filesystem. This patch updates generate_db() to pass master MDS
parameters only, so that the old lfsck does not corrupt it or
delete all of the files on other MDTs.
This patch also fixes a typo in run_lfsck_remote().

Test-Parameters: mdscount=2 mdtcount=2 testlist=lfsck mdsfilesystemtype=ldiskfs mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I28caaa5ca4a564aabbd6116325257c905ff22861
Reviewed-on: http://review.whamcloud.com/8206
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3319 procfs: move osp proc handling to seq_files

With 3.10 linux kernel and above proc handling now only
uses struct seq_files. This patch migrates the osp
layer proc entries over to using seq_files.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: If58826e11524a5fffd2e491c1386e3795015bc7e
Reviewed-on: http://review.whamcloud.com/8029
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3319 procfs: update shared server side core proc handling to seq_files

Several of the server side abstact layers such as mdt,mgs
etc share several common proc handling routines. This patch
adds the seq_file version so that the stack can gradually
be ported over to these new methods.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I2dd64046fdd4d2bb6f7550bb49cf1c9ef703c157
Reviewed-on: http://review.whamcloud.com/7933
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2880 ldiskfs: Added mount option to enable dirdata.

Added the code to set mount option for enabling the dirdata in
osd_mount(). This will set the dirdata option for lustre
filesystems only. The dirdata mount option would be checked in
get_dtype() to decide whether to pass dirdata flag as well while
reading the file type.

Signed-off-by: Manisha Salve <msalve@ddn.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I07f430c5cd7ad92b81746085b05b53c1202dd725
Reviewed-on: http://review.whamcloud.com/6495
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-3467 ofd: remove obsoleted OBD methods

Remove unused OBD methods from OFD, keep those needed
for echo client and rename them to indicate that

ofd_get_info() and ofd_set_info_async() are only needed
for local calls with single key.

Clean the usage of fti_fid, don't copy FID to it when
it is possible to point on FID in request buffers directly.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I5622e14bccacb809bca1c10499c23bcaf72e2a68
Reviewed-on: http://review.whamcloud.com/8580
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-4403 mds: extra lock during resend lock lookup.

1. If the request does not require open lock, MDT does not
need to search the lock in exp_hash_lock list, because the MDT
will release the lock anyway, and it can always re-enqueue the
request during resend.

2. Lock the resource when the MDS is trying to lookup the lock for
resend request(in mdt_intent_fixup_resent). Otherwise, if the
lock is being released, fixup_resent will return a released lock,
which will cause LBUG when this lock is being released later.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ice5a75f9a4cc3bcb45f8d54cf809b4a272805804
Reviewed-on: http://review.whamcloud.com/8680
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4349 target: count disconnected exports as stale

If export is disconnected during recovery then stale client
counter need to be increased.

Test-Parameters: envdefinitions=SLOW=yes testlist=conf-sanity,conf-sanity,conf-sanity,conf-sanity,conf-sanity
Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Id4f2a668dbe8d5cd2fa3d392b7df7b4d67a4265a
Reviewed-on: http://review.whamcloud.com/8785
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3618 ptlrpc: rq_commit_cb is called for twice

If a ptlrpc_request is already on imp::imp_replay_list, when it's
replayed and replied, after_reply() will call req::rq_commit_cb
for the request, then call it again in ptlrpc_free_committed.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I796c3351ad896aa3e1d0c2147ca7f775b7c14bfc
Reviewed-on: http://review.whamcloud.com/8815
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3319 procfs: update nodemap proc handling to seq_files

Migrate all nodemap proc handling to using strictly
seq_files.

Change-Id: I3dd338789de1cc266cdb37df7f4083b2cb9b6da5
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/8816
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Joshua Walgenbach <jjw@iu.edu>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3319 procfs: move llite proc handling over to seq_file

For lustre clients a special abstract layer so a lustre
client can be mounted. In order to support 3.10+ kernels
this client code being the llite,vvp,and clio layers proc
proc handling has been ported to using seq_files only.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Id2ac0956dbdf586ab1200e2edb00d489c15c5d50
Reviewed-on: http://review.whamcloud.com/7290
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4193 ldiskfs: increase max journal to 4GB

Increase the maximum journal size to 4GB to handle the increased
metadata operation rate possible since MDT SMP scaling is done.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I025b68f9f8b7c9d1084f704e2ebf8722753ebbe5
Reviewed-on: http://review.whamcloud.com/8111
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4444 tests: Skip conf-sanity/69 on zfs

Because file creates happen slowly on ZFS and the number of files
required to run the test is 100K, this test cannot run in a
reasonable amount of time.

Also bail out of test if createmany fails (possible if MDS or OST is
too small), this prevents the test from just timing out instead.

Test-Parameters: envdefinitions=SLOW=yes testlist=conf-sanity
Test-Parameters: envdefinitions=SLOW=yes testlist=conf-sanity mdsfilesystemtype=zfs mdtfilesystemtype=zfs ostfilesystemtype=zfs
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ic8286c970332e0c53525e4d89e4c5f0e32cf57cb
Reviewed-on: http://review.whamcloud.com/8841
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4384 target: don't set OBD_INCOMPAT_FID for OST

That cause downgrade to fall, though here is no incompatibility
actually.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I71de76496441dd4627cf235166f866404edd6cec
Reviewed-on: http://review.whamcloud.com/8810
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>

LU-1267 lfsck: framework (1) for MDT-OST consistency

1) New LFSCK component - lfsck_layout, its data structure, tracing
   file, lFSCK APIs, internal shared functions, and so on.

2) Extend the existing LFSCK:
2.1) new flag - LF_INCOMPLETE, for the case of some target (MDT/OST)
     not join the LFSCK or crashed during the LFSCK.
2.2) New status - LS_PARTIAL, corresponding to above LF_INCOMPLETE,
     if some target (MDT/OST) does not join the LFSCK processing or
     crashed during the LFSCK, when it finished, its status will be
     set as LS_PARTIAL.

3) Control the LFSCK speed by each component itself during the
   second-phase scanning.

4) Avoid lfsck engines to access freed lfsck_instance.

5) Some code cleanup.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I6dc7bb881dc831f6c760be14aac2a066ad75ffec
Reviewed-on: http://review.whamcloud.com/7146
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4106 scrub: Trigger OI scrub properly

There is the following race case between osd_fid_lookup() and object
unlink/detroy:

Both RPC service thread_1 and RPC service thread_2 try to find the
same obj_A at the same time. At the beginning, the obj_A is not in
cache. The thread_1 is in osd_fid_lookup() and finds the OI mapping
for obj_A. But before the thread_1 finding out related inode_A, the
thread_2 moves faster and finds the inode_A and unlinks the inode_A.
So the thread_1 will fail to find the inode_A. Under such case, the
thread_1 will try to check OI again to make sure whether related OI
mapping is still there or not. If no OI mapping, then it is normal
becuase someone has unlinked the file by race; otherwise, it may be
caused by file-level backup/restore, then thread_1 will trigger OI
scrub to rebuild OI files.

But we ignored a corner case that the thread_1 recheck the OI files
may just between the thread_2 has dropped the inode_A's referene to
zero and will remove related OI mapping from the OI file. Then the
thread_1 is misguided, and will trigger OI scrbu unexpectedly.

More initial OI scrub for the /ROOT/.lustre directory to make sure
the necessary files/directories for mount are ready before used.

This patch also enhances the ls_locate()/dt_locate_at() interface
to allow the caller to pass some hints to low layer, such as flag
LOC_F_NEW for create, to help the low layer to handle efficiently
and properly.

Test-Parameters: mdtcount=4 testlist=sanity-scrub
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ia0fa198fc4fa31056f6a32a6d3e75cf905832cd1
Reviewed-on: http://review.whamcloud.com/8002
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4263 osd-zfs: Avoid converting last ID FIDs to OST IDs

When obdfilter-survey first creates an object on a fresh ZFS OST, the
last ID object for FID_SEQ_ECHO has to be created in the first place.
The last ID FID, [FID_SEQ_ECHO:0:0], can not be converted to an OST ID
because the resulting OST ID would be indistinguishable from an
FID_SEQ_OST_MDT0 OST ID and would confuse ostid_id().  This patch
checks for last ID FIDs before converting them to OST IDs in
osd_get_idx_for_ost_obj().

Change-Id: I96cdf85b4725e4882cecabaf90466c7f77a5e0a6
Intel-bug-id: FF-182
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/8301
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4454 libcfs: warn if all HTs in a core are gone

libcfs cpu partition can't support CPU hotplug, but it is safe
when plug-in new CPU or enabling/disabling hyper-threading.
It has potential risk only if plug-out CPU because it may break CPU
affinity of Lustre threads.

Current libcfs will print warning for all CPU notification, this
patch changed this behavior and only output warning when we lost all
HTs in a CPU core which may have broken affinity of Lustre threads.

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I62267b62871c129beeb1593c4f69e7b81a79999d
Reviewed-on: http://review.whamcloud.com/8770
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-4372 gss: Compatibility cache_register_net 2.6.x/3.3 kernel

Since 3.4 cache_register/cache_unregister are removed.
This patch provides a compatibility function to support
kernels 2.6.x to 3.3. Note, since 2.6.37
cache_register_net/cache_unregister_net are defined,
but not exported (no EXPORT_SYMBOL) and thus cannot
be loaded by the module loader. In 3.3
cache_register_net/cache_unregister_net
are exported. This patch is related to LU-4012.

Signed-off-by: Thomas Stibor <thomas@stibor.net>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Iaee5aa7e60e1bd08735c345f413a2344c2850f57
Reviewed-on: http://review.whamcloud.com/8528
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-3834 mdt: handle swap_layouts failures during restore

Actually nothing occur after swap_layouts failures during restore,
this can lead to file being left in incoherent state and thus be
unavailable because HS_RELEASED is clear but LOV_PATTERN_F_RELEASED
is still set.
This patch will allow original layout to be recovered by the use of
SWAP_LAYOUTS_MDS_HSM flag. Additionaly this requires HSM xattr of
the data FID to be set.
Also adds layout-swap failure injection and related test.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Id0e9a005362e4a3854b33f6ce1888197d20e7dbf
Reviewed-on: http://review.whamcloud.com/7631
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3963 libcfs: convert DT objects atomic primitives

This patch convers all cfs_atomic primitives in
ofd, osc, osd-ldiskfs and osd-zfs to the linux
atomic api.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Signed-off-by: Peng Tao <tao.peng@emc.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I235cd45503115a936cf502e5469daf806cf16078
Reviewed-on: http://review.whamcloud.com/7076
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3963 libcfs: convert md[d/t]/mg[c/s] to linux atomic primitives

This patch convers all cfs_atomic primitives in
mdd, mdt, mgc and mgs.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Signed-off-by: Peng Tao <tao.peng@emc.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I052ca4da80b0cbd36df02a50f2ae0651eec28ea8
Reviewed-on: http://review.whamcloud.com/7074
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-3963 libcfs: convert include/lclient/ldlm/lfsck cfs_atomic

This patch converts all cfs_atomic primitives in
lustre/include, lclient, ldlm and lfsck directories.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Signed-off-by: Peng Tao <tao.peng@emc.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ie746993b917bd6ea8c2403a47488ef0e5a06d6fb
Reviewed-on: http://review.whamcloud.com/7072
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2675 lov: remove unused lov obd functions

Remove the unused liblustre functions llu_extent_lock,
llu_extent_unlock, and llu_glimpse_callback.

Remove the unused lov functions lov_getattr, lov_setattr, lov_brw,
lov_merge_lvb, lov_punch, lov_sync, lov_enqueue, lov_cancel,
lov_cancel_unused, lov_extent_calc, lov_setstripe, lov_setea,
lov_find_pool, and supporting functions.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I59daa224cfedc9e105a3ebe4d3bacecd5a9aa739
Reviewed-on: http://review.whamcloud.com/5581
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-3529 osp: init FID client for OSP on MDT.

1. Initialize FID client for OSP on MDT.
2. Pull out OSP precreate stuff into a separate structure,
so OSP on MDT does not have to allocate precreate.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I3c4a87e2fbea2a733fc6a4a296a50ba67652aa32
Reviewed-on: http://review.whamcloud.com/7158
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3772 ptlrpc: fix nrs cleanup

When service start failed due to short of memory, the cleanup code
could operate on uninitialized structure and cause crash at the end.

This patch fix the nrs_svcpt_cleanup_locked() to perform cleanup only
on the nrs which has been properly initialized.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ieafa5b144133490b662f5a80a7b99311a9970de3
Reviewed-on: http://review.whamcloud.com/7410
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4314 build: no dependency from ldiskfs sources

Fix absence dependency modules from ldiskfs sources. Otherwise the
build of modules begins early than actual sources prepared in ldiskfs
directory in case of parallel build (with -j <n> option).

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I24d3630d9132235a2dc5433eb5b8d3379c04acaa
Reviewed-on: http://review.whamcloud.com/8404
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3289 gss: gssnull security flavor

This change implements the gssnull security flavor for the purpose of
testing the Lustre GSS code.  It provides and uses a null GSS
mechanism so this testing doesn't have to involve any code related to
Kerberos or any other authentication method.

Signed-off-by: Andrew Korty <ajk@iu.edu>
Change-Id: Ic8378a052fd2a0f5a84877476a4a29aef7b0412a
Reviewed-on: http://review.whamcloud.com/8475
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Thomas Stibor <thomas@stibor.net>

LU-4299 kernel: kernel update [SLES11 SP3 3.0.101-0.8]

update target and config files for new kernel version

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ie8d7078768129bf19ce7dfd129ba509389d8a232
Reviewed-on: http://review.whamcloud.com/8762
Tested-by: Jenkins
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4360 Fix use after free in ksocknal_send

Call to ksocknal_launch_packet might schedule a callback that
might free the just sent message, and so subsequent access to it
via lntmsg->msg_vmflush goes to freed memory.

Instead we'll just remember if we are in the vmflush thread and
only restore if we happened to set mempressure flag.

Change-Id: I2f0f8b27e26e11b37ad60fde4c98e86c39768349
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/8667
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>

New tag 2.5.54

Change-Id: I6471a15e07373c2cc2c023840ddb31a492eb9d4c

LU-2753 lvfs: cleanup lvfs.h and collateral

Remove the unused struct lvfs_ucred. Remove the ucred pointer argument
from {push,pop}_ctxt() to which all called passed NULL. Remove the now
unused functions {push,pop}_group_info(). Delete lvfs_linux.h. Remove
the cb_ops member from lvfs_run_ctxt which was read but never set.
Remove the unused functions {l,ll}_dentry_open(). Reduce the number
of gratuitous includes of lvfs.h.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ia7f61e4c5a6af73381a39740dc76367655d18985
Reviewed-on: http://review.whamcloud.com/6660
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-4237 lfs: Include lfs mkdir in lfs man page

This patch adds the usage information for lfs mkdir to the lfs
man page.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I33a7ad1f48d2a14ad6b0c212a38140622f47c8a7
Reviewed-on: http://review.whamcloud.com/8221
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>

LU-1422 lnet: eliminate obsolete Cray SeaStar support

Remove the bulk of code for the no longer supported
SeaStar interconnect found on older Cray systems.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I29d07df9e7a5d33a700f7c9a14a49a9b3bf61dbe
Reviewed-on: http://review.whamcloud.com/7469
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Chuck Fossen <chuckf@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-946 lprocfs: List open files in filesystem

Added lprocfs file on MDT to list open files in per-export
directory for mdt.

Test-Parameters: testlist=sanity,sanityn
Signed-off-by: Girish Shilamkar <gshilamkar@ddn.com>
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: If8f233d95dca4cd4c4044d85bd117a027dabd80e
Reviewed-on: http://review.whamcloud.com/6386
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Swapnil Pimpale <spimpale@ddn.com>

LU-1095 debug: clean up console messages

Clean up overly verbose console error messages, improve others.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I480d61dd6febb81ca58709ff939a7807bc3ebbe5
Reviewed-on: http://review.whamcloud.com/8617
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4293 utils: handle lfs migrate failure in lfs_migrate

If "lfs migrate" returns an error, possibly because it is refusing
to migrate an IGIF FID, fall back to using rsync to copy the file
and rename it.  Print a message in this case so the user knows it
is not a fatal error yet.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I114006afb93d8c8d78923a874f3b914200500c1e
Reviewed-on: http://review.whamcloud.com/8616
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4429 llite: fix open lock matching in ll_md_blocking_ast()

In ll_md_blocking_ast() match open locks before all others, ensuring
that MDS_INODELOCK_OPEN is not cleared from bits by another open lock
with a different mode. Change the int flags parameter of
ll_md_real_close() to fmode_t fmode. Clean up verious style issues in
both functions.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I790cbbdbab75b25016c938b5f6340b20e09fc82e
Reviewed-on: http://review.whamcloud.com/8718
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4405 mdc: use ibits_known mask for lock match

Before revalidating a lock on the client, mask the lock bits against
the lock bits supported by the server (ibits_known), so newer clients
will find valid locks given by older server versions.

Xyratex-bug-id: MRP-1583

Signed-off-by: Patrick Farrell <paf@cray.com>
Signed-off-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Change-Id: I359b87e4bdc7b930e51538a4a854c47e87dd0520
Reviewed-on: http://review.whamcloud.com/8636
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2524 test: Modify tdir to be single directory

Currently, the tdir variable is a directory with a subdirectory.
This requires the '-p' option when calling mkdir. We've made tdir
be a single directory so calls to mkdir and test_mkdir do not
require the '-p' option in most cases.

tdir was changed from d0.${TESTSUITE}/d${base} to
d${testnum}.${TESTSUITE} and tfile was changed from
f.${TESTSUITE}.${testnum} to f${testnum}.${TESTSUITE}. Now tdir and
tfile are consistent in their format and the call to remove files
and directories at the beginning of many test scripts will remove
these files and directories.

Once this patch lands, we can remove the "-p" option from many
of the calls to mkdir.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ib49d7102a49ff6b5f3ec539a5b2f2f5186231a04
Reviewed-on: http://review.whamcloud.com/8123
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>

LU-3558 ptlrpc: Add the NRS TBF policy

The TBF (Token Bucket Filter) policy schedules and throttles all
types of RPCs for traffic control purposes. It divides RPCs into
different types according to their NIDs or job IDs, and enforces
a RPC rate limit on every type. The handling of a RPC will be delayed
until there are enough tokens for the type. Different types are
scheduled according to their deadlines, so that none of them will be
starving even though the service does not have the ability to satisfy
all the RPC rate requirments of types. The RPCs with the the same
types are queued in a FIFO manner.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I3f73dfbfb451cc44dfe5e0a575ec7ab5b90ac47e
Reviewed-on: http://review.whamcloud.com/6901
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3527 nodemap: add nodemap kernel module

The nodemap kernel module manages groups of mounted clients
and allows policies to be applied to them. A nodemap will be
defined as a nodemap name, and id, a tree of NID ranges that
represents the clients that are included in the nodemap, and
values that represent a policy for the behavior of the
filesystem with respect to those clients.

Initially, the policies implemented for nodemap will be static
UID/GID mapping from client IDs to a canonical filesystem IDs.
Additional flags are provided to allow the filesystem to trust
the IDs from the client (apply no mapping) and allow UID 0
(rootsquash applied to an entire nodemap).

A default nodemap is automatically provided which implicitly
contains all the clients not otherwise specified by an NID
range.

Nodemap will allow the unmapped UID/GIDs to be specified on a
case by case basis.

Nodemaps will be managed on the MGS, and configurations pushed
out the the other servers in the filesystem.

This patch adds nodemap to the build system, and defines the
basic operation for managing the addition and deletion of
the nodemap definitions.

The ioctl for management of the nodemaps from userspace was
added. The commands for adding and removing nodemaps were
added to lctl, and the handlers in the mgs.

The data structures for the module were added in
lustre/include/lustre_nodemap.

Additions were made to the make system and the m4 macros
to build nodemap with the rest of lustre.

Unit tests for adding and removing nodemaps were added
to sanity-sec.sh.

Range and idmap management functions will be added in the
two subsquent patches.

Signed-off-by: Joshua Walgenbach <jjw@iu.edu>
Change-Id: I2d34fde7aae3c2b4af512ff2c1ace5115ed40a6a
Reviewed-on: http://review.whamcloud.com/8034
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-3569 ofd: packing ost_idx in IDIF

For a normal FID, we can know on which target the related object
is allocated via querying FLDB; but it is not true for an IDIF.

To locate the OST via the given IDIF, when the IDIF is generated,
we pack the OST index in it. Then for any given FID, in spite of
it is a normal FID or not, we has the method to know which target
it belongs to. That is useful for LFSCK.

For old IDIF, the OST index is not part of the IDIF, means that
different OSTs may have the same IDIFs, that may cause the IFID
in LMA does not match the read FID. Under such case, we need to
make some compatible check to avoid to trigger unexpected.

tgt_validate_obdo() converts the ostid contained in the RPC body
to fid and changes the "struct ost_id" union, then the users can
access ost_id::oi_fid directly without call ostid_to_fid() again.

It also contains some other fixing and cleanup.

Test-Parameters: testlist=sanity-scrub
Signed-off-by: wang di <di.wang@intel.com>
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I228f2f6cd9310193a1724046cee15e3b2103c8e2
Reviewed-on: http://review.whamcloud.com/7053
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>

LU-3467 target: generic hpreq handler in target

Make high-priority request handling generic. Each request handler
may initialize now not only generic handler but also high-priority
handler. Move specific OST hp callbacks to the OFD.

Remove rq_recovery_session from ptlrpc_request and use rq_session
always. That additional session was needed when recovery request
was copied, so the normal session might become freed. Now request
is not copied but referenced and only single session is enough.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Iabf36d0828a86974bfe0638957f6018c919ac13b
Reviewed-on: http://review.whamcloud.com/7383
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-3939 tests: sanity-hsm/test_40 needs a local HSM_ARCHIVE

sanity-hsm/test_40 suffers frequent failures during auto-test due
to remote/NFS-mounted HSM_ARCHIVE causing the 400 archive requests
to take more than 100s to be drained from copytool requests queue.
This patch allows copytool_setup func to allow each sub-test to
specify a non-default hsm-root/HSM_ARCHIVE dir and test_40 uses it.
And for sure, remove test_40 from exception list.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I00e7df7b3cc5530cb96177a61ca6e07f1c784297
Reviewed-on: http://review.whamcloud.com/7703
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4430 mdt: check for MDS_FMODE_EXEC in mdt_mfd_open()

In the error path of mdt_mfd_open() check for MDS_FMODE_EXEC rather
than FMODE_EXEC in the open flags.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I04c53eb1af0fdeeb2c2b0c2f2ef1340b247921d8
Reviewed-on: http://review.whamcloud.com/8719
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3531 llite: move dir cache to MDC layer

Move directory entries cache from llite to MDC, so client
side dir stripe will use independent hash function(in LMV),
which does not need to be tightly coupled with the backend
storage dir-entry hash function. With striped directory, it
will be 2-tier hash, LMV calculate hash value according to the
name and hash-type in layout, then each MDT will store these
entry in disk by its own hash.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I14bb6bd81aad6fd59dcc22cf4bcea9d341dca2a1
Reviewed-on: http://review.whamcloud.com/7043
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4222 mdt: extra checking for getattr RPC.

Check whether getattr RPC can hold layout MD(RMF_MDT_MD),
in case the client sends some invalid RPC, which can
cause panic on MDT.

Client will retrieve cl_max_md_size/cl_default_md_size
from MDS during mount process, so it will initialize
cl_max_md_size/cl_default_md_size before sending getattr
to MDS.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I43bbe54c37360242bb7a3cd2aa8d90c2b9e0baf1
Reviewed-on: http://review.whamcloud.com/8599
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4343 tests: mkdir failing in sanity-hsm test 228

sanity-hsm test 228 calls mkdir on $tdir. Currently, the tdir
variable is two directories. This is changed in LU-2524. Until
LU-2524 lands, any call to mkdir with the tdir variable needs
the "-p" flag.

Also added removal of two files that the test creates and a new
routine to create small files with dd using the sync flag.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ic78a14625d8c45f405c013b96c84da909b9b9244
Reviewed-on: http://review.whamcloud.com/8542
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4318 lbuild: build failed if kernel source in command line

For OFED and ZFS builds the function $(find_linux_release) is used.
But it defined in $LBUILD_DIR/lbuild-$DISTRO that is not included
in case of build when kernel source tree was given on the command
line.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I5f7ae64f560c76e745dc7314465c7fa7d9ec275f
Reviewed-on: http://review.whamcloud.com/8421
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-2675 lod: remove lov and lod stuff from obd.h

Remove QOS related data structures from obd.h to the
lod_internal.h. Remove the unused functions lov_stripe_md_cmp() and
lov_lum_lsm_cmp(). Remove the declarations of several functions that
no longer exist. Move lov_lum_swab_if_needed() to the one file that
uses it.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I59874eda50aa333e5e991090fa3ac538ff8dc0f3
Reviewed-on: http://review.whamcloud.com/8687
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>

LU-2850 ptlrpc: handle sunrpc_cache_pipe_upcall change

Currently the ptlrpc GSS code has a wrapper to call
sunrpc_cache_pipe_upcall which takes three arguments.
The cache_request argument is already stored in the
cache_detail structure which is passed in already. So
for 3.8 the cache_request was removed with commit
21cd1254d3402a72927ed744e8ac1a7cf532f1ea. This patch
enabled Lustre to detect this change and run on newer
kernels.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I2be613d22aab5a0b8aa207a86e99fc63132affa0
Reviewed-on: http://review.whamcloud.com/8396
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Reviewed-by: Thomas Stibor <thomas@stibor.net>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3889 osc: Allow lock to be canceled at ENQ time

A cl_lock can be canceled when it's in CLS_ENQUEUED state.
We can't unuse this kind of lock in lov_lock_unuse() because
it will bring this lock into CLS_NEW state and then confuse
osc_lock_upcall().

Add a regression test case by Alexander Boyko.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Change-Id: I2acc7fd0176280062eb0d25dbe929b5d0144db50
Reviewed-on: http://review.whamcloud.com/8405
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3952 nfs: don't panic NFS server if MDS fails to find FID

When MDS fails to retrive the parent's fid, we'd handle it without
crashing the NFS server.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ic15c7f8f99aed38fc77c46d24da7775e1a12b4ff
Reviewed-on: http://review.whamcloud.com/8459
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

New tag 2.5.53

Change-Id: I226166f123c0c9b6cc6d655db6a6424b3f4b390b

LU-4217 build: bump build warnings to 2.7 development

The "acl" mount option on the client has been deprecated since
Lustre 1.8 (using ACLs is enforced by the admin on the MDS).
Unfortunately, there are still configs using this option, so
it cannot be removed yet, or it would cause an error at mount time.

The DNE cross-MDT locking depends on cross-MDT rename support, so
is also pushed to 2.6.53, but is expected to be fixed sooner.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Idcfe68eaec12df08538f1479a13c2c208c3ebbe5
Reviewed-on: http://review.whamcloud.com/8627
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4304 tests: fix auster to detect "SKIP" test status

This patch fixes run_suite() in auster to detect "SKIP"
test status for one test suite. If all of the sub-tests
in one test suite were skipped, then the status of the
test suite would be "SKIP" instead of "PASS".

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: Ice971aa4b15675c8a5f70f5b32092db69358565e
Reviewed-on: http://review.whamcloud.com/8381
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3319 procfs: move ost proc handling over to seq_file

Most of the current proc handling of the OST is already
based on seq_file handling except for the reporting of
the UUID of the OST. This patch moves this last piece
so that the OST layer will use strictly proc files with
seq_files.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Idf2bc014ada9292d545f761aa27c777412a66671
Reviewed-on: http://review.whamcloud.com/7928
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-1095 debug: quiet overly verbose info message

The client doesn't need to print a message for every client mount that
the layout lock feature is enabled.  This can be found at runtime via
the "import" proc file.

I also noticed that deleting OST objects logs into the debug log with
D_HA status, which is enabled by default.  Move this over to D_INODE
so it doesn't fill the OST debug logs.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ibd7b39fdd36020e62cd40883d1eac5cc7a0885dc
Reviewed-on: http://review.whamcloud.com/7918
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3974 llite: dentry d_compare changes in 3.11

In the linux 3.11 kernel the d_compare function has
removed passing in any struct inode arguments. This
patch provides support to handle this case.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I363057e4d0a119ad43a9907ec26e7e0079f7c305
Reviewed-on: http://review.whamcloud.com/7746
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3762 build: allow longer JIRA ticket letter

Allow a longer leading JIRA ticket letter to 9 characters.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I063ed7dbf6a3f648500463b0478768cc4a684a88
Reviewed-on: http://review.whamcloud.com/7338
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

LU-3319 procfs: move ldlm proc handling over to seq_file

In order to support 3.10+ kernels for clients we adapt
the ldlm proc handling to using seq_files.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Iaedae5fe1e1cd2985a6240314810db9bba3cd747
Reviewed-on: http://review.whamcloud.com/7293
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3319 procfs: move lmv proc handling over to seq_file

In order to support 3.10+ kernels for clients we adapt
the lmv proc handling to using seq_files.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I34f58c3a484ee79b41b59d4a60f6a21726373152
Reviewed-on: http://review.whamcloud.com/7291
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3963 libcfs: convert obdecho,obdclass code to atomics

This patch converts all cfs_atomic primitives in
target, obdclass and obdecho.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Signed-off-by: Peng Tao <tao.peng@emc.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I6ab6d7a476f5192e6bdfb39d4956361ce2b3e987
Reviewed-on: http://review.whamcloud.com/7075
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3386 lproc: improve osc/mdc "imports" connect data

Improve /proc/fs/lustre/{osc,mdc}/*/import files to print
the struct obd_connect_data data fields as "connect_data:"
in addition to the "connect_flags:" field.

Signed-off-by: Liu Ying <emoly.liu@intel.com>
Change-Id: If37043ee05c1ab387e0762f9ecb34fbe2602afc4
Reviewed-on: http://review.whamcloud.com/6588
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-2675 lov: remove unused lov llog code

Remove the unused functions lov_llog_init(), lov_llog_finish(),
their supporting functions, and the file lov_log.c.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I1eaad74bbea5849cc8b28388112b6bd86b7a9781
Reviewed-on: http://review.whamcloud.com/8539
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

LU-2875 mdt: use lu_name for rr_name and rr_tgt

In mdt_reint_record, use a struct lu_name for rr_name and
rr_tgt. Rename the rename target name rr_tgt to rr_tgt_name. Add a
function lu_name_is_valid() to check the validity of names. Remove
various assertions on the validity of names in MDT requests and
replace them with calls to mdt_reint_record() and returns of -EPROTO
on failure.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I6e0e207522a72eb8cec5d7918795c4aed5b9c4c2
Reviewed-on: http://review.whamcloud.com/7961
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-793 ptlrpc: fix ptlrpc_request_change_export()

The ptlrpc_request_change_export() is incompatible with recent
changes related to LU-793. It moves requests having rq_exp_list
to the new export exp_hp_req list assuming only hp requests have
rq_exp_list used, but now all requests use it. That may cause
normal requests to be moved to the hp queue.

Patch checks rq_ops instead and move request into proper list.
Second change fixes CFS_FAIL_TIMEOUT define, it uses sec * 1000
but 'sec' can be expression so it must be (sec) * 1000

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I2c385a824453a7cfaeede80180b3a83f72fc0f5b
Reviewed-on: http://review.whamcloud.com/8582
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>

LU-4364 fld: add error handler in fldb_seq_start

Check return value for load/key in fldb_seq_start,
in case there are no entries in FLDB.

Only insert special entries to the local FLDB in MDT0,
since they are all in MDT0.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: If1281800ad75396c96bd47021742ea21da7abad8
Reviewed-on: http://review.whamcloud.com/8534
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4221 osd: add case LCFG_PARAM to osd_process_config

Some proc parameters were moved from ofd to osd module and only
their symlinks were kept in ofd for interoperability/compatibility.
To process this kind of config params passed by ofd, this patch is
to do the following fixes:
- add case LCFG_PARAM to osd_process_config() to process parameters
  with prefix both PARAM_OSD and PARAM_OST.
- since these parameters are not included by the static lprocfs var
  list, a pre-check is added for them to avoid "unknown param" error
  message confuses the uses. If they are matched in this check, they
  will be passed to the osd directly.
- get rid of lprocfs_osd_init_vars() and use struct lprocfs_vars
  lprocfs_osd_{obd,module}_vars[] instead.
- improve the error messages in class_process_proc_param() and
  class_process_proc_seq_param() a little.
- add conf-sanity.sh test_28a to verify the patch and skip this test
  for ZFS OSTs since ZFS has no such kind of parameters.

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I8f64dcc29017c3ed9da010ff0a3e40c02327f05c
Reviewed-on: http://review.whamcloud.com/8238
Tested-by: Jenkins
Tested-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>

LU-3700 tests: zfs: work around blocks check for sanity-hsm/21

The number of blocks for ZFS reflects the number of blocks
written to disk.  To get an accurate number, the file on the OST needs
to be fully flushed.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I99ffac888a80b046b8c446021124e92f0f313f27
Reviewed-on: http://review.whamcloud.com/8575
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-4379 procfs: dont always check max_pages_per_rpc alignement

If connection is not established yet, cli->cl_chunkbits is
uninitilized so we cannot use it to check max_pages_per_rpc
alignment.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I4c782e2eeb5a2757d79194ee6ab6bff831a283fc
Reviewed-on: http://review.whamcloud.com/8558
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>

LU-2789 mdt: ensure constant striping during chown()

Prevent concurrent setstripe and chown() operations on a file by
acquiring mot_lov_mutex in mdt_attr_set(). This is needed to maintain
consistency between the ownership of files and their sub-objects and
to ensure that attribute changes on sub-objects are properly declared.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I75fe3476a284dccde5312a5c3f79b3698610ef25
Reviewed-on: http://review.whamcloud.com/6070
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-3528 mdt: check object exists for remote directory

Check whether the remote object exists before enqueue and
getattr to avoid LBUG.

Remove unnecssary remote object exist check in mdd_object_lock.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ia634a8c7b9cd2810515e854163c5fdd6bdf8716f
Reviewed-on: http://review.whamcloud.com/8371
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4396 tests: Remove unnecessary skip in sanity test_31n

Since test_31n now uses the function free_fd to find an unused file
descriptor, it no longer depends on fd 173 being free.  This change
removes that test on whether fd 173 is in use.

Signed-off-by: Ryan Haasken <haasken@cray.com>
Change-Id: I3b68e278d4f3a8af42e57b073ccbe45b80e76ecd
Reviewed-on: http://review.whamcloud.com/8622
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3528 osd: check LOCAL sequence file in special directory

Check Local sequence file in special directory, otherwise
special file like fldb might be missing during upgrade.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I483151510b943ae9b9d9ba0b13f5ce67c4bfa5bb
Reviewed-on: http://review.whamcloud.com/8354
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4152 mdt: Don't enqueue two locks on the same resource

Due to mechanics of ldlm internals, enqueueing two different ibits
lock on the same resource is deadlock prone.
As such change mdt_object_open_lock to release open lock if it becomes
necessary to get exclusive layout lock (to create objects).
It's ok to release the open lock right away as it's never guaranteed to
be issued anyway.

Change-Id: Ib669e68323ea72c75a0a8bea289d8bea079309b0
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/8083
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-3601 Do not create layout in lease-open

leases are not real opens so it makes no sense to create layouts
when the lease is taken.

Change-Id: Ica2d6a348c360bd20bb7bd27061839df84dae84b
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-on: http://review.whamcloud.com/8084
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-3857 osd: cleanup procfs after osd_shutdown

Since osd_procfs_fini will try cleanup all proc entries,
and osd_shutdown/qsd_fini will try to cleanup procfs
itself, so osd_procfs_fini should be done after qsd_fini,
otherwise the qsd entries will be destoryed twice, and
caused panic

Call Trace:
 [<ffffffffa081cc45>] lprocfs_remove+0x25/0x40 [obdclass]
 [<ffffffffa0b23dd0>] qsd_fini+0x80/0x450 [lquota]
 [<ffffffffa0d2ec78>] osd_shutdown+0x38/0xe0 [osd_ldiskfs]
 [<ffffffffa0d36bf9>] osd_device_fini+0x129/0x190 [osd_ldiskfs]
 [<ffffffffa0834913>] class_cleanup+0x573/0xd30 [obdclass]
 [<ffffffffa081233c>] ? class_name2dev+0x7c/0xe0 [obdclass]
 [<ffffffffa083663a>] class_process_config+0x156a/0x1ad0 [obdclass]
 [<ffffffffa06be9b8>] ? libcfs_log_return+0x28/0x40 [libcfs]
 [<ffffffffa082f202>] ? lustre_cfg_new+0x312/0x6e0 [obdclass]
 [<ffffffffa0836d19>] class_manual_cleanup+0x179/0x6e0 [obdclass]
 [<ffffffffa06be9b8>] ? libcfs_log_return+0x28/0x40 [libcfs]
 [<ffffffffa0d378b4>] osd_obd_disconnect+0x174/0x1e0 [osd_ldiskfs]
 [<ffffffffa083926b>] lustre_put_lsi+0x1ab/0xeb0 [obdclass]
 [<ffffffffa08414d8>] lustre_common_put_super+0x5c8/0xbe0 [obdclass]
 [<ffffffffa087081d>] server_put_super+0x1bd/0xed0 [obdclass]
 [<ffffffffa0871bbb>] server_fill_super+0x68b/0x1630 [obdclass]
 [<ffffffffa0840bb8>] lustre_fill_super+0x1d8/0x530 [obdclass]
 [<ffffffffa08409e0>] ? lustre_fill_super+0x0/0x530 [obdclass]

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: If12cebf971583afeeaa031bd24f69bb0fe0cdf1a
Reviewed-on: http://review.whamcloud.com/8506
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4321 build: Remove make_META.pl

This patch is a first pass at cleaning up version_tag.pl.

Primarily, it removes the strange "make_META.pl" symlink behavior.  It
is not at all clear to me why the symlink was being generated on the fly,
and then version_tag.pl changed its behavior based on its name.  Odd.

Instead version_tag.pl now take "--make_META" as the first option on
its command line.

Further, we haven't use CVS in a long time, so this patch removes
the cvs-related file and checks.

The is_git and is_cvs variables weren't even used, so those are removed
as well.

Change-Id: I651234bdf1d2d32db16151f332f2b3785ee123a0
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/8427
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4022 hsm: Set agent_actions start index from -1

aai_index was set to start from 0 which will cause the first entry
to be skipped. Set it to -1 instead.

Also take llog lock in proc_show() to avoid it from being changed
during the iteration.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I09013dbf8a85d198496d8470746e773b388d9bee
Reviewed-on: http://review.whamcloud.com/7879
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4365 quota: wait for global lock cancel

In qsd_qtype_fini(), we'd wait for the global lock cancel done.

Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes  mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs  testlist=recovery-small
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ib51dba4e4e4b13693c49560162d53cb28937e253
Reviewed-on: http://review.whamcloud.com/8583
Tested-by: Jenkins
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>