Whamcloud - gitweb
fs/lustre-release.git
5 years agoLU-10808 lod: remove DoM component if DoM is disabled 82/32482/5
Mikhail Pershin [Mon, 21 May 2018 18:24:05 +0000 (21:24 +0300)]
LU-10808 lod: remove DoM component if DoM is disabled

If file is created with DoM component but server disables
DoM file creation then remove DoM entry from file layout
and keep other components.
If layout has only DoM entry then just return error.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Ibafd0269d76dc5de4599efca064930607dc556eb
Reviewed-on: https://review.whamcloud.com/32482
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-11014 mdt: intent handling simplification 57/32357/3
John L. Hammond [Fri, 11 May 2018 14:01:32 +0000 (09:01 -0500)]
LU-11014 mdt: intent handling simplification

Remove the obsolete constants MDT_IT_CREATE, MDT_IT_READDIR,
MDT_IT_UNLINK, and MDT_IT_TRUNC from enum mdt_it_code. Also remove
MDT_IT_OCREAT, since (at this level) it can be handled identically to
MDT_IT_OPEN. Rename mdt_intent_reint() to mdt_intent_open() since it
only handles open. Move the definition of the mdt_it_flavor array down
and remove the then unneeded forward declarations of mdt_intent_*().
In struct mdt_it_flavor, remove the obsolete it_reint member and
rename the it_flags member to it_handler_flags to avoid confusion with
LDLM flags. Use 'enum tgt_handler_flags' rather than __u32 for several
parameters used to hold values of that type.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I297ef397c879fcc7711d725e0315e73439d95826
Reviewed-on: https://review.whamcloud.com/32357
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10977 test: add version check to sanity test_60ab 43/32343/4
Saurabh Tandan [Wed, 9 May 2018 21:27:15 +0000 (14:27 -0700)]
LU-10977 test: add version check to sanity test_60ab

Skip sanity.sh test_60ab if server is equal or
less than 2.11.51

Test-Parameters:trivial testlist=sanity envdefinitions=ONLY=60ab serverjob=lustre-b2_10 serverbuildno=69
Signed-off-by: Saurabh Tandan <saurabh.tandan@intel.com>
Change-Id: Ie9d2728790e19ac2a24c94e7c13ade28b5a5bbbe
Reviewed-on: https://review.whamcloud.com/32343
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-4423 obd: backport of lu_object changes upstream 25/32325/3
NeilBrown [Wed, 9 May 2018 02:46:29 +0000 (22:46 -0400)]
LU-4423 obd: backport of lu_object changes upstream

fold lu_object_new() into lu_object_find_at()

lu_object_new() duplicates a lot of code that is in
lu_object_find_at().
There is no real need for a separate function, it is simpler just
to skip the bits of lu_object_find_at() that we don't
want in the LOC_F_NEW case.

Linux-commit: 775c4dc274343e5e2959fa1171baf2fc01028840

discard extra lru count.

lu_object maintains 2 lru counts.
One is a per-bucket lsb_lru_len.
The other is the per-cpu ls_lru_len_counter.

The only times the per-bucket counters are use are:
 - a debug message when an object is added
 - in lu_site_stats_get when all the counters are combined.

The debug message is not essential, and the per-cpu counter
can be used to get the combined total.

So discard the per-bucket lsb_lru_len.

Linux-commit: e167b370360f8887cf21a2a82f83e7118a2aeb11

make struct lu_site_bkt_data private

This data structure only needs to be public so that
various modules can access a wait queue to wait for object
destruction.
If we provide a function to get the wait queue, rather than the
whole bucket, the structure can be made private.

Linux-commit: bc5e7fb40d36edb95ce8f661596811bec3f7d5cf

Change-Id: I26203f331a0c73ae4e23878eb10b15d9fcf546c5
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32325
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10971 tests: use changelog routines in lustre-rsync-test 08/32208/4
James Nunez [Mon, 30 Apr 2018 19:12:49 +0000 (13:12 -0600)]
LU-10971 tests: use changelog routines in lustre-rsync-test

The lustre-rsync-test script has two subroutines to register
and deregister changelog users. These subroutines should be
updated to use changelog_register() and changelog_deregister()
found in test-framework.sh.

Test-Parameters: trivial clientcount=2 mdscount=2 mdtcount=4 osscount=1 ostcount=8 mdtfilesystemtype=zfs ostfilesystemtype=zfs testlist=lustre-rsync-test
Test-Parameters: clientcount=2 mdscount=2 mdtcount=4 osscount=1 ostcount=8 mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs testlist=lustre-rsync-test
Test-Parameters: clientcount=2 mdscount=1 mdtcount=1 osscount=1 ostcount=8 mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs testlist=lustre-rsync-test
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ia54095a6e039f6835def0f9c49157b71088d9e51
Reviewed-on: https://review.whamcloud.com/32208
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10808 lod: align wrong DoM stripe values with defaults 73/32073/5
Mikhail Pershin [Thu, 19 Apr 2018 13:29:54 +0000 (16:29 +0300)]
LU-10808 lod: align wrong DoM stripe values with defaults

- Align DoM component size to the server limit size instead of
  returning a error. Error is returned still if DoM file creation
  is disabled on the server (DOM limit is set to 0)
- Correct wrong values for dom_stripesize parameter by using minimal
  stripe size if provided value is lower and by aligning it to be a
  multiple of that minimal size.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Ifcdf60fddda65acda92509bb7e69c9b2951fb6bd
Reviewed-on: https://review.whamcloud.com/32073
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-4423 ptlrpc: use delayed_work in sec_gc 24/31724/3
Dmitry Eremin [Thu, 22 Mar 2018 15:51:00 +0000 (18:51 +0300)]
LU-4423 ptlrpc: use delayed_work in sec_gc

The garbage collection for security contexts currently has a dedicated
kthread which wakes up every 30 minutes to discard old garbage.

Replace this with a simple delayed_work item on the system work queue.

Change-Id: I5cdb023783104b5e21f4139731065946ed162af1
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-on: https://review.whamcloud.com/31724
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10648 ldlm: Reduce debug to console during eviction 37/31237/4
Patrick Farrell [Fri, 26 Aug 2016 16:03:33 +0000 (11:03 -0500)]
LU-10648 ldlm: Reduce debug to console during eviction

During an eviction, Lustre calls ldlm_namespace_cleanup,
and it will sometimes end up dumping all of the locks on a
particular resource to the console log
(ldlm_resource_complain), which is very wasteful and only
rarely helpful.

Move the debug level for this to D_NETERROR since it is in the
default debug mask.

Change-Id: I8a00f030393ce1748914d70fa8edb4690273e08a
Cray-bug-id: LUS-1418
Signed-off-by: Chris Horn <hornc@cray.com>
Signed-off-by: Patrick Farrell <paf@cray.com>
Reviewed-on: https://review.whamcloud.com/31237
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10472 osc: add T10PI support for RPC checksum 80/30980/37
Li Xi [Tue, 23 Jan 2018 07:17:17 +0000 (02:17 -0500)]
LU-10472 osc: add T10PI support for RPC checksum

T10 Protection Information (T10 PI), previously known as Data
Integrity Field (DIF), is a standard for end-to-end data integrity
validation. T10 PI prevents silent data corruption, ensuring that
incomplete and incorrect data cannot overwrite good data.

Lustre file system already supports RPC level checksum which
validates the data in bulk RPCs when writing/reading data to/from
objects on OSTs. RPC level checksum can detect data corruption that
happens during RPC being transferred over the wire. However, it is
not capable to prevent silent data corruption happening in other
conditions, for example, memory corruption when data is cached in
page cache. And by using the existing checksum mechanism, only
disjoint protection coverage is provided. Thus, in order to provide
end-to-end data protection, T10PI support for Lustre should be added.

In order to provide end-to-end data integrity validation, the T10 PI
checksum of data in a sector need to be calculated on Lustre client
side and validated later on the Lustre OSS side. The T10 protection
information should be sent together with the data in the RPC.
However, in order to avoid significant performance degradation,
instead of sending all original guard tags for all sectors in a bulk
RPC, the existing checksum feature of bulk RPC will be integrated
together with the new T10PI feature.

When OST starts, necessary T10PI information will be extracted from
storage, i.e. the T10PI DIF type and sector size. The DIF type could
be one of TYPE1_IP, TYPE1_CRC, TYPE3_IP and TYPE3_CRC. And sector
size could be either 512 or 4K bytes.

When an OSC is connecting to OST, OSC and OST will negotiate about
the checksum types. New checksum types are added for T10PI support
including OBD_CKSUM_T10IP512, OBD_CKSUM_T10IP4K, OBD_CKSUM_T10CRC512,
and OBD_CKSUM_T10CRC4K. If the OST storage has T10PI suppoort, the
only selectable T10PI checksum type would have the same type with the
T10PI type of the hardware. The other existing checksum types (crc32,
crc32c, adler32) are still valid options for the RPC checksum type.

When calculating RPC checksum of T10PI, the T10PI checksums of all
sectors will be calculated first using the T10PI chekcsum type, i.e.
16-bit crc or IP checksum. And then RPC checksum will be calculated on
all of the T10PI checksums. The RPC checksum type used in this step is
always alder32. Considering that the checksum-of-checksums is only
computed on a * 4KB chunk of GRD tags for a 1MB RPC for 512B sectors,
or 16KB of GRD tags for 16MB of 4KB sectors, this is only 1/256 or
1/1024 of the total data being checksummed, so the checksum type used
here should not affect overall system performance noticeably.

obdfilter.*.enforce_t10pi_cksum can be used to tune whether to enforce
T10-PI checksum or not.

If the OST supports T10-PI feature and T10-PI chekcsum is enforced, clients
will have no other choice for RPC checksum type other than using the T10PI
chekcsum type. This is useful for enforcing end-to-end integrity in the
whole system.

If the OST doesn't support T10-PI feature and T10-PI chekcsum is enforced,
together with other checksums with reasonably good speeds (e.g. crc32,
crc32c, adler, etc.), all the T10-PI checksum types (t10ip512, t10ip4K,
t10crc512, t10crc4K) will be added to the available checksum types,
regardless of the speeds of T10-PI chekcsums. This is useful for testing
T10-PI checksums of RPC.

If the OST supports T10-PI feature and T10-PI chekcsum is NOT enforced,
the corresponding T10-PI checksum type will be added to the checksum type
list, regardless of the speed of the T10-PI chekcsum. This provide the
clients to flexibility to choose whether to enable end-to-end integrity
or not.

If the OST does NOT supports T10-PI feature and T10-PI chekcsum is NOT
enforced, together with other checksums with reasonably good speeds,
all the T10-PI checksum types with good speeds will be added into the
checksum type list. Note that a T10-PI checksum type with a speed worse
than half of Alder will NOT be added as a option. In this circumstance,
T10-PI checksum types has the same behavior like other normal checksum
types.

The clients that has no T10-PI RPC checksum support will not be affected
by the above-mentioned logic. And that logic will only be enforced to the
newly connected clients after changing obdfilter.*.enforce_t10pi_cksum on
an OST.

Following are the speeds of different checksum types on a server with CPU
of Intel(R) Xeon(R) E5-2650 @ 2.00GHz:

crc: 1575 MB/s
crc32c: 9763 MB/s
adler: 1255 MB/s
t10ip512: 6151 MB/s
t10ip4k: 7935 MB/s
t10crc512: 1119 MB/s
t10crc4k: 1531 MB/s

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I6468680edeab0917bb71dbd8cd9ea16c65e935f5
Reviewed-on: https://review.whamcloud.com/30980
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
5 years agoLU-8066 llite: Preparation to move /proc/fs/lustre/llite to sysfs 31/24031/23
James Simmons [Fri, 25 May 2018 01:16:18 +0000 (21:16 -0400)]
LU-8066 llite: Preparation to move /proc/fs/lustre/llite to sysfs

Add necessary infrastructure, add support for mountpoint
registration in /sys/fs/lustre/llite

This is a heavly modified version of

Linux-commit: fd0d04ba85f95169106701397417360541a983b3

due to the large amount of changes to the OpenSFS/Intel branch.

Change-Id: Ic9ca2044249a59dc79ebc86553c8b7ce7afbf710
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/24031
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10264 misc: fix possible array overflow 42/32242/2
Andreas Dilger [Fri, 9 Mar 2018 23:18:53 +0000 (16:18 -0700)]
LU-10264 misc: fix possible array overflow

Fix a static analysis error.

lustre/obdclass/obd_mount_server.c:1830 in osd_start(), buffer
    flagstr has size 16 but length of format string "%lu:%lu" is 31.
Increase the size of buffer to hold maximal-sized strings plus NUL.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I3cc80d66bbb537161a561f4f2ba7830dde2cab07
Reviewed-on: https://review.whamcloud.com/32242
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-8972 tests: remove conf-sanity test from ALWAYS_EXCEPT 20/32220/4
James Nunez [Tue, 1 May 2018 19:20:28 +0000 (13:20 -0600)]
LU-8972 tests: remove conf-sanity test from ALWAYS_EXCEPT

A patch landed to fix the issue reported in LU-8972. We need
to run conf-sanity test 101 to ensure that the issue is
fixed and does not regress.

Remove conf-sanity test 101 from the ALWAYS_EXCEPT list.

Test-Parameters: trivial
Test-Parameters: trivial clientcount=2 mdscount=2 mdtcount=4 osscount=1 ostcount=8 mdtfilesystemtype=zfs ostfilesystemtype=zfs testlist=conf-sanity
Test-Parameters: clientcount=2 mdscount=2 mdtcount=4 osscount=1 ostcount=8 mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs testlist=conf-sanity
Test-Parameters: clientcount=2 mdscount=1 mdtcount=1 osscount=1 ostcount=8 mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs testlist=conf-sanity
Test-Parameters: clientcount=2 mdscount=1 mdtcount=1 osscount=1 ostcount=8 mdtfilesystemtype=zfs ostfilesystemtype=zfs testlist=conf-sanity
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ic678c7527a60cab2de6139041cca81017d4aa75e
Reviewed-on: https://review.whamcloud.com/32220
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-6160 osd-zfs: Fix refcount_add call 44/28544/2
Giuseppe Di Natale [Mon, 14 Aug 2017 16:51:52 +0000 (09:51 -0700)]
LU-6160 osd-zfs: Fix refcount_add call

Correct the refcount_add in osd-zfs module's osd_fix_new_dnode
function. The variable 'tag' was undefined and caused osd-zfs
to fail builds against zfs packages with debug enabled.

This small change should enable lustre to be built against
zfs packages that have debug enabled.

Test-Parameters: trivial
Signed-off-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Change-Id: If95f0af6178cf0ea78724658edfaece1ee16a3f1
Reviewed-on: https://review.whamcloud.com/28544
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-9727 tests: exercise new changelog fields and records 35/32335/4
Sebastien Buisson [Fri, 19 Jan 2018 17:22:40 +0000 (02:22 +0900)]
LU-9727 tests: exercise new changelog fields and records

Add new tests in sanity-hsm to exercise new changelog fields
and also record types.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I1cd7282983d936105e1616aa859c47fd453e6017
Reviewed-on: https://review.whamcloud.com/32335
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10796 tests: standardize changelog testing in sanity-hsm 13/31613/13
Quentin Bouget [Fri, 9 Mar 2018 14:20:03 +0000 (14:20 +0000)]
LU-10796 tests: standardize changelog testing in sanity-hsm

To manage changelog users and changelog records, sanity-hsm used to
define:
 - changelog_setup
 - changelog_cleanup
 - changelog_get_flags

test-framework.sh implements similar functions:
 - changelog_register
 - changelog_deregister
 - changelog_dump
 - changelog2array (new in this patch)

This patch removes the implementations of sanity-hsm in favor of
those in test-framework.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: Ie3db8ef646fa48d06bf41b6025b3443de026cabd
Reviewed-on: https://review.whamcloud.com/31613
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-8130 ldlm: store name directly in namespace. 08/32408/2
NeilBrown [Tue, 15 May 2018 17:29:27 +0000 (13:29 -0400)]
LU-8130 ldlm: store name directly in namespace.

Rather than storing the name of a namespace in the
hash table, store it directly in the namespace.
This will allow the hashtable to be changed to use
rhashtable.

Linux-commit: 648ae363628c84faa8d8861e3246e096b8c0a392

Change-Id: Ie5bb8092c9e1831fbc38beade46be6d35f3256dc
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/32408
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
5 years agoLU-11017 quota: ignore quota for CAP_SYS_RESOURCE properly 78/32378/10
Wang Shilong [Wed, 16 May 2018 02:13:13 +0000 (10:13 +0800)]
LU-11017 quota: ignore quota for CAP_SYS_RESOURCE properly

Currently, lustre quota will ignore this type of quota if
quota id is 0 or we force to ignore.

For write, we have passed CAP_SYS_RESOURCE properly, but
For metadata operations this is not done.

Test-Parameters: testlist=sanity-quota
Change-Id: Ibcdc0e53ad125042d4889ac51a9a9ead4066c0c8
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/32378
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-11015 lov: Move lov_tgts_kobj init to lov_setup 67/32367/4
Oleg Drokin [Fri, 18 May 2018 01:43:18 +0000 (21:43 -0400)]
LU-11015 lov: Move lov_tgts_kobj init to lov_setup

and free it in lov_cleanup.
This looks like a more robust solution vs doint it in lov_putref
esp. since we know refcount there crosses 0 repeatedly, confusing
things.

Change-Id: I49b1a1e97464bd388fe20a97b903468139730213
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/32367
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
5 years agoLU-11010 tests: remove calls to return after skip() 46/32346/2
James Nunez [Wed, 9 May 2018 22:11:19 +0000 (16:11 -0600)]
LU-11010 tests: remove calls to return after skip()

The skip() routine now contains a call to exit. All calls
to skip() and skip_env() should be reviewed and calls to
return that followed skip should be removed.

A problem with the skip message not being printed is corrected.

Test-Parameters: trivial testlist=sanity-pfl
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I1a52e9bd79a71de4ab4c0cea9c569f379115a603
Reviewed-on: https://review.whamcloud.com/32346
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-11003 ldlm: fix for l_lru usage 09/32309/3
Yang Sheng [Mon, 7 May 2018 15:59:01 +0000 (23:59 +0800)]
LU-11003 ldlm: fix for l_lru usage

Fixes for lock convert code to prevent false assertion and
busy locks in LRU:
- ensure no l_readers and l_writers when add lock to LRU after
  convert.
- don't verify l_lru without ns_lock.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I8bcbdef3cb72db241ad03c50f5ce2b968e3ee3e4
Reviewed-on: https://review.whamcloud.com/32309
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10855 llog: remove obsolete llog handlers 02/32202/2
John L. Hammond [Mon, 30 Apr 2018 14:08:48 +0000 (09:08 -0500)]
LU-10855 llog: remove obsolete llog handlers

Remove the obsolete llog RPC handling for cancel, close, and
destroy. Remove llog handling from ldlm_callback_handler(). Remove the
unused client side method llog_client_destroy().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ieab44f3796971a7d3c65d6044e4c0be4afb4b508
Reviewed-on: https://review.whamcloud.com/32202
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10938 ptlrpc: Add WBC connect flag 41/32241/5
Oleg Drokin [Wed, 2 May 2018 07:03:38 +0000 (03:03 -0400)]
LU-10938 ptlrpc: Add WBC connect flag

It denotes ability of the node to understand additional
types of intent requests, exclusive metadata locks issued
to clients and server operations performed under such
locks while still held by clients.

Test-Parameters: trivial
Change-Id: I72c1ddfdf94edea3b357d82da6c410bc2d79a75c
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/32241
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
5 years agoLU-10945 ldlm: fix l_last_activity usage 33/32133/3
Alexander Boyko [Tue, 24 Apr 2018 07:06:42 +0000 (03:06 -0400)]
LU-10945 ldlm: fix l_last_activity usage

When race happen between ldlm_server_blocking_ast() and
ldlm_request_cancel(), the at_measured() is called with wrong
value equal to current time. And even worse, ldlm_bl_timeout() can
return current_time*1.5.
Before a time functions was fixed by LU-9019(e920be681) for 64bit,
this race leads to ETIMEDOUT at ptlrpc_import_delay_req() and
client eviction during bl ast sending. The wrong type conversion
take a place at pltrpc_send_limit_expired() at cfs_time_seconds().

We should not take cancels into accoount if the BLAST is not send,
just because the last_activity is not properly initialised - it
destroys the AT completely.
The patch devides l_last_activity to the client l_activity and
server l_blast_sent for better understanding. The l_blast_sent is
used for blocking ast only to measure time between BLAST and
cancel request.

For example:
 server cancels blocked lock after 1518731697s
 waiting_locks_callback()) ### lock callback timer expired after 0s:
 evicting client

Signed-off-by: Alexander Boyko <c17825@cray.com>
Change-Id: I44962d2b3675b77e09182bbe062bdd78d6cb0af5
Cray-bug-id: LUS-5736
Reviewed-on: https://review.whamcloud.com/32133
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-4684 xattr: add list support for remote object 26/31426/9
Lai Siyao [Sun, 21 Jan 2018 07:57:22 +0000 (15:57 +0800)]
LU-4684 xattr: add list support for remote object

XATTR_LIST may be issued to a remote object in directory migration,
add this support for OSP and OUT.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I9681e149703de2837a04dc1448d1bd583659205d
Reviewed-on: https://review.whamcloud.com/31426
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-4684 mdt: improve directory stripe lock 25/31425/9
Lai Siyao [Sun, 21 Jan 2018 05:22:24 +0000 (13:22 +0800)]
LU-4684 mdt: improve directory stripe lock

Striped directory has an implication that the first stripe is
local, and others are remote, but this is not true for migrating
directory because its stripes consists of both the original and
the newly created stripes.

This patch also put striped directory master object locking and
stripes locking into one function called mdt_reint_striped_lock(),
which simplifies locking code.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I4724447e5b10c301b6799e1827f6d13a40876945
Reviewed-on: https://review.whamcloud.com/31425
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-11026 lustre-dkms should require patch or quilt 31/32431/3
Joe Grund [Wed, 16 May 2018 17:18:56 +0000 (13:18 -0400)]
LU-11026 lustre-dkms should require patch or quilt

Add patch requirement to lustre-dkms.spec.in
as it (or quilt) are needed for lustre-build-ldiskfs.

  - Add requires patch.

Change-Id: I640bae382511502c02a0237694c93c304047f339
Signed-off-by: Joe Grund <joe.grund@intel.com>
Reviewed-on: https://review.whamcloud.com/32431
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
5 years agoLU-10964 build: armv7 client build fixes 94/32194/8
Andrew Perepechko [Sat, 28 Apr 2018 08:32:03 +0000 (11:32 +0300)]
LU-10964 build: armv7 client build fixes

This commit is supposed to fix armv7 Lustre client
build, mostly 64-bit division related changes.

Change-Id: I93d83d577351c1a1053e39a162cb1e85fc4e8aa3
Signed-off-by: Andrew Perepechko <c17827@cray.com>
Reviewed-on: https://review.whamcloud.com/32194
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-7943 mdd: Move assignment after LASSERT() 76/32376/2
Arshad Hussain [Sat, 12 May 2018 08:43:54 +0000 (14:13 +0530)]
LU-7943 mdd: Move assignment after LASSERT()

This patch moves 'sname->ln_namelen' assignment call after LASSERT() call.
This avoids a case when 'sname' parameter is NULL and dereferencing the
NULL pointer would fault before it reaches LASSERT()

Change-Id: I68b07f7ca33fd21ee0599b7bb73d6e41546bd2d8
Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.whamcloud.com/32376
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
5 years agoLU-10897 kernel: kernel upgrade RHEL7.5 [3.10.0-862.2.3.el7] 70/32370/5
Bob Glossman [Thu, 10 May 2018 14:46:35 +0000 (07:46 -0700)]
LU-10897 kernel: kernel upgrade RHEL7.5 [3.10.0-862.2.3.el7]

With this mod we switch our supported el7 version to RHEL 7.5

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Iedcea9498591d15eab69187274e4c32c57879e4e
Reviewed-on: https://review.whamcloud.com/32370
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-11019 build: Update ZFS/SPL to 0.7.9 88/32388/2
Nathaniel Clark [Mon, 14 May 2018 19:06:45 +0000 (15:06 -0400)]
LU-11019 build: Update ZFS/SPL to 0.7.9

This updates the ZFS version to 0.7.9.

https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.7.9

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I9452a589d9dc719de7a63d3ed287dec8b6f7c0b6
Reviewed-on: https://review.whamcloud.com/32388
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
5 years agoLU-4423 libcfs: disable preempt while sampling processor id. 60/32360/3
NeilBrown [Sat, 12 May 2018 17:52:58 +0000 (13:52 -0400)]
LU-4423 libcfs: disable preempt while sampling processor id.

Calling smp_processor_id() without disabling preemption
triggers a warning (if CONFIG_DEBUG_PREEMPT).
I think the result of cfs_cpt_current() is only used as a hint for
load balancing, rather than as a precise and stable indicator of
the current CPU.  So it doesn't need to be called with
preemption disabled.

So disable preemption inside cfs_cpt_current() to silence the warning.

Linux-commit : dbeccabf5294e80f7cc9ee566746c42211bed736

Change-Id: Iaa930acc7a2633c0e40bcabbe6bd309a3d767325
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32360
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-9859 libcfs: rearrange placement of CPU partition management code. 59/32359/3
NeilBrown [Sat, 12 May 2018 13:51:30 +0000 (09:51 -0400)]
LU-9859 libcfs: rearrange placement of CPU partition management code.

Currently the code for cpu-partition tables lives in various places.
The non-SMP code is partly in libcfs/libcfs_cpu.h as static inlines,
and partly in lnet/libcfs/libcfs_cpu.c - some of the functions are
tiny and could well be inlines.

The SMP code is all in lnet/libcfs/linux/linux-cpu.c.

This patch moves all the trivial non-SMP functions into
libcfs_cpu.h as inlines, and all the SMP functions into libcfs_cpu.c
with the non-trival !SMP code.

Now when you go looking for some function, it is easier to find both
versions together when neither is trivial.

There is no code change here - just code movement.

Linux-commit: 93aa2c2a5091bd47819a3ead4af70fb57fda5065

Change-Id: I5250a52cad576eaeec17de176a3ca45ad076c4b9
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32359
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10997 build: add files to .gitignore 04/32304/2
James Simmons [Sat, 5 May 2018 18:00:30 +0000 (14:00 -0400)]
LU-10997 build: add files to .gitignore

To avoid by accident adding files created during the build
process add them to the .gitignore. For Ubuntu 18 we add
in .cache.mk and *.o.ur-safe which only show up on that
platform

Test-Parameters: trivial

Change-Id: Ie1329e765f080cbdac1bd3efdd63f83a65d45989
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32304
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
5 years agoLU-10886 build: fix WARNING: modpost: missing MODULE_LICENSE() 03/32303/3
James Simmons [Sat, 5 May 2018 21:35:30 +0000 (17:35 -0400)]
LU-10886 build: fix WARNING: modpost: missing MODULE_LICENSE()

With newer kernels modpost will print a warning when no license is
provided. This could give false results if the kernel test set
the -Werror flag. Update LB_LANG_PROGRAM macro to include the
missing MODULE_LICENSE().

Test-Parameters: trivial

Change-Id: Ia21d0fa5ee6c224d05b7949540ef805d09d3c7c5
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32303
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10906 checksum: enable/disable checksum correctly 95/32095/7
Emoly Liu [Fri, 20 Apr 2018 11:10:00 +0000 (19:10 +0800)]
LU-10906 checksum: enable/disable checksum correctly

There are three ways to set checksum support in Lustre. Their
order during client mount is:
- 1. configure --enable/disable-checksum, this(ENABLE_CHECKSUM)
  only affects the default mount option and is set in function
  client_obd_setup().
- 2. lctl set_param -P osc.*.checksums=0/1, when processing llog,
  this value will be set by osc_checksum_seq_write().
- 3. mount option checksum/nochecksum, this will be checked in
  ll_options() and be set in client_common_fill_super()->
  obd_set_info_async().

This patch fixes one issue in 3. That is if mount option
"-o checksum/nochecksum" is specified, checksum will be changed
accordingly, no matter what is set by "set_param -P" or the
default option; and if no mount option is specified, the value
set by "set_param -P" will be kept. Also, test_77k is added to
sanity.sh to verify this patch.

What's more, a minor initialization issue of cl_supp_cksum_types
is fixed. cl_supp_cksum_types should be always initialized no
matter checksum is enabled or not.

Change-Id: I95d73122d800f5cd44b5fabb0cf00b5be0a35443
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-on: https://review.whamcloud.com/32095
Reviewed-by: Yingjin Qian <qian@ddn.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
5 years agoLU-9474 tests: remove unneeded hsm_set_param for raolu tests 29/31429/3
Quentin Bouget [Tue, 27 Feb 2018 15:22:08 +0000 (15:22 +0000)]
LU-9474 tests: remove unneeded hsm_set_param for raolu tests

A previous commit introduced the stack_trap() utility function,
the patch should have removed any duplicate cleanup actions but
missed a few in test_26{a,b,c,d} of sanity-hsm.

This commits removes them.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I63e1df6610d0fbceb2f88a5b59b8263b8ccaf525
Reviewed-on: https://review.whamcloud.com/31429
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
5 years agoLU-739 utils: remove references to lustre 1.4 upgrade flag 42/27342/6
Gregoire Pichon [Sun, 13 May 2018 01:43:18 +0000 (21:43 -0400)]
LU-739 utils: remove references to lustre 1.4 upgrade flag

Remove all references to LDD_F_UPGRADE14 flag since we don't support
1.4 upgrade any more. Also we can remove CFG_F_COMPAT146.

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: Ib0ac680646751f9372923875142ce04ea5c8ec53
Reviewed-on: https://review.whamcloud.com/27342
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-6399 lnet: socket cleanup 79/14179/8
Alexander Boyko [Mon, 7 May 2018 13:56:51 +0000 (09:56 -0400)]
LU-6399 lnet: socket cleanup

The ioctl request was designed to get all needed information
through socket from usermode. But the same patterns with tricks
was used at kernel by lib-socket..
The patch changes this behavior from socket to kernel socket.
For libcfs_sock_ioctl tricks with usermode call removed,
kernel_sock_ioctl is used instead. But this call handle SIOCGIFADDR
and SIOCGIFNETMASK. For SIOCGIFFLAGS we take device flag directly.
Function libcfs_ipif_enumerate() which are used SIOCGIFCONF command
totally rewriten to get device names without ioctl requests.

Signed-off-by: Alexander Boyko <c17825@cray.com>
Change-Id: Idf0da800a49dbefa419fc5fecaa9ee1bd4d85327
Reviewed-on: https://review.whamcloud.com/14179
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <dougso@me.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-11024 osd-zfs: properly detect ZFS dnode accounting 18/32418/3
Fan Yong [Wed, 16 May 2018 08:00:29 +0000 (16:00 +0800)]
LU-11024 osd-zfs: properly detect ZFS dnode accounting

Properly detect if native ZFS dnode accounting is available for
ZFS 0.7.x releases that do not contain ZFS project quota.  The
function signature changed after ZFS Project Quota was landed,
but we still need to check for the old function for 0.7.x.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Icd40e0bdaa0b7738e9aa761836167780843ebbe5
Reviewed-on: https://review.whamcloud.com/32418
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
5 years agoLU-10972 test: get client uuid correctly 89/32289/3
Emoly Liu [Fri, 4 May 2018 05:19:28 +0000 (13:19 +0800)]
LU-10972 test: get client uuid correctly

Correct sanity.sh test_0d script to get client uuid value only.

Test-Parameters:trivial testlist=sanity envdefinitions=ONLY=0d \
serverjob=lustre-b2_11 serverbuildno=2

Change-Id: I6167d9b9c3c3e86f2a5581f8ac4ccab8f1137ce7
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-on: https://review.whamcloud.com/32289
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
5 years agoLU-10989 utils: correct lustre_rsync changelog clear logic 47/32247/2
John L. Hammond [Wed, 2 May 2018 14:43:07 +0000 (09:43 -0500)]
LU-10989 utils: correct lustre_rsync changelog clear logic

In the non-extended rename case of lr_replicate() copy the record
number from ext to info. Then remove the spurious rename record
handling from lr_clear_cl().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I615ec2f384f5f9d7807156acb3ce66ac47ca1e77
Reviewed-on: https://review.whamcloud.com/32247
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10978 utils: preserve lustre_rsync state 46/32246/2
John L. Hammond [Wed, 2 May 2018 14:09:05 +0000 (09:09 -0500)]
LU-10978 utils: preserve lustre_rsync state

In lustre_rsync, if debugging is enabled then we zero-out info and ext
in each iteration of the main loop of lr_replicate(). This may prevent
the consumed changelog entries from being cleared at the end of
lr_replicate().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ibd089524ca6cd6f488fc3558882cc431fdf65de7
Reviewed-on: https://review.whamcloud.com/32246
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10980 test: correct the version code check in sanityn.sh 39/32239/3
Emoly Liu [Wed, 2 May 2018 06:29:54 +0000 (14:29 +0800)]
LU-10980 test: correct the version code check in sanityn.sh

The following patch was landed to 2.11.50 instead of 2.10.58, so
the version code check in sanityn.sh test_77ja should be corrected:
- Lustre-commit: e0cdde123c14729a340cf937cf9580be7c9dd9c1
- Lustre-change: https://review.whamcloud.com/27608

Test-Parameters: trivial testlist=sanityn envdefinitions=ONLY=77ja \
serverjob=lustre-b2_11 serverbuildno=2

Change-Id: I619d5fd6fa2297663084d3b5b8bdbc1b4642cdc1
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-on: https://review.whamcloud.com/32239
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10979 test: correct the version code check in sanity-sec.sh 38/32238/3
Emoly Liu [Wed, 2 May 2018 05:57:19 +0000 (13:57 +0800)]
LU-10979 test: correct the version code check in sanity-sec.sh

The following patch was landed to 2.11.50 instead of 2.10.59, so
the version code check in sanity-sec.sh test_27a/b should be
corrected.
- Lustre-commit: 5b64d9fb0d5c5f292548c23e7841cc30f7a8423e
- Lustre-change: https://review.whamcloud.com/#/c/31450

Test-Parameters: trivial testlist=sanity-sec envdefinitions=ONLY=27 \
serverjob=lustre-b2_11 serverbuildno=2

Change-Id: I641ada8fb75d6347873a86ef453ebafd029876af
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-on: https://review.whamcloud.com/32238
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
5 years agoLU-9325 libcfs: handle complex strings in cfs_str2num_check 17/32217/3
James Simmons [Tue, 8 May 2018 13:30:15 +0000 (09:30 -0400)]
LU-9325 libcfs: handle complex strings in cfs_str2num_check

Originally the function cfs_str2num_check used simple_strtoul
but has been updated to kstrtoul. The string passed into
cfs_str2num_check can be a very complex, for example we could
have 10.37.202.[59-61]. When simple_strtoul was used the first
number until we hit a non-digit character could be extracted
but testing showed that kstrtoul will not return any value if
it detects any non-digit character. Because of this change in
behavior a different approach is needed to handle these types
of complex strings. The use of sscanf was investigated to see
if it could be used to extract numbers from the passed in
string but unlike its glibc counterpart the kernel version
also just reported a error with no results if a non-digit value
in the string was encountered. Another possible approach would
be to use __parse_int directly but that class of functions is
not exported by the kernel. So the approach in this patch is
to scan the string passed in for the first non-digit character
and replace that character with a '\0' so kstrtoul can be used.
Once completed the original character is restored. We also
restore a original behavior that was removed to return 0 when
we encounter any non digit character before the nob count.

Linux-commit : 3ad6152d766039cb8ffd8633d971fb79402e5464

Change-Id: If73d605499e2f05224a14417b0029036db38c8ba
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32217
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10907 tests: improve nodemap_test_setup() 18/32018/5
Emoly Liu [Tue, 17 Apr 2018 03:08:04 +0000 (11:08 +0800)]
LU-10907 tests: improve nodemap_test_setup()

Do wait_nm_sync() after modifying default.admin_nodemap, otherwise
it might cause the latter sync to fail.

Also, this patch checks the number of clients for sanity-sec.sh
test_23a, 2 clients at least for nodemap c0 and c1.

Test-Parameters: trivial testlist=sanity-sec

Change-Id: I7c7d5929b2f2b5aaaa0e7e884fa296fdf3cc1b57
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-on: https://review.whamcloud.com/32018
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Saurabh Tandan <saurabh.tandan@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10855 llog: remove llog_cancel() 95/31795/5
John L. Hammond [Tue, 27 Mar 2018 17:17:10 +0000 (12:17 -0500)]
LU-10855 llog: remove llog_cancel()

Remove llog_cancel() and replace its sole use with a direct call to
llog_changelog_cancel(). Simplify the prototype of
llog_changelog_cancel() according to its use.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I32f5adf34ca10b918e5860e03ef0d8028c941697
Reviewed-on: https://review.whamcloud.com/31795
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10699 hsm: add local object storage to MDTs 58/31758/2
John L. Hammond [Fri, 23 Mar 2018 17:00:51 +0000 (12:00 -0500)]
LU-10699 hsm: add local object storage to MDTs

Add local object storage (LOS) setup and cleanup to MDTs. LOS will be
used for the HSM actions index.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I1f9590350beeb49ebf14bfc602923c4cc8f6a15f
Reviewed-on: https://review.whamcloud.com/31758
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-9325 obdclass: handle strings correctly in lmd_find_delimiter 58/26558/7
James Simmons [Tue, 1 May 2018 15:42:28 +0000 (11:42 -0400)]
LU-9325 obdclass: handle strings correctly in lmd_find_delimiter

The feedback from the pushing of the lmd_find_delimiter() work
upstream was not to directly parse the string data passed in.

Change the return value to a bool and return true when the
character is found. Currently even though this function is
named lmd_find_delimiter() it return 1 when nothing is found
which is counter intuitive. Use strcspn() to determine the
position where the first delimiter is found. Add a test to
make sure the mount string is valid.

Change-Id: I7ef53f29a6d3284acd01225c115712d3674e8435
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/26558
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-9230 ldlm: speed up preparation for list of lock cancel 27/26327/31
Yang Sheng [Mon, 25 Sep 2017 13:01:02 +0000 (21:01 +0800)]
LU-9230 ldlm: speed up preparation for list of lock cancel

Keep the skipped locks in lru list will cause serious
contention for ns_lock. Since we have to travel them
every time in the ldlm_prepare_lru_list(). So we will
use a cursor to record position that last accessed
lock in lru list.

Change-Id: Ibda36a90e54076cb785a65910b34300639b3e140
Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Signed-off-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-on: https://review.whamcloud.com/26327
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10898 tests: enable conf-sanity 32a/32d 91/31991/7
Nathaniel Clark [Fri, 13 Apr 2018 16:27:22 +0000 (12:27 -0400)]
LU-10898 tests: enable conf-sanity 32a/32d

Stop zed service because it hold zfs module open.
Export zpools before trying to rmmod zfs.

Test-Parameters: trivial testlist=conf-sanity
Test-Parameters: trivial testlist=conf-sanity mdtfilesystemtype=zfs ostfilesystemtype=zfs
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Id1b891cadb91d9e3631a2d067c9d76a2965c34ff
Reviewed-on: https://review.whamcloud.com/31991
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10843 mgs: allow snapshot after MGS remount 79/31779/9
Nathaniel Clark [Mon, 16 Apr 2018 21:10:49 +0000 (17:10 -0400)]
LU-10843 mgs: allow snapshot after MGS remount

If the MGS is unmounted/mounted without the MDS reconnecting,
the fsdb FSNAME-barrier would not be created.

This change allows mgs_barrier_freeze (called from snapshot_create)
to attempt the create the required fsdb.

This adds a test to sanity-lsnapshot.sh for this issue.

Test-Parameters: testlist=sanity-lsnapshot mgtfilesystemtype=zfs ostfilesystemtype=zfs mdtfilesystemtype=zfs combinedmdsmgs=false ostcount=2 standalonemgs=true
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I2432cc0bdaddb07f024744065ca2ced77288fd7b
Reviewed-on: https://review.whamcloud.com/31779
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10472 osd-ldiskfs: add T10PI support for BIO 92/30792/7
Li Xi [Mon, 8 Jan 2018 01:09:46 +0000 (10:09 +0900)]
LU-10472 osd-ldiskfs: add T10PI support for BIO

This patch enables data integrity check in osd-ldiskfs when submiting
bio.

A fault injection mechanism is added to make sure the data integrity
check works well. On a OST with T10PI feature enabled, following
results are expected:

$ lctl set_param fail_loc=0x243
fail_loc=0x243
$ dd if=/dev/zero of=/mnt/lustre/file bs=1048576 count=100
dd: error writing '/mnt/lustre/file': Invalid or incomplete
multibyte or wide character
34+0 records in
33+0 records out
34603008 bytes (35 MB) copied, 0.510675 s, 67.8 MB/s

When doing fault injection, the write operation will wait until the
value is returned from BIO. Otherwise, returned error number may not
be returned to the application.

This implies a problem: because of the async submit of BIO, even the
OST has T10PI enabled, the application might not be able get error
notification when data corruption happens. However, there is nothing
we can do to improve this (unless write performance is not important),
because async commit is essential for good performance.

Change-Id: I76cc14b42feed835158100d35f65aedae0d79a5c
Signed-off-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/30792
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
5 years agoNew tag 2.11.52 2.11.52 v2_11_52 v2_11_52_0
Oleg Drokin [Wed, 16 May 2018 18:55:53 +0000 (14:55 -0400)]
New tag 2.11.52

Change-Id: Ie2c8d2a2d9d655f0646324780b8e02f0d031c4ba
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10805 libcfs: timer_setup() API changes 90/31790/4
Li Dongyang [Sun, 6 May 2018 14:28:58 +0000 (10:28 -0400)]
LU-10805 libcfs: timer_setup() API changes

Linux kernel 4.15 replaced setup_timer() with the new
interface timer_setup().
Introduce cfs wrappers to handle the API changes.

Linux-commit: e99e88a9d2b067465adaa9c111ada99a041bef9a

Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: Ib79495f9ab7e955d6f72f1e9390cec0e23e2d641
Reviewed-on: https://review.whamcloud.com/31790
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10560 llite: remove assigning f_version in ll_readir. 68/32268/2
Yang Sheng [Thu, 3 May 2018 06:04:40 +0000 (14:04 +0800)]
LU-10560 llite: remove assigning f_version in ll_readir.

Backport upstream patch:
 commit: 8ec426c7019ed9600d9dc0cf758445adcdbfc14e
         lustre: don't set f_version in ll_readdir

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I296ce2cf8a7f4f91cf051f281206a7af60b7fb38
Reviewed-on: https://review.whamcloud.com/32268
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10988 lfsck: load object attr when prepare LFSCK request 45/32245/4
Fan Yong [Wed, 2 May 2018 16:00:40 +0000 (00:00 +0800)]
LU-10988 lfsck: load object attr when prepare LFSCK request

It will avoid empty attrs in LFSCK request (lfsck_namespace_req).
The patch also shows invalid mode for dt_mode_to_dft() for debug.

Other cleanup for lfsck_namespace_striped_dir_rescan().

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I2874160d9a0c9e3084d0d3d7f365940108c82018
Reviewed-on: https://review.whamcloud.com/32245
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10944 kernel: kernel update [SLES12 SP3 4.4.126-94.22] 39/32139/3
Bob Glossman [Mon, 23 Apr 2018 22:25:31 +0000 (15:25 -0700)]
LU-10944 kernel: kernel update [SLES12 SP3 4.4.126-94.22]

Update target, kernel_config, and ldiskfs files for new version
ldiskfs for sles12sp2 and sles12sp3 are no longer identical,
so revise some build files for sles12sp2 here too.

Test-Parameters: clientdistro=sles12sp3 testgroup=review-ldiskfs \
  mdsdistro=sles12sp3 ossdistro=sles12sp3 \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: If554b24644f763462002d81309f8edfa016210e8
Reviewed-on: https://review.whamcloud.com/32139
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10924 tests: conf-sanity test_116 server version check 50/32050/2
Artem Blagodarenko [Wed, 18 Apr 2018 15:52:49 +0000 (18:52 +0300)]
LU-10924 tests: conf-sanity test_116 server version check

Patch "LU-10520 mkfs: enable extents for big MDT" adds test
that check if extents are enabled with 64bit option. But
Lustre FS before this patch sets ^extents opiton. So interop
testing fails.

This patch adds version check for MDT to conf-sanity test_116.

Signed-off-by: Artem Blagodarenko <artem.blagodarenko@gmail.com>
Cray-bug-id: LUS-5812
Change-Id: Ia42597174944ec1627dfe0f6a7cf8149e674cecc
Reviewed-on: https://review.whamcloud.com/32050
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10560 osd: bi_bdev is replaced by bi_disk 75/31975/6
Yang Sheng [Wed, 11 Apr 2018 08:15:20 +0000 (16:15 +0800)]
LU-10560 osd: bi_bdev is replaced by bi_disk

The member bi_bdev is replaced by bi_disk in struct bio.
 - upstream commit: 74d46992e0d9dee7f1f376de0d56d31614c8a17a
   block:  replace bi_bdev with a gendisk pointer and partitions index

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Ice1fb53f8371fb744af5dbac6c076ce817770213
Reviewed-on: https://review.whamcloud.com/31975
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
5 years agoLU-10857 tests: allow to disable project quotas 96/31796/5
Elena Gryaznova [Tue, 27 Mar 2018 17:59:55 +0000 (20:59 +0300)]
LU-10857 tests: allow to disable project quotas

Project quotas is disabled by default, sanity-quota.sh
enables it always if project quota supported.
Patch adds the possibility to disable project quotas
for regular user/group quota testing by using
ENABLE_PROJECT_QUOTAS=false.

Test-Parameters: trivial testlist=sanity-quota envdefinitions=ENABLE_PROJECT_QUOTAS=false
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Change-Id: I48176898fd940d66d0ebee4ef085a0bcece02ee9
Reviewed-on: https://review.whamcloud.com/31796
Tested-by: Jenkins
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-4423 ptlrpc: use workqueue for pinger 28/31728/3
Dmitry Eremin [Thu, 22 Mar 2018 17:02:08 +0000 (20:02 +0300)]
LU-4423 ptlrpc: use workqueue for pinger

lustre has a "pinger" kthread which periodically pings peers to ensure
all hosts are functioning.

This can more easily be done using a work queue.

The SVC_EVENT functionality to wake up the thread can be replaced with
mod_delayed_work().

Change-Id: I6064e9baea186323ab2eb0cca61ed23fcf8b55f7
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/31728
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-8066 mdc: move mdc-specific procfs files to sysfs 61/30961/6
Oleg Drokin [Thu, 12 Apr 2018 02:15:34 +0000 (22:15 -0400)]
LU-8066 mdc: move mdc-specific procfs files to sysfs

This moves max_rpcs_in_flight and max_pages_per_rpc to
/sys/fs/lustre/mdc/.../

Linux-commit : 2ee26222d497cf75eb9a02f324dee8b6b16e1e67

Add better error handling.

Change-Id: If6b50d2a3ef26325df0018ffb990a74ce2b75c10
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/30961
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Sonia Sharma <sonia.sharma@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-8066 fld: move all files from procfs to debugfs 51/26651/14
James Simmons [Wed, 18 Apr 2018 02:45:34 +0000 (22:45 -0400)]
LU-8066 fld: move all files from procfs to debugfs

Port of the upstream client work done by Dmitry Eremin.

Linux-commit: 827650494fbe9390052502d0498c8b90b2e329ec

Move all server side procfs handling to debugfs.

Change-Id: I6b0311d148576dec1ab1ef7ca1afce0ab2e4ed3b
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: Dmitry Eremin <dmiter4ever@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/26651
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-9857 lmv: dir page is released while in use 80/32180/3
Lai Siyao [Fri, 27 Apr 2018 16:11:55 +0000 (00:11 +0800)]
LU-9857 lmv: dir page is released while in use

When popping stripe dirent, if it reaches page end,
stripe_dirent_next() releases current page and then reads next one,
but current dirent is still in use, as will cause wrong values used,
and trigger assertion.

This patch changes to not read next page upon reaching end, but
leave it to next dirent read.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I28865d5719dfe95c0efba2f5543661a07aca6bd2
Reviewed-on: https://review.whamcloud.com/32180
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-7763 tests: improve sanityn 33d 46/31946/3
Lai Siyao [Wed, 11 Apr 2018 08:40:12 +0000 (16:40 +0800)]
LU-7763 tests: improve sanityn 33d

Sanityn.sh 33d verifies that CoS (Commit-on-Sharing) for DNE, but if
transaction commits before unlock, CoS will not be triggered. To
avoid this, do all tests twice, it's unlikely that both will fail.

Test-Parameters: trivial testlist=sanityn mdscount=2 mdtcount=4
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I590327111f7be2df233568ce408ee56a5af1d73a
Reviewed-on: https://review.whamcloud.com/31946
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10992 llite: decrease sa_running if fail to start statahead 87/32287/2
Fan Yong [Fri, 4 May 2018 02:18:26 +0000 (10:18 +0800)]
LU-10992 llite: decrease sa_running if fail to start statahead

Otherwise the counter of ll_sb_info::ll_sa_running will leak as
to the umount process will be blocked for ever.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I6b3a0dfb39f120a0c58184a22211b52376b41bd9
Reviewed-on: https://review.whamcloud.com/32287
Tested-by: Jenkins
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
5 years agoLU-10900 osd: wrong assertion in osd_transfer_project 47/31947/3
Li Xi [Wed, 11 Apr 2018 10:10:38 +0000 (06:10 -0400)]
LU-10900 osd: wrong assertion in osd_transfer_project

When project ID feature is not enabled on ldiskfs, the project
ID of any inode should be zero. osd_transfer_project() made
the opposite assertion.

Fixes: LU-10565 osd: move ext4_tranfer_project to osd
Test-Parameters: trivial testlist=sanity-quota testgroup=review-ldiskfs \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I8c065b9453e0e2b3f9f26e39fc82e8e73902df91
Reviewed-on: https://review.whamcloud.com/31947
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-1644 ptlrpc: fix return type of boolean functions 88/32088/4
Andreas Dilger [Thu, 19 Apr 2018 18:03:18 +0000 (12:03 -0600)]
LU-1644 ptlrpc: fix return type of boolean functions

Some functions are returning type int with values 0 or 1 when
they could be returning bool.  Fix up the return type of:

    lustre_req_swabbed()
    lustre_rep_swabbed()
    ptlrpc_req_need_swab()
    ptlrpc_rep_need_swab()
    ptlrpc_buf_need_swab()

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I758afbf2d50ee9c0930d3080016803a299ecab07
Reviewed-on: https://review.whamcloud.com/32088
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10175 ldlm: IBITS lock convert instead of cancel 02/30202/15
Mikhal Pershin [Fri, 3 Nov 2017 07:08:19 +0000 (10:08 +0300)]
LU-10175 ldlm: IBITS lock convert instead of cancel

For IBITS lock it is possible to drop just conflicting
bits and keep lock itself instead of cancelling it.
Lock convert is only bits downgrade on client and then
on server.
Patch implements lock convert during blocking AST.

Test-Parameters: mdssizegb=20 testlist=sanity-dom,dom-performance,racer,racer,racer
Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: Id073f14ec3283479a9a48fd80090ecd3b0967b6d
Reviewed-on: https://review.whamcloud.com/30202
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10985 mdt: properly handle unknown intent requests 37/32237/3
Oleg Drokin [Wed, 2 May 2018 04:48:46 +0000 (00:48 -0400)]
LU-10985 mdt: properly handle unknown intent requests

Invalid intent requests should be rejected early on,
so the later code does not make any assumptions about
validity of various structures like extended pills
and such.

Change-Id: I74de899ae6cea7017ae8ca9a8c488ca801a38c5d
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/32237
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
5 years agoLU-10575 build: make lustre-test dpkg depend on mpi-default-dev 79/32179/6
James Simmons [Mon, 30 Apr 2018 17:41:35 +0000 (13:41 -0400)]
LU-10575 build: make lustre-test dpkg depend on mpi-default-dev

Currently Ubuntu testing fails to run MPI based applications in
the lustre test suite. This is due improper dpkg setup. The
solution is to make lustre-test to depend on mp-default-dev
meta-package which will pull MPICH or OpenMPI depending on the
platform.

Test-Parameters: trivial testlist=performance-sanity clientdistro=ubuntu1604

Change-Id: I6b162d4d7b2485289d3e1c4dc23bcd6263fee682
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32179
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Wei Liu <wei3.liu@intel.com>
Reviewed-by: Thomas Stibor <t.stibor@gsi.de>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10413 ldlm: expose dirty age limit for flush-on-glimpse 13/32113/2
Mikhail Pershin [Mon, 23 Apr 2018 07:48:53 +0000 (10:48 +0300)]
LU-10413 ldlm: expose dirty age limit for flush-on-glimpse

Glimpse request may cancel old lock and cause data flush.
That helps to cache stat results on client locally early.
The time limit was hardcoded to 10s and is exposed now
as ns_dirty_age_limit namespace value, it can be set/check
via /sys/fs/lustre/ldlm/namespaces/<namespace>/dirty_age_limit

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Ie5b74eeb1720d552ab3f667e38af3925855f40ac
Reviewed-on: https://review.whamcloud.com/32113
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-8837 obd: remove linux directory 00/32100/2
James Simmons [Fri, 20 Apr 2018 15:28:25 +0000 (11:28 -0400)]
LU-8837 obd: remove linux directory

With only two files in obdclass/linux we can move the code around.
Rename linux-obdo.c to obdo_server.c since its only server code.
Move the ioctl handling in linux-module.c to class_obd.c and
rename linux-module.c to obd_sysfs.c since it only contains sysfs/
debugfs code.

Test-Parameters: trivial

Change-Id: Ie4b99cf37f83158f9d43eb38c971c1602f805c6f
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32100
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10922 osd-zfs: return -ENOENT if object destoryed 42/32042/3
Fan Yong [Sat, 21 Apr 2018 04:08:47 +0000 (12:08 +0800)]
LU-10922 osd-zfs: return -ENOENT if object destoryed

Originally, if the target object does not exist or just destroyed,
the osd_xattr_get_internal() will return -ENOENT to the caller, it
is right. But one of the callers: osd_xattr_get(), converts it as
-ENODATA before returning to the upper layer by wrong.

This patch fixes it.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I9bfd03c53dda165af37a4ae62b4efbc5875833e5
Reviewed-on: https://review.whamcloud.com/32042
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10902 mdd: force disable changelogs early and safely 07/32007/6
Bruno Faccini [Mon, 16 Apr 2018 13:31:28 +0000 (15:31 +0200)]
LU-10902 mdd: force disable changelogs early and safely

In mdd_changelog_user_purge() changelogs were disabled too late
allowing for new records to be unnecessarily gathered.
Also, a race was possible, causing changelogs to be disabled even
with registered users.

sanity/tests_160h has been added to check race is fixed.
Also, changelog_[unregister,clear]() functions in test-framework.sh
had to be modified to handle facets in same order than
changelog_register() to allow the race simulation to work in
DNE environment.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I78f07068e89b1cf2cf0c83dd2068f82f8a4a6dd5
Reviewed-on: https://review.whamcloud.com/32007
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10850 fld: handle empty ranges in fld_index_init() 71/31871/2
John L. Hammond [Wed, 4 Apr 2018 15:13:16 +0000 (10:13 -0500)]
LU-10850 fld: handle empty ranges in fld_index_init()

In fld_index_init(), ignore the empty range corresponding to the
zeroed-out phantom record that is included in newly created ldiskfs
IAM indexes. (This phantom record will be removed in a subseuent
patch.) Ensure that lfs_new will be set when the FLD index was empty
of valid ranges. (Previously it was properly set on ZFS backed targets
but not on ldiskfs.)

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I9850eac686e1488ffbd627f100075ce6f29200d8
Reviewed-on: https://review.whamcloud.com/31871
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10856 tests: remove deprecated lfs getstripe -M usage 08/31808/5
Andreas Dilger [Wed, 28 Mar 2018 06:38:34 +0000 (00:38 -0600)]
LU-10856 tests: remove deprecated lfs getstripe -M usage

The "lfs getstripe -M" and "lfs mv -M" options are deprecated
since 2.9 and should not be used in the tests.

Clean up some of the nearby tests to match current coding style:
- lower-case local variable names
- tabs for indentation
- replace $GETSTRIPE/$SETSTRIPE with "$LFS getstripe/setstripe"

Test-Parameters: trivial testlist=replay-dual,sanity-flr
Test-Parameters: testlist=recovery-small,replay-single mdtcount=2
Test-Parameters: testlist=testlist=sanity,sanityn,sanity-hsm mdtcount=2
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Id86395a03379bef2de24dfa92642c928f51cab07
Reviewed-on: https://review.whamcloud.com/31808
Tested-by: Jenkins
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Elena Gryaznova <c17455@cray.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10805 ldlm: convert ACCESS_ONCE() to READ_ONCE() 89/31789/3
Li Dongyang [Mon, 30 Apr 2018 19:06:51 +0000 (15:06 -0400)]
LU-10805 ldlm: convert ACCESS_ONCE() to READ_ONCE()

Linux kernel 4.15 removed ACCESS_ONCE, use READ_ONCE instead.

Linux-commit: b899a850431e2dd0943205a63a68573f3e312d0d

Test-Parameters: trivial
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: I139e4cab95b99f362a85c37efe236137835f9c54
Reviewed-on: https://review.whamcloud.com/31789
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
5 years agoLU-10838 tests: replay-single/89 to wait for recovery 29/31729/2
Alex Zhuravlev [Thu, 22 Mar 2018 17:48:06 +0000 (20:48 +0300)]
LU-10838 tests: replay-single/89 to wait for recovery

with a single client (e.g. local setup) test 89 should wait
for recovery completion. client_up (i.e. lfs df) is not enough
as the new client's OSC will be skipped being not active.

Change-Id: I859edfc5a38cbac44377c7660fabb87db55978e9
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/31729
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10624 tests: conf-sanity needs larger ZFS filesystems 92/31192/4
Alex Zhuravlev [Wed, 7 Feb 2018 04:22:13 +0000 (07:22 +0300)]
LU-10624 tests: conf-sanity needs larger ZFS filesystems

otherwise some tests can take a lot (like ~2000 seconds
for test 106) due to very limited space and short
transactions. with the patch 106 passes in ~440 seconds.

Change-Id: I0597072cf72315411b7e656b1e87a0fceabb171f
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/31192
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
5 years agoLU-8066 lov: Move suitable variables from procfs to sysfs 60/30960/6
Oleg Drokin [Thu, 12 Apr 2018 02:11:53 +0000 (22:11 -0400)]
LU-8066 lov: Move suitable variables from procfs to sysfs

Migrate procfs files with simple output to sysfs. Create
sysfs symlinks for target_obds.

Linux-commit : cb1debff6d8b254536ec6ff642a2bdd22c86cb85

Change-Id: I97e403694fd1aab9f70363a2961beb08c6959447
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/30960
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-8066 lmv: Move suitable entries from procfs to sysfs 60/28460/13
Oleg Drokin [Thu, 5 Apr 2018 20:01:43 +0000 (16:01 -0400)]
LU-8066 lmv: Move suitable entries from procfs to sysfs

Linux-commit: b5fa70d76fb1c532dd4468b8083e2945c9df9ddb

Change-Id: I89ffcf5a8efe870a3bd02462681b24952a788dab
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/28460
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-8293 lnet: Add insserv header to lnet init script 35/20835/7
Chris Horn [Thu, 16 Jun 2016 17:57:37 +0000 (12:57 -0500)]
LU-8293 lnet: Add insserv header to lnet init script

The lnet init script does not contain header information as described
by the insserv man page. This header information is needed to ensure
the lnet init script is not run until openibd has been able to start

Signed-off-by: Chris Horn <hornc@cray.com>
Change-Id: I6f778e827f88ce34199dff70be5d5089f0ba51b9
Reviewed-on: https://review.whamcloud.com/20835
Tested-by: Jenkins
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10932 libcfs: properly handle failure cases in SMP code 85/32085/2
James Simmons [Thu, 19 Apr 2018 17:06:53 +0000 (13:06 -0400)]
LU-10932 libcfs: properly handle failure cases in SMP code

While pushing the LU-8703 and LU-7734 work upstream some bugs were
pointed out by Dan Carpenter in the code. Due to single err label
in cfs_cpt_table_alloc() and cfs_cpu_init() a few items were being
cleaned up that were never initialized. This can lead to crashed
and other problems. In those initialization function introduce
individual labels to jump to only the thing initialized get freed
on failure. Lastly in cfs_cpt_table_alloc() handle the failure
case instead of the passed case which is the perferred style.

Change-Id: I969f4e327888042a517bc321b68d55fc691b074e
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/32085
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-6349 mds: remove obsolete MDS_VTX_BYPASS flag 84/31984/3
Andreas Dilger [Fri, 13 Apr 2018 03:44:52 +0000 (21:44 -0600)]
LU-6349 mds: remove obsolete MDS_VTX_BYPASS flag

The MDS_VTX_BYPASS flag is only set and never checked.  This is true
since 2.3.53-66-g54fe979 "LU-2216 mdt: remove obsolete DNE code", but
it was already obsolete for a long time before that.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ia9510146326e71a134fef85dd8febf7b752cab07
Reviewed-on: https://review.whamcloud.com/31984
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-4684 ptlrpc: add dir migration connect flag 14/31914/6
Lai Siyao [Tue, 3 Apr 2018 02:55:18 +0000 (10:55 +0800)]
LU-4684 ptlrpc: add dir migration connect flag

Add dir migration connect flag to prevent collision with other
features. Though dir migration code exists, it will be reworked,
and the new RPC protocol won't be compatible with current one.

Also handle the previously-added OBD_CONNECT2_FLR flag.

Test-Parameters: trivial
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Ie3eaec8602664170b52e3778b76059a0899ee7b1
Reviewed-on: https://review.whamcloud.com/31914
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
5 years agoLU-9019 libcfs: remove the remaining cfs_time wrappers 68/31068/8
James Simmons [Thu, 12 Apr 2018 14:00:11 +0000 (10:00 -0400)]
LU-9019 libcfs: remove the remaining cfs_time wrappers

In the lustre code various small bits are left of the libcfs time
wrappers. Remove all the remaining wrappers except for the inline
function cfs_time_seconds(). For cfs_time_seconds() we have to
move to nsec_to_jiffies() since msecs_to_jiffies() has become an
inline function in jiffies.h which means HZ can be different on
the installation node verses what the target node is configured
for. Since nsec_to_jiffies() is a normal built in kernel function
we can avoid the LU-5443 issues. For cfs_duration_secs() we use
the internal kernel function jiffies_to_msec() since only the
newest kernels have jiffies_to_nsec() and that function is just
a wrapper around jiffies_to_msec(). All the reset of the libcfs
time abstractions are gone.

Change-Id: I166973071304f1d55a15b1e21fcfbe434ff58199
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/31068
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-9780 tests: Testing Round-Robin allocation 75/28075/5
Ashish Maurya [Fri, 16 Feb 2018 17:38:00 +0000 (20:38 +0300)]
LU-9780 tests: Testing Round-Robin allocation

Adding a test for fix in LU-977 which shows that
in absence of protection to lqr_start_idx there is
possibility of imblance in allocating objects on OSTs
with round-robin algorithm.

This test checks the protection of lqr_start_idx by using
a new reproducer, rr_alloc which uses MPI to create files in
parallel, and checking the even file distribution over OSTs.

Distribution check formula is adjusted as per implementation,i.e
some factors like - exhaustion of pre-created objects and counter
‘lrq_start_count’(lod_qos.c) reseed are also taken care so that
object allocation is not affected by these factors.

Test-Parameters: trivial osscount=3 clientcount=6                  envdefinitions=ONLY=rr_alloc testlist=parallel-scale
Signed-off-by: Ashish Maurya <ashish.maurya@seagate.com>
Signed-off-by: Rahul Deshmukh <rahul.deshmukh@seagate.com>
Signed-off-by: Elena Gryaznova <c17455@cray.com>
Cray-bug-id: MRP-2723
Reviewed-by: Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Vikram Jadhav <jadhav.vikram@seagate.com>
Change-Id: I55f798c6dc8e607f002365f4a22ccf59a454fe1d
Reviewed-on: https://review.whamcloud.com/28075
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-3665 tests: Cleanup echo client after obdfilter-survey 50/9350/25
Nathaniel Clark [Fri, 21 Feb 2014 17:53:36 +0000 (12:53 -0500)]
LU-3665 tests: Cleanup echo client after obdfilter-survey

Some failures of obdfilter-survey do not cause an error in
obdfilter-survey.sh In some cases obdfilter-survey did not
cleanup echo clients it had created and that could hang
umount of osts.
Change test-framework.sh::cleanupall to remove echo clients before
trying to umount to prevent the echo clients from holding the OST or
MDS/MGS open forever.

Test-Parameters: trivial testlist=obdfilter-survey osscount=1 ostcount=2 mdscount=1 mdtcount=1
Test-Parameters: trivial testlist=obdfilter-survey mdtfilesystemtype=zfs ostfilesystemtype=zfs osscount=1 ostcount=2 mdscount=1 mdtcount=1
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I63ae59da84101c782aa9d5e7216cce3b3b1ff2fe
Reviewed-on: https://review.whamcloud.com/9350
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-6655 ptlrpc: skip delayed replay requests 05/23205/15
Hongchao Zhang [Mon, 23 Apr 2018 06:35:11 +0000 (14:35 +0800)]
LU-6655 ptlrpc: skip delayed replay requests

During recovery, there could be some delayed replay requests
after the final recovery completion ping request was handled,
and it should be skipped.

Change-Id: Ie0d5ff92c75f9d078b8ae28e899d4a821113194f
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: https://review.whamcloud.com/23205
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-1644 mgc: remove obsolete IR swabbing workaround 87/32087/3
Andreas Dilger [Tue, 17 Apr 2018 04:53:02 +0000 (22:53 -0600)]
LU-1644 mgc: remove obsolete IR swabbing workaround

The OBD_CONNECT_MNE_SWAB check was added to the MGC for compatibility
with servers in the 2.2.0-2.2.55 range (in 2012) with big-endian
clients.  2.2 was not an LTS release and is no longer being used.

Remove the checks on the client for OBD_CONNECT_MNE_SWAB being set,
and assume that the server does not have this bug.  This will allow
the removal of the rest of this workaround from the server code once
there are no more clients depending on the presence of this flag.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ie128745e3af8e9d41454e095922e0f6bc8fcab07
Reviewed-on: https://review.whamcloud.com/32087
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10910 mdd: deny layout swap for DoM file 44/32044/5
Mikhail Pershin [Wed, 18 Apr 2018 07:17:10 +0000 (10:17 +0300)]
LU-10910 mdd: deny layout swap for DoM file

Layout swap is prohibited for DoM files until LU-10177
will be implemented. The only exception is the new layout
having the same DoM component.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I8b9deaea7aab0d5694a4c5d9fe2f9f36d2cdd382
Reviewed-on: https://review.whamcloud.com/32044
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10909 utils: enable non-shared libmount_utils_ldiskfs.a 70/31970/5
Alex Zhuravlev [Thu, 12 Apr 2018 08:32:32 +0000 (11:32 +0300)]
LU-10909 utils: enable non-shared libmount_utils_ldiskfs.a

It depends on libcfs.a (due to cfs_abs_path()) so that library must
come after it in the list of libraries provided to the linker.

Test-Parameters: trivial
Change-Id: Id3026705ebac65b74d1ea48964c25aa6c4834552
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-on: https://review.whamcloud.com/31970
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-930 doc: include status in "lfs df" man page 98/31998/3
Andreas Dilger [Fri, 13 Apr 2018 21:30:47 +0000 (15:30 -0600)]
LU-930 doc: include status in "lfs df" man page

Describe the "DRSI" target states in the lfs-df.1 man page.

Improve the overall description of how "lfs df [-i]" works, with
estimation of total/free inodes for ZFS, and show actual output.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I2ae4190c9ebd3467b5243df89938501d301cab07
Reviewed-on: https://review.whamcloud.com/31998
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Joseph Gmitter <joseph.gmitter@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10903 obdecho: use OBD_ALLOC_LARGE for lnb 64/31964/2
Andreas Dilger [Wed, 11 Apr 2018 18:13:34 +0000 (12:13 -0600)]
LU-10903 obdecho: use OBD_ALLOC_LARGE for lnb

When allocating the niobuf_local, if there are a large number of
(potential) fragments this allocation can be quite large.  Use
OBD_ALLOC_LARGE(lnb) and OBD_FREE_LARGE(lnb) to avoid allocation
errors and console noise.  This was causing sanity test_180c to
fail in a VM on occasion, and could also be problem in real use.

Tidy up test_180[abc] to use stack_trap to handle echo device
cleanup rather than having a series of conditional checks.
Use "error" to report errors rather than error numbers.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ib0e721f58fc0a62acca29b31fb4dfac5021cab07
Reviewed-on: https://review.whamcloud.com/31964
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10886 build: fix warnings during autoconf 04/31904/2
Andreas Dilger [Thu, 26 Jun 2014 23:53:52 +0000 (17:53 -0600)]
LU-10886 build: fix warnings during autoconf

Quiet warnings in configure checks to avoid confusion when debugging
autoconf/configure scripts, and to avoid potential errors during
detection.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I2916a120b57fb9fa529bf7050cf65511233ebbe5
Reviewed-on: https://review.whamcloud.com/31904
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-8200 test: improve sanityn.sh 33c 73/31673/4
Lai Siyao [Fri, 16 Mar 2018 14:36:11 +0000 (22:36 +0800)]
LU-8200 test: improve sanityn.sh 33c

If transaction is committed before unlock, lock won't be saved. So
trigger Sync-Lock-Cancel twice in sanityn.sh 33c, it's unlikely to
fail both.

Test-Parameters: trivial clientcount=2 mdscount=2 mdtcount=4 \
osscount=1 ostcount=8 mdtfilesystemtype=zfs ostfilesystemtype=zfs \
testlist=sanityn
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I5c99e5a8261df8c9f463aea7ed67df95baaf3e6c
Reviewed-on: https://review.whamcloud.com/31673
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-10699 hsm: simplify mdt_hsm_{add,get}_actions() 85/31385/2
John L. Hammond [Thu, 22 Feb 2018 18:27:14 +0000 (12:27 -0600)]
LU-10699 hsm: simplify mdt_hsm_{add,get}_actions()

Split the mdt_hsm_get_actions() functionality of
hsm_find_compatible_cb() into a new llog callback
(hsm_find_action_cb()). Simplify hsm_find_compatible() since it only
needs to handle adding new requests. Simplify the MDS_HSM_ACTIONS
handling code.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ie1e7eac85397d82ad9cc4790582bf8ba51713e70
Reviewed-on: https://review.whamcloud.com/31385
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
5 years agoLU-8066 osc: move suitable values from procfs to sysfs 62/30962/6
Oleg Drokin [Thu, 12 Apr 2018 02:19:22 +0000 (22:19 -0400)]
LU-8066 osc: move suitable values from procfs to sysfs

All single-value controls are moved from /proc/fs/lustre/osc/.../
to /sys/fs/lustre/osc/.../

Linux-commit : aab38b00ac19347bf982cf42c71aab14a9301dee

Change-Id: I69744804bc424ab171da8f649336a3ad450f0d05
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/30962
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>