Whamcloud - gitweb
fs/lustre-release.git
6 months agoLU-11085 nodemap: switch interval tree to in-kernel impl. 24/39724/9
Mr NeilBrown [Tue, 25 Aug 2020 00:03:17 +0000 (10:03 +1000)]
LU-11085 nodemap: switch interval tree to in-kernel impl.

Switch nodemap_range to use the in-kernel interval tree.
This has the same functionality, though often in a different form.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I7bf119bf8cd8f14dc66deb2736c2c97562bb0743
Reviewed-on: https://review.whamcloud.com/39724
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-13485 libcfs: FIELD_SIZEOF macro removed 10/39710/6
Shaun Tancheff [Sun, 30 Aug 2020 18:53:09 +0000 (13:53 -0500)]
LU-13485 libcfs: FIELD_SIZEOF macro removed

Linux v4.15-rc2-5-g4229a470175b introduced sizeof_field() macro
Linux v5.5-rc4-1-g1f07dcc459d5 removed FIELD_SIZEOF() macro

Provide a sizeof_field() macro in terms of FIELD_SIZEOF()
when sizeof_field() is not provided.

Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I48ca9abb931d58919d788199e5089984c9e854dd
Reviewed-on: https://review.whamcloud.com/39710
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
6 months agoLU-12678 o2iblnd: convert peers hash table to hashtable.h 03/39303/4
Mr NeilBrown [Mon, 6 Jul 2020 12:34:40 +0000 (08:34 -0400)]
LU-12678 o2iblnd: convert peers hash table to hashtable.h

Using a hashtable.h hashtable, rather than bespoke code, has several
advantages:

 - the table is comprised of hlist_head, rather than list_head, so
   it consumes less memory (though we need to make it a little bigger
   as it must be a power-of-2)
 - there are existing macros for easily walking the whole table
 - it uses a "real" hash function rather than "mod a prime number".

In some ways, rhashtable might be even better, but it can change the
ordering of objects in the table are arbitrary moments, and that could
hurt the user-space API.  It also does not support the partitioned
walking that ksocknal_check_peer_timeouts() depends on.

Note that new peers are inserted at the top of a hash chain, rather
than appended at the end.  I don't think that should be a problem.

Also various white-space cleanups etc.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I2917024835abdd327c7da11dee3fd369570a9671
Reviewed-on: https://review.whamcloud.com/39303
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-12678 lnet: discard LNET_MD_PHYS 01/39301/2
Mr NeilBrown [Mon, 6 Jul 2020 12:34:42 +0000 (08:34 -0400)]
LU-12678 lnet: discard LNET_MD_PHYS

This macro has no value and is never set.
It claims "compatibility with Cray Portals", yet cray-dvs
   git://github.com/glennklockwood/cray-dvs.git
does not use it in any non-trivial way.

Much has changed in lnet and lib-md since 2007 when this
value was added - it seems likely that this really
is dead.

So remove it.  If/when this results in problems, it can
easily be re-added and more details can be provided at
that time.

Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Idef9389f4c0993adbdf088d0ccd9a0dc1449e86e
Reviewed-on: https://review.whamcloud.com/39301
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-12678 lnet: use init_wait() rather than init_waitqueue_entry() 95/39295/3
Mr NeilBrown [Mon, 6 Jul 2020 12:34:43 +0000 (08:34 -0400)]
LU-12678 lnet: use init_wait() rather than init_waitqueue_entry()

init_waitqueue_entry(foo, current)

is equivalent to

  init_wait(foo)

 So use the shorter version.

Change-Id: Ic63e99d75986211d9655a89f56721394c7b3abb6
Signed-off-by: Mr. NeilBrown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/39295
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-9859 libcfs: use wait_event_timeout() in tracefiled(). 93/39293/4
Mr NeilBrown [Mon, 6 Jul 2020 12:34:45 +0000 (08:34 -0400)]
LU-9859 libcfs: use wait_event_timeout() in tracefiled().

By using wait_event_timeout() we can make it more clear what is being
waited for, and when the loop terminates.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I5d015a3cbdbb342a5117e2c328680b3ec13aeb58
Reviewed-on: https://review.whamcloud.com/39293
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-12678 lnet: discard WIRE_ATTR 14/37914/5
James Simmons [Sat, 26 Dec 2020 17:19:19 +0000 (12:19 -0500)]
LU-12678 lnet: discard WIRE_ATTR

This macro adds nothing of value, and make the code harder to
read for new readers so it was remove for the Linux client.
We still want to keep track of what data structures are
transmitted over the wire and ensure the protocol does not get
broken. Move the wire protocol structures to their own header
files and add wire checking.

Linux-commit: 3e60455d1953fcee6042c107a8d5657886aa9c58

Test-Parameters: trivial
Change-Id: I4867e80bf8e8f0598d1920865d6f1b9ba920ce5b
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/37914
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-13239 ldiskfs: pass inode timestamps at initial creation 56/37556/11
Shaun Tancheff [Thu, 10 Dec 2020 16:31:51 +0000 (10:31 -0600)]
LU-13239 ldiskfs: pass inode timestamps at initial creation

A previous patch https://github.com/Cray/lustre/commit/6d4fb6694
"LUS-4880 osd-ldiskfs: pass uid/gid/xtime directly to ldiskfs"
was intended to be ported to upstream lustre but was lost.

The patch https://review.whamcloud.com/34685/
"LU-12151 osd-ldiskfs: pass owner down rather than transfer it"
passed the inode UID and GID down to ldiskfs at inode allocation
time to avoid the overhead of transferring quota from the inode
(initially created as root) over to the actual user of the file.

The two patches differed slightly in that the LUS-4880 included
passing the a/m/ctimes from osd-ldiskfs to ldiskfs at inode
creation time avoids overhead of setting the timestamps afterward.

Benchmarks using MDTEST:
  mdtest -f 32 -l 32 -n 16384 -i 5 -p 120 -t -u -v -d mdtest

                            master                 patched
   Operation                  Mean    Std Dev         Mean   Std Dev
   ---------                  ----    -------         ----   -------
   Directory creation:   17008.593     72.700    17099.863   155.461
   Directory stat    :  170513.269   1456.002   170105.207  2349.934
   Directory removal :   80796.147   2633.832    84480.222   892.536
   File creation     :   39227.419   7014.539    40429.900  6643.868
   File stat         :  101761.395   2979.802   103818.800  1146.689
   File read         :   86583.370    871.982    85725.254   965.862
   File removal      :   74923.504    761.048    75075.180   723.966
   Tree creation     :     588.570    244.534      608.332   123.939
   Tree removal      :      39.874      1.873       44.357     2.350

This patch also reorganizes the ldiskfs patch series in
order to accommodate struct iattr being added to
ldiskfs_create_inode.
All supported server platforms RHEL 7.5+, SUSE 12+ and
ubuntu 18+ are affected.

HPE-bug-id: LUS-7378, LUS-4880, LUS-8042, LUS-9157, LUS-8772, LUS-8769
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I87e9c792b5240820bfd3a7268e477970ebac8465
Reviewed-on: https://review.whamcloud.com/37556
Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-12780 ofd: don't use ptlrpc_thread for consistency verification 62/36262/13
Mr NeilBrown [Wed, 23 Oct 2019 00:30:50 +0000 (11:30 +1100)]
LU-12780 ofd: don't use ptlrpc_thread for consistency verification

The ofd module runs a consistency verification thread to verify parent
FID.  Rather than using ptlrpc_thread to manage this, use native
kthreads functionality.

- startup-up code is moved out of the thread to before the
  thread is started, which make error handling clearer.
  As part of this, the lfsck_req_local struct is combined with
  an lu_env and ofd_device pointer into a new oivm_args
  which is passed to the thread a arguments - now it doesn't need
  to allocate anything itself.
- Cleanup remains in the thread, so we add a completion to be
  sure the thread has started before there is any chance of
  kthread_stop() being called.

- kthread_stop() and kthread_should_stop() are used for stopping
  the thread.  wake_up_process() is used to wake it.
  The thread sets TASK_IDLE at the top of the loop, and sets
  TASK_RUNNING if anything is found to do.  At the bottom of
  the loop the 'schedule()' will only block if nothing was found
  to be done.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Iec1de307ea48f7d26c60edf5d86eb0b7bf78f49a
Reviewed-on: https://review.whamcloud.com/36262
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-11085 ldlm: change lock_matches() to return bool. 54/33854/12
Mr. NeilBrown [Thu, 19 Nov 2020 14:09:19 +0000 (09:09 -0500)]
LU-11085 ldlm: change lock_matches() to return bool.

The name of the function lock_matches() sounds like it
performs a test (it does) and so should return a bool.
Returning a bool gives a slight code simplification (in
search_queue) and more simplification in future patches.

Linux-commit: e16983d96c775eb4527208d3c3d13f57e6d6233c

Change-Id: I1e3a09a0768abd0ab1cfada0fd69216cb9e85df7
Signed-off-by: Mr. NeilBrown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/33854
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
6 months agoLU-6142 libcfs: discard cfs_strrstr() 61/40861/2
Mr NeilBrown [Tue, 24 Nov 2020 02:45:39 +0000 (13:45 +1100)]
LU-6142 libcfs: discard cfs_strrstr()

cfs_strrstr() is only used in one place, and it can easily be open
coded there without increasing code complexity.  In particular the
fact that the "needle" cannot meaningfully be at the start of the
"haystack", means a simple loop does all we need.

In fact, there is room to improve the code in lwp_setup()
 - sprintf isn't needed as the result is a constant that can
   be calculated at compile time
 - adding the nul termination is then not needed as the buffer
   being copied to was initialised to zeroes.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I52b4abb36cf809d3bd9eebcc752959b0a81bfc13
Reviewed-on: https://review.whamcloud.com/40861
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-6142 libcfs: discard cfs_firststr 60/40860/2
Mr NeilBrown [Tue, 24 Nov 2020 02:28:01 +0000 (13:28 +1100)]
LU-6142 libcfs: discard cfs_firststr

The effect of cfs_firststr() can easily achieved with
skip_space() and strsep().

So use that instead.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Idcf8aa50b6aad052f7ee5341ce6d635495aa4990
Reviewed-on: https://review.whamcloud.com/40860
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-6142 libcfs: discard PO2_ROUNDUP_TYPED, LOWEST_BIT_SET 59/40859/3
Mr NeilBrown [Fri, 20 Nov 2020 02:20:23 +0000 (13:20 +1100)]
LU-6142 libcfs: discard PO2_ROUNDUP_TYPED, LOWEST_BIT_SET

LOWEST_BIT_SET() is never used.

PO2_ROUNDUP_TYPED() has the same function as 'round_up()'.

osd_roundup2blocksz() can be further simplified using
DIV_ROUND_UP_ULL().

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I075328ec935eef49d3aeaf5ea1b79b943aadfa2e
Reviewed-on: https://review.whamcloud.com/40859
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
6 months agoLU-6142 llite: ll_lookup_finish_locks clean up 30/40830/3
Mr NeilBrown [Thu, 15 Oct 2020 04:10:50 +0000 (15:10 +1100)]
LU-6142 llite: ll_lookup_finish_locks clean up

The 'inode' var can be used rather than dereferencing dentry->d_inode
a second time.
Also re-indent the function.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I16fb5480f7e2a8192a82e0c821597e14aad00b40
Reviewed-on: https://review.whamcloud.com/40830
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-6142 llite: don't cast arg to d_lustre_invalid() 24/40824/2
Mr NeilBrown [Thu, 15 Oct 2020 03:32:02 +0000 (14:32 +1100)]
LU-6142 llite: don't cast arg to d_lustre_invalid()

There is no need to cast the arg to d_lustre_invalid() as it
is always of exactly the correct type.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I48b103abe2755a2d8892439974744dea232db515
Reviewed-on: https://review.whamcloud.com/40824
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
6 months agoLU-6142 lustre: use is_root_inode() 93/40293/7
Mr NeilBrown [Fri, 16 Oct 2020 03:29:32 +0000 (14:29 +1100)]
LU-6142 lustre: use is_root_inode()

Lustre has multiple tests to see if a given inode is the root of the
filesystem.  Linux has (since 3.19) a helper function is_root_inode().
Use that throughout.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ib5997e8976b6e935c268ad7ac9328fcd4d071b1f
Reviewed-on: https://review.whamcloud.com/40293
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
6 months agoLU-6142 llite: remove ll_dir_chain 90/40290/2
Mr NeilBrown [Thu, 15 Oct 2020 22:10:04 +0000 (09:10 +1100)]
LU-6142 llite: remove ll_dir_chain

ll_dir_chain is an empty structure that was introduced 13 years ago
and never used.  As such it is clutter.
Remove it an all references to it.  If the need which proposed it
ever does get addresses, having a clean slate in which to create
a suitable implementation is probably preferred.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I1eff80add8851ea11fff979f9b2d25916d6d3eed
Reviewed-on: https://review.whamcloud.com/40290
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-6142 lustre: remove non-static 'inline' markings. 89/40289/4
Mr NeilBrown [Thu, 15 Oct 2020 00:10:47 +0000 (11:10 +1100)]
LU-6142 lustre: remove non-static 'inline' markings.

There is rarely any point in marking a non-static function as
'inline'.  The result is to compile a state-alone function that other
files can refer to, and also to inline the code where it is used in
the same file.

In many cases the non-static inline functions are not used in the same
file, so the 'inline' marking has no effect.  In other cases it may
have an effect, but it can only be needed in highly performance
critical situations where a function call must be avoided, and that
doesn't seem like in any of these cases.

So just remove the "inline".

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ic3243ee80f9bfd75a67dd8c89ea07d08dc36425c
Reviewed-on: https://review.whamcloud.com/40289
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
6 months agoLU-6142 lustre: convert snprintf to scnprintf as appropriate 44/39744/2
Mr NeilBrown [Thu, 27 Aug 2020 06:52:13 +0000 (16:52 +1000)]
LU-6142 lustre: convert snprintf to scnprintf as appropriate

The return value of snprintf() is the number of bytes that would have
been copies into the buffer if it was large enough.
Many places in the code use it as though it were the number of bytes
actually copied.  In practice this (almost?) never makes a difference.
However it is poor style to use the wrong interfaces as it might one
day be copied to somewhere that it does make a difference.

So change these instances of snprintf to scnprintf which DOES return
the number of bytes actually copied.
This is all places where the return value is simply returned to the
call, and a couple of others.

Also change the declared buffer size in a couple of places to the
actual buffer size (PAGE_SIZE in these cases).

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I5428760689eaca98e6bf2bceae119f1b7acf35ac
Reviewed-on: https://review.whamcloud.com/39744
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-6142 lustre: mark strings in char arrays as const 42/39742/2
Mr NeilBrown [Thu, 27 Aug 2020 04:34:25 +0000 (14:34 +1000)]
LU-6142 lustre: mark strings in char arrays as const

Most array of strings are marked 'const', but the strings within the
arrays often aren't.
This patch mark all strings in const arrays of strings as const.
This allows them to be placed in read-only memory.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I93b627e646e446fc48d9b76f1d5e9da9d12b4395
Reviewed-on: https://review.whamcloud.com/39742
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-6142 mdc: minor function cleanups. 38/39738/2
Mr NeilBrown [Thu, 27 Aug 2020 03:28:42 +0000 (13:28 +1000)]
LU-6142 mdc: minor function cleanups.

mdc_getxattr_pack() doesn't exist, so remove declaration

mdc_get_lustre_md(), mdc_free_lustre_md(), and
mdc_clear_open_replay_data() are local to mdc_request.c so make them
static and remove declaration from mdc_interal.h.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ie925585f51f280a4ad4e3ae7b488dc4166fdaa8d
Reviewed-on: https://review.whamcloud.com/39738
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-6142 osc: minor function cleanups. 37/39737/2
Mr NeilBrown [Thu, 27 Aug 2020 01:36:06 +0000 (11:36 +1000)]
LU-6142 osc: minor function cleanups.

osc_cleanup() doesn't exist, so remove declaration.

Return value of osc_extent_release() is always zero and never used, so
change it to 'void'.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I048e4ec1ec62d2b576f2a1e3c6148c0121cd91b6
Reviewed-on: https://review.whamcloud.com/39737
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-6142 lustre: change various operations structs to const 00/39400/3
Mr NeilBrown [Thu, 16 Jul 2020 04:56:38 +0000 (14:56 +1000)]
LU-6142 lustre: change various operations structs to const

Nearly all of
  struct cl_io_operations
  struct cl_lock_operations
  struct iam_operations
  struct iam_leaf_operations
  struct llog_operations
  struct lquota_entry_operations
  struct md_dir_operations

are now const.  The one exception is changelog_orig_logops.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I91dc9528b62a77f6bcedc80d6066a74a8eeaa3fa
Reviewed-on: https://review.whamcloud.com/39400
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-6142 lfsck: make all 'struct lfsck_operations' to const 97/39397/3
Mr NeilBrown [Thu, 16 Jul 2020 04:07:56 +0000 (14:07 +1000)]
LU-6142 lfsck: make all 'struct lfsck_operations' to const

Operations structures should be 'const' wherever possible.
So change both lfsck_operations and lfsck_assistant_operations
to const.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ic457c0e52ceafab58cb788db383957ed4ab39df4
Reviewed-on: https://review.whamcloud.com/39397
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-6142 lustre: change all 'struct seq_operations' to const 96/39396/2
Mr NeilBrown [Thu, 16 Jul 2020 03:59:44 +0000 (13:59 +1000)]
LU-6142 lustre: change all 'struct seq_operations' to const

All 'struct seq_operations' are now 'static const'.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ia0f6a2863835ec02fb7d283a099eea489f65ab3a
Reviewed-on: https://review.whamcloud.com/39396
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-6142 lustre: change super/file/inode operations to const 94/39394/5
Mr NeilBrown [Thu, 16 Jul 2020 03:49:36 +0000 (13:49 +1000)]
LU-6142 lustre: change super/file/inode operations to const

All 'struct file_operations', 'struct inode_operations', 'struct
export_operations' and 'struct super_operations' are changed to
'const'.  This potenetially allows them to be placed in read-only
memory, and ensure they are never changed.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I8b236f0248eca11f91f11da02fe18be3f6d2e17c
Reviewed-on: https://review.whamcloud.com/39394
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
6 months agoLU-6142 obdclass: use cl_object_for_each more broadly. 91/39391/2
Mr NeilBrown [Thu, 16 Jul 2020 00:32:02 +0000 (10:32 +1000)]
LU-6142 obdclass: use cl_object_for_each more broadly.

There are several loops which follow the pattern of the macros
cl_object_for_each() or cl_object_for_each_reverse(), but
don't use those macros.

Change to use the macros.  In some case we change the name of an 'obj'
arg to a function to 'top', to avoid overloading the name 'obj'.

Also include some minor formatting improvements.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I544d4148719a9284f33fd61730bd733c0d40e97c
Reviewed-on: https://review.whamcloud.com/39391
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-6142 lustre: remove module_vars arg to class_register_type() 83/39383/3
Mr NeilBrown [Wed, 15 Jul 2020 06:32:47 +0000 (16:32 +1000)]
LU-6142 lustre: remove module_vars arg to class_register_type()

The module_vars arg to class_register_type() is always NULL.  So it
can be removed.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ie8d7e79075ba068dec606cc9dfcc38a90e371e5b
Reviewed-on: https://review.whamcloud.com/39383
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
6 months agoLU-6142 lov: style cleanups in lov_set_osc_active() 82/39382/3
Mr NeilBrown [Wed, 15 Jul 2020 05:42:23 +0000 (15:42 +1000)]
LU-6142 lov: style cleanups in lov_set_osc_active()

1/ don't pre-declare the function as it is not used before it
   is defined.
2/ " TEST ? 1 : 0" is identical to "TEST", so remove the "? 1 : 0".
3/ When the 'then' part of an 'if' ends with a GOTO, there is
   not point having an 'else', particularly if there is also
   code in the block following the "if".  The 'else' code and
   the following code should be together.

4/ other minor changes, and conversion of spaces to tabs.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I00cb8b1efe25c0ad9adc76ba21cca003aed203e2
Reviewed-on: https://review.whamcloud.com/39382
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-6142 lustre: change obd_ioctl_getdata() args 81/39381/4
Mr NeilBrown [Wed, 15 Jul 2020 06:16:35 +0000 (16:16 +1000)]
LU-6142 lustre: change obd_ioctl_getdata() args

Instead of passing a pointer to 'char *' to obd_ioctl_getdata(),
pass a pointer to 'struct obd_ioctl_data *'.
This seems more natural, and avoid needing extra 'char *data'
variables wherever the function is called.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I0287cd2fafd071a2580d48a52498469ceded3e79
Reviewed-on: https://review.whamcloud.com/39381
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
6 months agoLU-6142 lov: chnage lsm_op_find() to a non-inline function. 77/39377/2
Mr NeilBrown [Wed, 15 Jul 2020 04:57:28 +0000 (14:57 +1000)]
LU-6142 lov: chnage lsm_op_find() to a non-inline function.

Making lsm_op_find() non-inline improves encapsulation and means that
various _ops structures can now be static.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ifee2ab4f023e56bfa1e663a0106590cf7858856b
Reviewed-on: https://review.whamcloud.com/39377
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-6142 lustre: make various 'struct file_operations' static 41/39741/2
Mr NeilBrown [Thu, 27 Aug 2020 04:06:40 +0000 (14:06 +1000)]
LU-6142 lustre: make various 'struct file_operations' static

These 'struct file_operations' are only used locally, so make them
static.
Except lprocfs_evict_client_fops() which isn't used at all and doesn't
exist, so discard the declaration.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ib6c51683c1e765db202b3f72d2accebe17191303
Reviewed-on: https://review.whamcloud.com/39741
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
6 months agoLU-9859 libcfs: discard TCD_MAX_TYPES 68/39668/4
Mr. NeilBrown [Thu, 13 Aug 2020 12:44:25 +0000 (08:44 -0400)]
LU-9859 libcfs: discard TCD_MAX_TYPES

As well as CFS_TCD_TYPE_CNT we have TCD_MAX_TYPES which has a larger
value but a similar meaning.  Discard it and just use
CFS_TCD_TYPE_CNT.

Two places relied on the fact that TCD_MAX_TYPES was larger and so
there would be NULLs at the end of the array.  Change
them to check the array size properly.

Linux-commit: 22627ffa5e7fae0b0b62f90a8f30fc6dae0fff3c

Change-Id: I73a54a409674d3b569bcbaf72e2296209542b74c
Signed-off-by: Mr. NeilBrown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/39668
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-10391 lnet: allow lnet_connect() to use IPv6 addresses. 06/37706/14
Mr NeilBrown [Fri, 24 Jan 2020 01:10:55 +0000 (12:10 +1100)]
LU-10391 lnet: allow lnet_connect() to use IPv6 addresses.

Now that we can created and bind IPv6 sockets, connecting to them is
trivial.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I6032ea8a5937711922d9c318f72bc64c7020cc6d
Reviewed-on: https://review.whamcloud.com/37706
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-10391 lnet: allow creation of IPv6 socket. 05/37705/12
Mr NeilBrown [Wed, 22 Jan 2020 23:26:37 +0000 (10:26 +1100)]
LU-10391 lnet: allow creation of IPv6 socket.

With this patch, lnet_sock_create() can make IPv6 sockets.  If an
interface and destination is given, completely different code is
needed to bind the local address.

When no interface or destination is given, an IPv6 socket is always
created, so lnet_acceptor() needs to request that IPv4 connections are
accepted as well, and lnet_sock_getaddr() needs to detect mapped v4
addresses and present them as true v4 addresses.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Iaef1c08a9afd578b60f54cc83fd2d6882a021e6b
Reviewed-on: https://review.whamcloud.com/37705
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-9679 ptlrpc: list_for_each improvements. 33/39133/4
Mr NeilBrown [Sun, 7 Jun 2020 23:24:24 +0000 (19:24 -0400)]
LU-9679 ptlrpc: list_for_each improvements.

1/ use list_for_each_entry_safe() instead of
   list_for_each_safe() and similar.

2/ use list_first_entry() and list_last_entry() where appropriate.

3/ When removing everything from a list, use
   while ((x = list_first_entry_or_null()) {
   as it makes the intent clear

Linux-commit: ef8e5dbbb09035a0c41aa47a328e6248702d4d2b

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I490afcb70d18170e4ecc5d062d327d141668d3db
Reviewed-on: https://review.whamcloud.com/39133
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14275 tests: add ior_CLEANUP 79/41079/2
Elena Gryaznova [Tue, 22 Dec 2020 17:41:22 +0000 (20:41 +0300)]
LU-14275 tests: add ior_CLEANUP

Sometimes we need to run read IOR test separately from
write test. IOR -r test requires file exists before test starts.
If ior_CLEANUP set to false $testdir is not removed and
$testdir/iorData file can be used for IOR -r further test.

Test-Parameters: trivial testlist=parallel-scale
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
HPE-bug-id: LUS-9560
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@hpe.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Change-Id: Iac790976a46db2bf5be374a42e3facdce074228b
Reviewed-on: https://review.whamcloud.com/41079
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
6 months agoLU-14388 utils: always enable ldiskfs project quota 70/41370/3
Andreas Dilger [Sat, 30 Jan 2021 19:42:36 +0000 (12:42 -0700)]
LU-14388 utils: always enable ldiskfs project quota

Always enable project quota for newly-formatted ldiskfs filesystems.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1b0f745bc04b5c42592bcc4fd9823d068fef2a79
Reviewed-on: https://review.whamcloud.com/41370
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14353 obd: move debug.c to obdecho 90/41290/2
Mr NeilBrown [Fri, 30 Oct 2020 04:18:21 +0000 (15:18 +1100)]
LU-14353 obd: move debug.c to obdecho

The code in obdclass/debug.c is only used from obdecho/, so move the
file there and clean it up a little on the way (e.g. spaces to TABs).

Similarly lustre_debug.h contains very little that is used outside of
obdecho.  Move that little (LL_CDEBUG_PAGE()) into lustre_lib.h, and
add the declarations needed for debug.c into echo_internal.h.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ia1bcf3f3e1453130ed1adc75918bdad23eb5b5c7
Reviewed-on: https://review.whamcloud.com/41290
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14305 ldiskfs: add parameters for mb_c123_threshold 93/41193/10
Andreas Dilger [Wed, 6 Jan 2021 01:46:03 +0000 (18:46 -0700)]
LU-14305 ldiskfs: add parameters for mb_c123_threshold

Add mount options for /sys/fs/ldiskfs/*/mb_c[123]_threshold values
so that they can be set persistently via mount options.

The /sys/fs/ldiskfs/*/mb_c[123]_threshold values are always shown
rounded down to the next lower percentage value due to integer
division, since internal values are stored as blocks for efficiency.

Round up the values shown to the next percent to match what was
used to originally set these parameters.

Fixes: 95f8ae567749 ("LU-12103 ldiskfs: don't search large block range if disk full")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie36a6667f8bca7481aa8179ab5b97c85d449d619
Reviewed-on: https://review.whamcloud.com/41193
Reviewed-by: Artem Blagodarenko <artem.blagodarenko@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14289 ptlrpc: move heap.c from libcfs to ptlrpc 29/41129/4
Mr NeilBrown [Mon, 23 Nov 2020 02:26:22 +0000 (13:26 +1100)]
LU-14289 ptlrpc: move heap.c from libcfs to ptlrpc

cfs_binheap is only used by the nrs code in ptlrpc, so move it out of
libcfs to the one place it is used.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I4687d4de0a83e0ce106d4f4cfd5ea95faef27dfc
Reviewed-on: https://review.whamcloud.com/41129
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14291 lustre: only include nrs headers when needed 28/41128/4
Mr NeilBrown [Thu, 29 Oct 2020 04:01:35 +0000 (15:01 +1100)]
LU-14291 lustre: only include nrs headers when needed

Various lustre_nrs_*.h header are only needed on the server.  Make
this explicit, so it is easier to see which files are needed on the
client.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I9bc30acc4f62ea9af516055edbb7833919811089
Reviewed-on: https://review.whamcloud.com/41128
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14291 ptlrpc: support nrs_delay for client-only builds 27/41127/4
Mr NeilBrown [Thu, 29 Oct 2020 03:41:18 +0000 (14:41 +1100)]
LU-14291 ptlrpc: support nrs_delay for client-only builds

nrs_delay is intended for use on the client, but is currently disabled
for client-only builds.  So move the ptlrpc_nrs_policy_register() call
outside of the #ifdef.

Also change the nrs modules that *are* server-only to not have #ifdef
in the C file, but that rather to be identified in the Makefile as
only being built when server code is wanted.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Ie25ed5ac837ff5f891824fad31ab258923840ec0
Reviewed-on: https://review.whamcloud.com/41127
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14285 utils: Add error message when osd_init fails 18/41118/3
Arshad Hussain [Thu, 31 Dec 2020 19:12:05 +0000 (00:42 +0530)]
LU-14285 utils: Add error message when osd_init fails

It was observed that mkfs.lustre fails silently under
function osd_init(). One such case where it silently
fails is when back-end is 'zfs' and libz.so library
is not loaded or under search path.

This patch adds error message in case osd_init() call
fails. It makes end user observe the failure more easily.

Before Patch:
~~~~~~~~~~~~~
$ echo $LD_LIBRARY_PATH
/root/zfs_spl_install/0840/zfs/lib
$ unset LD_LIBRARY_PATH
$ mkfs.lustre --reformat --index 0 --backfstype=zfs --fsname=lustre --mgsnode=192.168.50.140@tcp --mgs --mdt gpool/metadata /dev/sdb1
$ echo $?
22

After Patch:
~~~~~~~~~~~~
$ echo $LD_LIBRARY_PATH
/root/zfs_spl_install/0840/zfs/lib
$ unset LD_LIBRARY_PATH
$ mkfs.lustre --reformat --index 0 --backfstype=zfs --fsname=lustre --mgsnode=192.168.50.140@tcp --mgs --mdt gpool/metadata /dev/sdb1
mkfs.lustre: osd_init() failed : 22 (Invalid argument)
$

Test-Parameters: trivial
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: Ie75b00974852d7a4b95b52d26501e55b66db5891
Reviewed-on: https://review.whamcloud.com/41118
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-12477 llite: remove unused ll_teardown_mmaps() 86/41086/2
Andreas Dilger [Thu, 24 Dec 2020 01:30:46 +0000 (18:30 -0700)]
LU-12477 llite: remove unused ll_teardown_mmaps()

The ll_teardown_mmaps() function is no longer used and can be
removed.

Fixes: f114d4dac6f8 ("LU-12477 llite: use generic_error_remove_page()")
Test-Parameters: trivial
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I42506210e028da3bf8036a28d4d61b1e7d3ebbe5
Reviewed-on: https://review.whamcloud.com/41086
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-930 doc: fix format man page sections for lctl 75/41075/4
James Nunez [Tue, 22 Dec 2020 15:56:45 +0000 (08:56 -0700)]
LU-930 doc: fix format man page sections for lctl

For lctl man pages, add proper formatting for the NAME
and TITLE sections so that 'man -k lctl*' reports the
real explanation for the command/utility and not
"(unknown subject)".

Modifications in this patch include:
- add a dash between multi-word name commands in NAME
- format the date to YYYY-MM-DD in TITLE
- capitalize command in TITLE
- change hyphen (-) to underscore in command where necessary

Test-Parameters: trivial
Signed-off-by: James Nunez <jnunez@whamcloud.com>
Change-Id: I946a3683b0a48b662c4e3b7b5783d389fa710e08
Reviewed-on: https://review.whamcloud.com/41075
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
6 months agoLU-14272 tests: different mpirun options for different users 72/41072/3
Elena Gryaznova [Tue, 22 Dec 2020 15:00:08 +0000 (18:00 +0300)]
LU-14272 tests: different mpirun options for different users

Patch adds the ability to use different mpirun options for
different mpi users, for example we use:
  --wdir=/home/quota15_1 for user quota15_1
and
  --wdir=/home/quota15_2 for user quota15_2.

Per mpi user option can be specified after ':' in ha_mpi_users:
Example:
  ha_mpi_users="quota15_1:--wdir=/home/quota15_1
                quota15_2:--wdir=/home/quota15_2
                quota15_3
                quota15_4:--wdir=/home/quota15_4"
Several options for user  can be set as:
  ha_mpi_users="user1:<option11>
                user1:<option12>
                user2:<option21>"
MPIRUN_OPTIONS will be applied for all users.

Patch adds the ability to execute the command before loads start.
Example: ha_precmd="for i in 1 2 3 4; do
                lfs quota -v -u quota15_$i $ha_test_dir; done"

Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
HPE-bug-id: LUS-9069
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@hpe.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Change-Id: I35feadcc856dbc3d6d6d5f2720ab49c6c111161b
Reviewed-on: https://review.whamcloud.com/41072
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14271 tests: add new node crash method 71/41071/3
Elena Gryaznova [Tue, 22 Dec 2020 14:06:09 +0000 (17:06 +0300)]
LU-14271 tests: add new node crash method

Patch adds a new node crash method: node crash
is simulated by command executed on the victim nodes.
Example:
  POWER_DOWN=sysrqcrash
triggers a crash  on all victim nodes.

Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
HPE-bug-id: LUS-8162
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@hpe.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Tested-by: Vitaly Fertman <vitaly.fertman@hpe.com>
Change-Id: Id39b0f58fbb59236b3af51516fe968c888b04a17
Reviewed-on: https://review.whamcloud.com/41071
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14270 tests: delay node's power up 70/41070/3
Elena Gryaznova [Tue, 22 Dec 2020 13:21:47 +0000 (16:21 +0300)]
LU-14270 tests: delay node's power up

In CS a node in UNCLEAN state can be STONITHed after
it has passed power down and power up already.

The patch adds ha_power_up_delay() in order to delay
node's power up until CRM state became OFFLINE when
failover pair is set and for $NODE_UP_DELAY seconds otherwise.
LOAD_TIMEOUT is added in order to ha_load_timeout be tunable.
The failover pair list for all victims are to be set via new
-f option.

ha_powermanage() is added to control node state by pm, (designed
by Maxim Storchak <maxim.storchak@hpe.com>).
PM_HOST is added in order to specify powermanage host, default
is $(hostname).

Test-Parameters: trivial
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
HPE-bug-id: LUS-8610
Reviewed-by: Maxim Storchak <maxim.storchak@hpe.com>
Reviewed-by: Andriy Skulysh <andriy.skulysh@hpe.com>
Change-Id: Ia192200b9ff61c17a67fc16858938f24ade46e23
Reviewed-on: https://review.whamcloud.com/41070
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14262 utils: lfs to set component flags by pool name 24/41024/5
Alex Zhuravlev [Thu, 17 Dec 2020 09:15:50 +0000 (12:15 +0300)]
LU-14262 utils: lfs to set component flags by pool name

so it'd be easy to set flags (like prefer) on the components
residing on specific OST identified by pool.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I733f92fe186682dc8d34512edf75b49e565c457f
Reviewed-on: https://review.whamcloud.com/41024
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14195 lustre: remove 'fs' from 'struct lvfs_run_ctxt' 10/40910/3
Mr NeilBrown [Wed, 9 Dec 2020 02:11:18 +0000 (13:11 +1100)]
LU-14195 lustre: remove 'fs' from 'struct lvfs_run_ctxt'

The code protected by push_ctxt() and pop_ctx() never tries to access
any user-space data, so call set_fs() to KERNEL_DS is not needed.

So remove the 'fs' field and related code.

In linux-5.10 this code fails to compile as set_fs() is deprecated.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Idb2744d656dc4228375b6da54673e38cc1c112f5
Reviewed-on: https://review.whamcloud.com/40910
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14195 osd: don't use set_fs() for ->fiemap() calls. 09/40909/3
Mr NeilBrown [Wed, 9 Dec 2020 02:00:16 +0000 (13:00 +1100)]
LU-14195 osd: don't use set_fs() for ->fiemap() calls.

->fiemap() only accesses kernel-space data, so does not need, and
never has needed, set_fs() calls.
In Linux 5.10, these calls are deprecated.
So remove the unnecessary code.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: Id336855b4787ddbf656dfa3b8d0b12f663564795
Reviewed-on: https://review.whamcloud.com/40909
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14195 build: Adjust Makefile for Linux build changes. 07/40907/3
Mr NeilBrown [Wed, 9 Dec 2020 00:28:16 +0000 (11:28 +1100)]
LU-14195 build: Adjust Makefile for Linux build changes.

Since v5.10-rc1~51^2~19, "KBUILD_BUILTIN" has been unset
for module builds.  This means that "targets-for-builtin"
isn't built, and that is how "extra-y" is built.

So we need another way to force LUSTRE_KERNEL_TEST to be built.

Since v5.6-rc1~1^2~5 any target listed in "always-y" will always get
built.  So we can assign LUSTRE_KERNEL_TEST to this macro.

Assigning both macros is safe, even for those kernels which include
both in the list of targets.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I508b3710579c068dec93baf81ee383f3f03bd370
Reviewed-on: https://review.whamcloud.com/40907
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14178 ldlm: return error from ldlm_namespace_new() 51/40851/3
Andreas Dilger [Thu, 3 Dec 2020 09:40:35 +0000 (02:40 -0700)]
LU-14178 ldlm: return error from ldlm_namespace_new()

Return the underlying error in ldlm_namespace_new() from
ldlm_namespace_sysfs_register() to the caller instead of NULL.
Otherwise, the callers convert the NULL to -ENOMEM and this
is incorrectly reported as an allocation error to the user.

  sysfs: cannot create duplicate filename
     '/fs/lustre/ldlm/namespaces/lustre-OST0002-osc-ffff89f33be70000'
  mount.lustre: mount mgs:/lfs at /lfs failed: Cannot allocate memory

Change ldlm_namespace_new() to return errors via PTR_ERR() and
change the callers to use IS_ERR().

Fix associated CERROR() messages to follow proper code style.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I7d590c7242399549b32b1c4189e46ff8748c8096
Reviewed-on: https://review.whamcloud.com/40851
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14073 ofd: remove use of smp_read_barrier_depends() 94/40394/2
Mr NeilBrown [Mon, 26 Oct 2020 02:30:22 +0000 (13:30 +1100)]
LU-14073 ofd: remove use of smp_read_barrier_depends()

Linux 5.9 removes smp_read_barrier_depends(), so lustre must stop
using it.

There is only one use: in ofd_access_log.c.
This use is unnecessary and can simply be removed.

The code is based on "Documentation/core-api/circular-buffers.rst"
which gives no indication that this barrier is needed.

The comment say its purpose is to ensure the index is read before the
data is read. This is unnecessary.
The data is written in osl_write_entry(), then a barrier is issued
(smp_store_release) before the ->head is written.
oal_read_entry() issues a barrier (smp_load_acquire()) before reading
that head.
'tail' is read without a barrer, but it then compared against ->head
in CIRC_CNT().  Even if reading ->tail was racey, the fact that
comparing it wilth ->head succeeded means that the data written at
->tail must have been safely written, and we can now read it without
any further barrier.

Test-Parameters: trivial
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I9d0f0aeb67e1188d2012f4ae2e14b3656211c3e2
Reviewed-on: https://review.whamcloud.com/40394
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14047 lustre: change EWOULDBLOCK to EAGAIN 07/40307/4
John L. Hammond [Tue, 20 Oct 2020 14:20:35 +0000 (09:20 -0500)]
LU-14047 lustre: change EWOULDBLOCK to EAGAIN

On linux, EWOULDBLOCK has always been defined as an alias for
EAGAIN. In the interest of readability we should not use two names for
the same thing. So change the remaining uses of EWOULDBLOCK to EAGAIN
and add EWOULDBLOCK||EAGAIN to spelling.txt.

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ib48b8a1e58bfa961d2a4ba411c038c476bfc300d
Reviewed-on: https://review.whamcloud.com/40307
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-13783 libcfs: switch from ->mmap_sem to mmap_lock() 88/40288/5
Mr NeilBrown [Fri, 16 Oct 2020 06:18:29 +0000 (17:18 +1100)]
LU-13783 libcfs: switch from ->mmap_sem to mmap_lock()

In Linux 5.8, ->mmap_sem is gone and the preferred interface
for locking the mmap is to suite of mmap*lock() functions.

So provide those functions when not available, and use them
as needed in Lustre.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I4ce3959f9e93eae10a7b7db03e2b0a1525723138
Reviewed-on: https://review.whamcloud.com/40288
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
6 months agoLU-9325 osd-ldisk: replace simple_strto* with kstr* functions 19/40119/8
James Simmons [Tue, 3 Nov 2020 14:27:43 +0000 (09:27 -0500)]
LU-9325 osd-ldisk: replace simple_strto* with kstr* functions

The parsing of mount parameters from the config llog is done in
some cases with simple_strto* which is considered obsolete.
Replace simple_strto* with the kstrto* equivalent functions.

Change-Id: I7c26d14d02828c9f9a96f31a086a65bb39f3ea87
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/40119
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-9859 lod: use linux kernel bitmap API 76/39876/2
James Simmons [Thu, 10 Sep 2020 14:35:55 +0000 (10:35 -0400)]
LU-9859 lod: use linux kernel bitmap API

Now that modern Linux kernels support a bitmap API we can move
away from the libcfs specific bitmap API. This patch changes
the last bitmap in the lod module to use the Linux kernel
bitmap API.

Change-Id: I92c494bf2af62e31d7b9527b3f44580322e48fd3
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/39876
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-9859 libcfs: replace all CFS_CAP_* macros with CAP_* 75/39875/3
Mr. NeilBrown [Thu, 10 Sep 2020 13:49:30 +0000 (09:49 -0400)]
LU-9859 libcfs: replace all CFS_CAP_* macros with CAP_*

Lustre defines a few CFS_CAP_* macros which are exactly the
same as the corresponding CAP_* macro, with one exception.

CFS_CAP_SYS_BOOT is 23
CAP_SYS_BOOT is 22.

CFS_CAP_SYS_BOOT is only used through CFS_CAP_FS_MASK and
causes capability 23 (CAP_SYS_NICE) to be dropped in certain
circumstances.
It is probable that the intention was to drop CAP_SYS_BOOT,
and this is what is now done.

CFS_CAP_CHOWN_MASK and CFS_CAP_SYS_RESOURCE_MASK are never
used, so they have been removed.

Linux-commit: 5ebaa2d14850205e44757c4d5fdd4097712d01ef

Change-Id: Ifb90c0a36e204c76b90ff23ac609345d11b878da
Signed-off-by: Mr. NeilBrown <neilb@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-on: https://review.whamcloud.com/39875
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-13100 lov: grant deadlock if same OSC in two components 95/37095/8
Andriy Skulysh [Thu, 24 Oct 2019 12:11:25 +0000 (15:11 +0300)]
LU-13100 lov: grant deadlock if same OSC in two components

The same osc can be involved in several components but osc layer
leaves active last used extent, so an RPC can't be sent if grants
are required from the same OST for another component.

Add cl_io_extent_release() to release active extent before
switching to the next component.

Change-Id: Idadda6eaecd1d47b78880c81e1fb7513d5be2419
Cray-bug-id: LUS-8038
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Alexander Zarochentsev <c17826@cray.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-on: https://review.whamcloud.com/37095
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-7853 lod: fixes bitfield in lod qos code 12/18812/15
Rahul Deshmkuh [Thu, 14 Jul 2016 06:02:45 +0000 (23:02 -0700)]
LU-7853 lod: fixes bitfield in lod qos code

Updating bitfields in struct lod_qos struct is protected
by lq_rw_sem in most places but an update can be lost
due unprotected bitfield access from
lod_qos_thresholdrr_seq_write() and qos_prio_free_store().
This patch fixes it by replacing bitfields with named bits
and atomic bitops.

Cray-bug-id: LUS-4651
Signed-off-by: Rahul Deshmukh <rahul.deshmukh@seagate.com>
Signed-off-by: Alexander Zarochentsev <c17826@cray.com>
Change-Id: I28299ce4960e91be551d7f6e43a3b598daf4d7a2
Reviewed-on: https://review.whamcloud.com/18812
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14180 utils: verify setstripe comp_end is valid 39/41239/3
Andreas Dilger [Tue, 26 Jan 2021 02:14:10 +0000 (18:14 -0800)]
LU-14180 utils: verify setstripe comp_end is valid

Verify that the "lfs setstripe -E <component_end>" value is valid.
Otherwise, if "-S" is not specified at the same time, then an
invalid file layout can be created and the file cannot be deleted
normally, only via "lfs rmdif <FID>".

Allow values < 4096 (e.g. '64' or '128' which would all be invalid
anyway) to be interpreted as KiB units.

Update usage messages and man pages to match.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Change-Id: I47fe7729ffd447c1c1cc098e5117e456263ebbe5
Reviewed-on: https://review.whamcloud.com/41239
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14207 mgs: delete "add failnid" sections on replace_nids 30/40930/3
Artem Blagodarenko [Wed, 14 Oct 2020 01:54:36 +0000 (21:54 -0400)]
LU-14207 mgs: delete "add failnid" sections on replace_nids

Replace_nids left old nids in add_conn field of failnid
section of client llog. This leads to connecton errors.

Let's delete such sections. New failnids, if any, are
added by replace_nids.

Change-Id: I7fab00827035bd864aeb95fb4852a59c458bb2ba
HPE-bug-id: LUS-9440
Signed-off-by: Artem Blagodarenko <artem.blagodarenko@hpe.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Reviewed-by: Alexander Boyko <c17825@cray.com>
Tested-by: Jenkins Build User <nssreleng@cray.com>
Reviewed-on: https://review.whamcloud.com/40930
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Alexander Boyko <alexander.boyko@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-13974 llog: check stale osp object 42/40742/3
Alexander Boyko [Tue, 24 Nov 2020 05:34:11 +0000 (00:34 -0500)]
LU-13974 llog: check stale osp object

The logic of osp_attr_get has 2 path,
1) return attributes from a cache for health osp object
2) make an out update request and return attributes for stale
osp object, object lose stale state.

When some out update request with llog writes failed, osp object
become stale. But llog handle stay inconsistent (bitmap,count,
last_index), and a next llog_add->llog_osd_write_rec do dt_attr_get,
gets attributes and makes osp object valid, and uses wrong llog
handle data. The result is index jump at llog file - recX, recX+2.
And it makes an error during update log processing if failover take
a place.
The fix adds dt_object_stale function to check osp_object.
llog_osd_write_rec check it and return ESTALE. llog_add would fail
with ESTALE error and doesn't corrupt update log.

HPE-bug-id: LUS-9030
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: Iadf53fd816e1c5bde0a19d4c537f0408796c864a
Reviewed-on: https://review.whamcloud.com/40742
Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@hpe.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-12961 mdd: avoid double call to mdd_changelog_fini() 37/36737/11
Alex Zhuravlev [Tue, 12 Nov 2019 17:45:50 +0000 (20:45 +0300)]
LU-12961 mdd: avoid double call to mdd_changelog_fini()

the first call is done from mdd_prepare() as part of error
handling, another call is done from mdd_device_shutdown().

in the similar cases fini routines checks whether the state
is initialized. e.g. mdd_orphan_index_fini() releases the
object and sets mdd_orphans to NULL, then all subsequent
calls to mdd_orphan_index_fini() return immediately.
mdd_changelog_fini() can't do this way, so the excplicit
state has been introduced.

Change-Id: Ifd21569e68c836f44bb59adea4e8fed6ccef1c7b
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/36737
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
6 months agoLU-14439 build: require a newer version of e2fsprogs 82/41682/4
Andreas Dilger [Wed, 17 Feb 2021 23:50:59 +0000 (16:50 -0700)]
LU-14439 build: require a newer version of e2fsprogs

Require a build version of e2fsprogs-1.44.3 for osd-ldiskfs builds to
get EXT2_FLAG_IGNORE_SB_ERRORS and ext2fs_has_feature_{mmp,quota}()
functions.

Require a minimum install version of ldiskfsprogs-1.45.6.wc1.

Test-Parameters: trivial
Fixes: 7dc8aa7e7848 ("LU-13241 utils: use libext2fs for ldiskfs operations")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I240823266c7342338c72126011485120543ebbe5
Reviewed-on: https://review.whamcloud.com/41682
Reviewed-by: Olaf Faaland-LLNL <faaland1@llnl.gov>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-13609 mgs: fix config_log buffer handling 78/41478/5
Stephane Thiell [Thu, 11 Feb 2021 00:15:02 +0000 (16:15 -0800)]
LU-13609 mgs: fix config_log buffer handling

Fix buffer handling in mgs_list_logs() to list all MGS config_logs
using multiple ioctl calls when we have a large number of targets.

Fixes: 1d97a8b4cd3d ("LU-13609 llog: list all the log files correctly on MGS/MDT")
Signed-off-by: Stephane Thiell <sthiell@stanford.edu>
Change-Id: I1bf32e918e242f4da83c3d1624b7285a18a88d01
Reviewed-on: https://review.whamcloud.com/41478
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-10391 socklnd: use sockaddr instead of __u32 addresses. 04/37704/17
Mr NeilBrown [Thu, 14 May 2020 22:51:54 +0000 (08:51 +1000)]
LU-10391 socklnd: use sockaddr instead of __u32 addresses.

LNet/socklnd often uses __u32 to hold an ipv4 address.
As we want to extend socklnd to work with IPv6 addresses too,
this needs to change.

This patch changes many __u32s to variants of 'struct sockaddr'.

Library code from sunrpc is used for copying and comparing addresses
   rpc_copy_addr() and rpc_cmp_addr()
and for extracting or setting the port:
   rpc_get_port() and rpc_set_port().

The "%pIS" printf format is used for printing a sockaddr (works for
both IPv4 and IPv6), and "%pISp" for printing the address with the
port.

The __u32 is in host-byte-order, while addresses in sockaddr are
always network-order, so htonl and ntohl are used as needed.

When storing an address (e.g. in a structure), 'struct
sockaddr_storage' is used.  When passing an address to a function,
'struct sockaddr' is used.  When an address is known to be IPv4 (i.e.
when converting to or from __u32), 'struct sockaddr_in' is used.

The following functions are changed to take a 'struct sockaddr*'
argument:

 lnet_connect()
 lnet_connect_console_error()
 lnet_sock_getaddr()
 ksocknal_ip2iface()
 ksocknal_ip2index()
 ksocknal_create_route()
 ksocknal_connecting()
 ksocknal_close_peer_conns_locked()
 ksocknal_peer_del_interface_locked()

The following structures have had fields changed to 'struct
sockaddr_storage'

 struct ksock_interface:
      ksni_ipaddr -> ksni_addr
 struct ksock_conn
      ksnc_myipaddr -> ksnc_myaddr
      ksnc_ipaddr and ksnc_port -> ksnc_peeraddr
 struct ksock_route
      ksnr_myipaddr -> ksnr_myaddr
      ksnr_ipaddr and ksnr_port -> ksnr_addr

Assorted strings have been joined onto a single line, and various
indented have been changed from space to tabs.

There should be no behaviour changes, though the structures mentioned
above will now be a little larger.

Signed-off-by: Mr NeilBrown <neilb@suse.de>
Change-Id: I42d12260185638407b5b611391fc69bfd9f91754
Reviewed-on: https://review.whamcloud.com/37704
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Aurelien Degremont <degremoa@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-13929 lnet: modify assertion in lnet_post_send_locked 49/40749/4
Serguei Smirnov [Wed, 25 Nov 2020 00:05:48 +0000 (16:05 -0800)]
LU-13929 lnet: modify assertion in lnet_post_send_locked

Check that the pointer to the local interface is not NULL
before asserting. While checking if local ni is the destination,
the assertion may attempt to dereference pointer to local
interface after it has already been cleaned up on shutdown.

Test-Parameters: trivial
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I0f4be04a728a7243823bec70f9efbe52bcb104b3
Reviewed-on: https://review.whamcloud.com/40749
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14362 tests: sanity-flr to prepare stuff before checks 07/41307/6
Alex Zhuravlev [Mon, 25 Jan 2021 05:37:12 +0000 (08:37 +0300)]
LU-14362 tests: sanity-flr to prepare stuff before checks

otherwise /mnt/lustre may be missing and sanity-flr fails

Test-Parameters: trivial testlist=sanity-flr
Change-Id: Ic5502ee07bab557162fbf16718ca5fb5beed45e9
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/41307
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
6 months agoLU-14423 osd: recognize holes in osd_is_mapped() 81/41481/3
Alex Zhuravlev [Thu, 11 Feb 2021 14:33:01 +0000 (17:33 +0300)]
LU-14423 osd: recognize holes in osd_is_mapped()

ldiskfs_fiemap() can return {0,0,0} for last non-allocated
region.  osd_is_mapped() should be able to recognize and
cache this state.

Fixes: 144b5a65c1 ("LU-7132 osd-ldiskfs: speedup rewrites")
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I03883038c2c0ec84754377a442c4947c7e3021a9
Reviewed-on: https://review.whamcloud.com/41481
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14398 llapi: add llapi_fid2path_at() 06/41406/2
John L. Hammond [Wed, 3 Feb 2021 19:06:16 +0000 (13:06 -0600)]
LU-14398 llapi: add llapi_fid2path_at()

Add llapi_fid2path_at() which works like llapi_fid2path() takes an
open FD on the moint point instead of a 'fsname or dirirectory path'
and a const struct lu_fid * instead of a const char *.

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I76234bc28de231587b65c5d866954441e0893aac
Reviewed-on: https://review.whamcloud.com/41406
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14398 llapi: simplify llapi_fid2path() 05/41405/2
John L. Hammond [Wed, 3 Feb 2021 18:33:26 +0000 (12:33 -0600)]
LU-14398 llapi: simplify llapi_fid2path()

Simplify llapi_fid2path(). Remove the fid_is_sane() check. Remove the
call to root_ioctl() and use get_root_path() directly.

Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: Ib70b8c9e239c77da8b46408de8341fc8aaf4d1c3
Reviewed-on: https://review.whamcloud.com/41405
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
6 months agoLU-14390 gnilnd: Use DIV_ROUND_UP to calculate niov 71/41371/2
Shaun Tancheff [Sun, 31 Jan 2021 14:47:21 +0000 (08:47 -0600)]
LU-14390 gnilnd: Use DIV_ROUND_UP to calculate niov

Use DIV_ROUND_UP to calculate niov, also remove 'unlikely'
as this is the common case.

Fixes: 7a74d382 ("LU-13004 modules: replace lnet_kiov_t with struct bio_vec")
Test-Parameters: trivial
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Iaf62326743ebe5e92bb3aa4b5780b47a5cfdfb18
Reviewed-on: https://review.whamcloud.com/41371
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14301 lustre: add ENOTSUPP to spelling.txt 80/41280/2
John L. Hammond [Wed, 20 Jan 2021 15:22:54 +0000 (09:22 -0600)]
LU-14301 lustre: add ENOTSUPP to spelling.txt

Add a spelling check for ENOTSUPP to suggest use of EOPNOTSUPP
instead. Note:

ENOTSUPP (524) and defined only in the kernel errno.h and is a NFSv3
specific errno. If ENOTSUPP is returned to userspace then strerror()
will print "Unknown error 524".

EOPNOTSUPP (95) is defined in kernel and userspace errno.h.

ENOTSUP is defined in userspace errno.h as an alias for EOPNOTSUPP.

Test-Parameters: trivial
Signed-off-by: John L. Hammond <jhammond@whamcloud.com>
Change-Id: I13b0389c9ec0853f43d8ab4a8f6538eb24c8a2ad
Reviewed-on: https://review.whamcloud.com/41280
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
6 months agoLU-12766 test: convert time to seconds properly 59/41259/4
Wang Shilong [Mon, 18 Jan 2021 02:26:31 +0000 (10:26 +0800)]
LU-12766 test: convert time to seconds properly

According to test logs, grace time could be 3m56s,
and it will be converted to 21305s by wrongly which
caused timeout of test sytem.

Actually, grace time could be something like 1w2d3h4m5s
Fix calculations for this.

Test-Parameters: trivial testlist=sanity-quota
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I1c02e83f5ec15de2a4a6b312dd6e36b55dd4a7bc
Reviewed-on: https://review.whamcloud.com/41259
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: James Nunez <jnunez@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-13903 build: make lustre-devel buildable for Linux client 88/41188/2
James Simmons [Sat, 9 Jan 2021 15:24:16 +0000 (10:24 -0500)]
LU-13903 build: make lustre-devel buildable for Linux client

Recently a new lustre-devel rpm was created which always has a
dependency on the kernel detected by the kmod macros. This doesn't
work in the case of running a custom Linux kernel. Addreess this
by not making use of the kmod macros when only building the Lustre
utilies i.e configure --disable-modules.

Test-Parameters: trivial
Fixes: 16af4e5ed634 ("LU-9215 build: Re-add the lustre-devel package")
Signed-off-by: James Simmons <jsimmons@infradead.org>
Change-Id: I324fbaa0f5b03e2095b493f3d8e00b74ca64298a
Reviewed-on: https://review.whamcloud.com/41188
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14313 utils: mount error when no server support 82/41182/4
Olaf Faaland [Wed, 6 Jan 2021 09:57:26 +0000 (01:57 -0800)]
LU-14313 utils: mount error when no server support

When the mount utility was built without server support,
and this causes the mount to fail, an error message should
say so.

The error values returned by main() should be positive as
they are exposed to the user.

Upon mount() failure, errno must be checked to determine
the reason for the failure.  The return value is -1.

Test-Parameters: trivial
Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Change-Id: I49906340f6ddbe16b7503663cedabe0085daf113
Reviewed-on: https://review.whamcloud.com/41182
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14268 lod: fix layout generation inc for mirror split 68/41068/4
Bobi Jam [Tue, 22 Dec 2020 05:58:41 +0000 (13:58 +0800)]
LU-14268 lod: fix layout generation inc for mirror split

Mirror split does not increase the layout generation properly.

Mirror split does not change FLR state of the file, even when it
contains 1 mirror afterwards, and FLR state should be LCM_FL_NONE
instead.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I9c9621d67d901f2e9ca6ed3e0684cd308c396076
Reviewed-on: https://review.whamcloud.com/41068
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14098 obdclass: try to skip corrupted llog records 54/40754/10
Alex Zhuravlev [Thu, 26 Nov 2020 12:07:24 +0000 (15:07 +0300)]
LU-14098 obdclass: try to skip corrupted llog records

if llog's header or record is found corrupted, then
ignore the remaining records and try with the next one.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: If47ec1fc1e2eaf64be7ba08d3aa9c2b93903c0cf
Reviewed-on: https://review.whamcloud.com/40754
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-9820 osd-ldiskfs: OI scrub speed limit fix 91/40591/3
Lai Siyao [Tue, 10 Nov 2020 02:44:17 +0000 (10:44 +0800)]
LU-9820 osd-ldiskfs: OI scrub speed limit fix

The OI scrub speed limit is set at start time, and shouldn't be
changed in the middle, otherwise lfsck is run with speed control,
while OI scrub may finish before it, thus sanity-scrub 9 failed.

Allow wider margin in speed control since the speed control is
heuristic.

Test-Parameters: testlist=sanity-scrub,sanity-scrub,sanity-scrub,sanity-scrub,sanity-scrub,sanity-scrub,sanity-scrub,sanity-scrub,sanity-scrub,sanity-scrub,sanity-scrub,sanity-scrub,sanity-scrub
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I995679db085a193c71fc7914f961b538b5930c69
Reviewed-on: https://review.whamcloud.com/40591
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14099 build: Fix for unconfigured arch_stackwalk 03/40503/7
Shaun Tancheff [Sat, 23 Jan 2021 15:22:48 +0000 (09:22 -0600)]
LU-14099 build: Fix for unconfigured arch_stackwalk

On aarch64 CONFIG_ARCH_STACKWALK is not defined and
print_stack_trace is not available.

Replace print_stack_trace with an open-coded variant
using %pB introduced in Linux v2.6.38-6557-g0f77a8d37825

This also fixes the symbols lookup of stack_trace_save_tsk
using kallsyms at module init time over the use of
symbol_get.

HPE-bug-id: LUS-9518
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: I04c3a0a84bb1a05d813a90502d1ed0f5bb2e33ab
Reviewed-on: https://review.whamcloud.com/40503
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Tested-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-14044 llog: check fid after convert 94/40294/6
Yang Sheng [Sat, 17 Oct 2020 15:28:43 +0000 (23:28 +0800)]
LU-14044 llog: check fid after convert

We should convert from llog_id and then check fid. Also
change fid-lookup to error check instead LASSERT.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I673d8f16ff9e57a0482d6a3ec3ee3db33699f57f
Reviewed-on: https://review.whamcloud.com/40294
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-13620 tests: pool_add_targets() fix 03/38803/4
Sergey Cheremencev [Tue, 2 Jun 2020 08:03:08 +0000 (11:03 +0300)]
LU-13620 tests: pool_add_targets() fix

Fix pool_add_targets to don't fail if number of
OSTs is >= 10 - lctl expects them in a hex view.
Check result of "lctl pool_add". Pass only if
the result is either 0 or 17(EEXIST).
Finally, make pool_add_targets to check that were
added only requested OSTs, i.e. don't fail to add
OST0 if OST1 is already in a pool.

Test-Parameters: trivial testlist=ost-pools
HPE-bug-id: LUS-8723
HPE-bug-id: LUS-8941
Change-Id: I841b3db3a89dbc86075cd23b7d71764ffb849181
Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
Signed-off-by: Sergey Cheremencev <sergey.cheremencev@hpe.com>
Reviewed-on: https://review.whamcloud.com/38803
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-13584 tests: gather_logs() fix 60/38660/6
Elena Gryaznova [Tue, 19 May 2020 10:29:32 +0000 (13:29 +0300)]
LU-13584 tests: gather_logs() fix

Fix gather_logs() to work on real HW where server nodes
do not have the access to clients.

Signed-off-by: Elena Gryaznova <elena.gryaznova@hpe.com>
HPE-bug-id: LUS-8888
Reviewed-by: Alexander Lezhoev <alexander.lezhoev@hpe.com>
Reviewed-by: Vladimir Saveliev <vladimir.saveliev@hpe.com>
Change-Id: Ifeea54e20d3123ee64582e32b92a4573e60ff33e
Reviewed-on: https://review.whamcloud.com/38660
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Vladimir Saveliev <c17830@cray.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-13513 osp: make neterr not fatal for precreate_reserve 72/38472/10
Vladimir Saveliev [Mon, 2 Nov 2020 10:10:42 +0000 (13:10 +0300)]
LU-13513 osp: make neterr not fatal for precreate_reserve

When OST_CREATE (not resendable rpc) sent by precreate thread fails
with network error, osp_pre_update_status() sets d->opd_pre_status to
EIO. osp_precreate_reserve() considers EIO as fatal and does not wait
for another attempt from precreate thread. That may make
mdt_intent_open() to return ENOSPC confusing a caller.  ENOSPC comes
from lod_alloc_rr().

osp_precreate_send(): in case of network error switch EIO to ENOTCONN.

Test to illustrate the issue is added.

Cray-bug-id: LUS-8811
Signed-off-by: Vladimir Saveliev <c17830@cray.com>
Change-Id: Iffaad9bd16f216f758c784b708e21b525c999b14
Reviewed-on: https://review.whamcloud.com/38472
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
6 months agoLU-13453 osd-ldiskfs: do not leak inode if OI insertion fails 35/38235/16
Alex Zhuravlev [Wed, 15 Apr 2020 14:54:07 +0000 (17:54 +0300)]
LU-13453 osd-ldiskfs: do not leak inode if OI insertion fails

osd_create() should destroy just created inode if OI insertion
fails.

also fixes lustre_index_restore() to drop nlink for object to
be removed.

the patch adds two tests:
 - ENOSPC on OI insertion
 - ENOSPC on .. insertion, i.e. directory block allocation

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I2a5db657c7dab54b8dc2c50bc29365d5ee754a2e
Reviewed-on: https://review.whamcloud.com/38235
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
6 months agoMerge "LU-9121 lnet: User Defined Selection Policy (UDSP)"
Gerrit Code Review [Fri, 26 Feb 2021 07:21:34 +0000 (07:21 +0000)]
Merge "LU-9121 lnet: User Defined Selection Policy (UDSP)"

6 months agoLU-12682 llite: fake symlink type of foreign file/dir 56/35856/52
Bruno Faccini [Thu, 22 Aug 2019 08:22:53 +0000 (10:22 +0200)]
LU-12682 llite: fake symlink type of foreign file/dir

This patch implements a "fake symlink" specific usage of
"foreign" LOV/LMV format. It basically allows these
particular type of foreign files/dirs to behave as a
symlink from VFS point of view, by allowing to construct
a relative path from the LOV/LMV foreign content, to
complement it with a prefix, and then to expose it to
the VFS as a symlink destination. The default/internal
mechanism simply takes the full foreign free string as
the relative path, and for more complex internal formats
an upcall has been implemented to provide format's
details (presently just in terms of constant strings
and substrings positions in EA, but this can be enhanced)
to llite layer.
Using this feature, instead of real symlinks or user EA,
will permit to benefit from the special features (lock,
prefetch, caches) already implemented to handle both
LOV/LMV EAs.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Id3c262e3b042447aa09aad25f682ff02787b350d
Reviewed-on: https://review.whamcloud.com/35856
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Ben Evans <beevans@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 months agoLU-14444 gss: handle empty reqmsg in sptlrpc_req_ctx_switch 85/41685/2
Sebastien Buisson [Thu, 18 Feb 2021 11:03:31 +0000 (20:03 +0900)]
LU-14444 gss: handle empty reqmsg in sptlrpc_req_ctx_switch

In sptlrpc_req_ctx_switch(), everything is already there to handle
the case of a ptlrpc_request that has an empty rq_reqmsg.
But assertions were left over at the beginning of the function, so
just remove them from here.

Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I6ae1f8b9da9600d3b57b9efc9018c2461114f2fe
Reviewed-on: https://review.whamcloud.com/41685
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Oleg Drokin <green@whamcloud.com>
7 months agoLU-14455 mdt: fix DoM lock prolong logic 01/41701/3
Mikhail Pershin [Fri, 19 Feb 2021 20:50:54 +0000 (23:50 +0300)]
LU-14455 mdt: fix DoM lock prolong logic

- don't stop at the first found lock if it is not PW or EX lock
- add LCK_GROUP lock as valid mode of lock to check

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: If947f8565008953cc34146b6f0ac1e0f0a038bb5
Reviewed-on: https://review.whamcloud.com/41701
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 months agoLU-14436 tgt: only use T10PI guard when doing full sector read 77/41677/2
Li Dongyang [Tue, 16 Feb 2021 12:40:05 +0000 (23:40 +1100)]
LU-14436 tgt: only use T10PI guard when doing full sector read

The T10PI guard was generated on full sectors, if we
do we partial read and still use the guard, the rpc
checksum won't match.

Test-Parameters: trivial
Change-Id: I40d481d703a46b9711021a162208b86a956bd8d1
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/41677
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 months agoLU-14435 doc: include lfs-flushctx manpage inside packages 76/41676/4
Sebastien Buisson [Tue, 16 Feb 2021 08:58:25 +0000 (17:58 +0900)]
LU-14435 doc: include lfs-flushctx manpage inside packages

lfs manpage redirects to lfs-flushctx(1), so it has to be
included in the Lustre packages.

Test-Parameters: trivial
Fixes: c246a9ba04 ("LU-14263 gss: unlink revoked key")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I5c55f8a74eb6dac20fa85b6ea0663ad701341006
Reviewed-on: https://review.whamcloud.com/41676
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
7 months agoLU-14430 mdd: fix inheritance of big default ACLs 94/41494/6
Mikhail Pershin [Fri, 12 Feb 2021 07:16:24 +0000 (10:16 +0300)]
LU-14430 mdd: fix inheritance of big default ACLs

If the number of default ACLs in directory is more than 31, then
mdd_acl_init() fails to inherit them for a newly created file.
This limitation is caused by using a fixed-size def_acl_buf buffer
in the mdd_create()->mdd_acl_init() call chain. Instead, the
default ACL buffer should be increased when it is needed.

Patch adds check for -ERANGE after mdd_acl_init(), reallocates
default ACL buffer with required size and calls mdd_acl_init()
again. Thus big default ACL are processed as expected.

Fixes: 6350af100c20 ("LU-3437 mdd: Fix ACL/def_ACL during object creation")
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I700da90c09f824955fcb8dc7ca0bc2f581f916a0
Reviewed-on: https://review.whamcloud.com/41494
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
7 months agoNew tag 2.14.50 2.14.50 v2_14_50
Oleg Drokin [Mon, 22 Feb 2021 19:24:50 +0000 (14:24 -0500)]
New tag 2.14.50

Change-Id: I760c8197bafa0f7eb8d1f5d78863900d647ccddc
Signed-off-by: Oleg Drokin <green@whamcloud.com>
7 months agoLU-9121 lnet: User Defined Selection Policy (UDSP) 18/41718/1
Amir Shehata [Mon, 22 Feb 2021 17:20:31 +0000 (09:20 -0800)]
LU-9121 lnet: User Defined Selection Policy (UDSP)

User Defined Selection Policies (UDSP) are introduced to add
ability of fine traffic control. The policies are instantiated
on LNet constructs and allow preference of some constructs
over others as an extension of the selection algorithm.
The order of operation is defined by the selection
algorithm logical flow:

   1. Iterate over all the networks that a peer can be reached on
      and select the best local network
      - The remote network with the highest priority is examined
        (Network Rule)
      - The local network with the highest priority is selected
        (Network Rule)
      - The local NI with the highest priority is selected
        (NID Rule)
   2. If the peer is a remote peer and has no local networks,
      - then select the remote peer network with the highest
        priority (Network Rule)
      - Select the highest priority remote peer_ni on the network
        selected (NID Rule)
      - Now that the peer's network and NI are decided, select
        the router in round robin from the peer NI's preferred
        router list. (Router Rule)
      - Select the highest priority local NI on the local net of the
        selected route.
        (NID Rule)
   3. Otherwise for local peers, select the peer_ni from the peer.
      - highest priority peer NI is selected
        (NID Rule)
      - Select the peer NI which has the local NI selected on its
        preferred list.
        (NID Pair Rule)

   Accordingly, the User Interface allows for the following:
   - Adding a local network udsp: if multiple local networks are
     available, each one can have a priority.
   - Adding a local NID udsp: after a local network is chosen,
     if there are multiple NIs, each one can have a priority.
   - Adding a remote NID udsp: assign priority to a peer NID.
   - Adding a NID pair udsp: allows to specify local NIDs
     to be added on the list on the specified peer NIs
     When selecting a peer NI, the one with the
     local NID being used on its list is preferred.
   - Adding a Router udsp: similar to the NID pair udsp.
     Specified router NIDs are added on the list on the specified
     peer NIs. When sending to a remote peer, remote net is selected
     and the peer NID is selected. The router which has its nid on
     the peer NI list is preferred.

Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Change-Id: I0d94d160bdb8c09c1cf467f1c155e14ab761a47e

7 months agoLU-9121 lnet: Add info on udsp to lnetctl man page multi-rail
Serguei Smirnov [Thu, 21 May 2020 23:28:39 +0000 (19:28 -0400)]
LU-9121 lnet: Add info on udsp to lnetctl man page

Adding description of UDSP commands to lnetctl man page.
Listing each UDSP rule type with specific parameters.
Adding some examples of UDSP commands.

Test-Parameters: trivial testlist=lnet-selftest,sanity-lnet
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I1bde8a4da217c9ba89f31b5d9a1e7d26658bfb40
Reviewed-on: https://review.whamcloud.com/38698
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Amir Shehata <ashehata@whamcloud.com>
7 months agoLU-9121 lnet: add show udsp command
Amir Shehata [Tue, 2 Apr 2019 23:16:59 +0000 (16:16 -0700)]
LU-9121 lnet: add show udsp command

Add the show udsp command in liblnetconfig and lnetctl

Test-Parameters: trivial testlist=lnet-selftest,sanity-lnet
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: Ibddfa60a7b257b136a6be403e94d6f73fb444222
Reviewed-on: https://review.whamcloud.com/34580
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
7 months agoLU-9121 lnet: Delete a selection policy
Sonia Sharma [Fri, 29 Mar 2019 10:51:01 +0000 (06:51 -0400)]
LU-9121 lnet: Delete a selection policy

This patch adds the function lustre_lnet_del_udsp()
which takes index of the udsp as input and calls
the IOC_LIBCFS_DEL_UDSP ioctl handler to
delete the udsp with that index value.

Change-Id: Ib81c79fd4e76f4e8fbd84709e08cb5d29a059e63
Test-Parameters: trivial testlist=lnet-selftest,sanity-lnet
Signed-off-by: Amir Shehata <ashehata@whamcloud.com>
Signed-off-by: Sonia Sharma <sharmaso@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34553
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>