Whamcloud - gitweb
Patrick Farrell [Sun, 24 Sep 2023 21:19:34 +0000 (17:19 -0400)]
LU-13814 clio: add cp_inode to page allocation
cp_inode can be set correctly during page allocation,
rather than after. This is a prelude to moving cp_inode to
the osc_transfer_page, but that's better done in a separate
patch.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I509f6cfbae8e5a6ec6b07c8253d68f6dd2794e59
Patrick Farrell [Sat, 9 Sep 2023 22:48:19 +0000 (18:48 -0400)]
LU-13814 osc: rename osc_async_flags
Async flags are used for all osc pages, rename them
accordingly.
Test-Parameters: trivial
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ibf808bcad5f03d950b4d2d05fb11f047fd28f311
Patrick Farrell [Tue, 5 Sep 2023 00:10:40 +0000 (20:10 -0400)]
LU-13814 osc: rename osc_prep_async_page
This is another piece of renaming osc_async_page to
osc_transfer_page. This is kept separate to make the
previous patch as focused as possible.
Test-Parameters: trivial
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ifd57eec46aeeb059ac836e09aa47322a69cc1493
Patrick Farrell [Sun, 24 Sep 2023 21:12:26 +0000 (17:12 -0400)]
LU-13814 osc: rename osc_async_page
osc_async_page isn't for async IO - it's for all data IO.
Rename it osc_transfer_page so the name fits the usage.
Test-Parameters: trivial
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I930aae585763f95d9085bea179765a0431bccf52
Patrick Farrell [Sun, 24 Sep 2023 21:09:49 +0000 (17:09 -0400)]
LU-13814 osc: Move osc_page members to osc_async_page
We're going to start using osc_async_page separately from
osc_page, since it's used for both DIO and buffered, but
osc_page is not.
This moves all of the members which are needed for DIO.
This commit deliberately ignores some packing related
issues which will be handled *much* later, since the names
and members of these structs will keep changing.
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I994af9563142201a2c7193735ac02568735bd8d5
Patrick Farrell [Mon, 4 Sep 2023 23:52:03 +0000 (19:52 -0400)]
LU-13814 osc: Remove usage of cdp_cl_pages
As part of eliminating cl_page in the DIO path, we need to
clear out all uses of it.
This is one more minor one - there's no need to clear this
list before returning from this function, because if it
fails we give up entirely.
Test-Parameters: trivial
Test-Parameters: testlist=sanity-sec env=ONLY=52,59a,59b
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I9dc1053c542ce7a903a93f7b9a1fb0bfc6ac1641
Patrick Farrell [Wed, 30 Aug 2023 19:02:46 +0000 (15:02 -0400)]
LU-13814 osc: remove use of cp_obj
Since we're going to lose the cl_page, we need to remove
usage of its members from the code which handles DIO pages.
This removes cp_obj usage from the one place where it's
used by the DIO code, by adding it to the BRW async args.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ic53b872d930305c345a03c75dc21a613874bf3c8
Patrick Farrell [Sun, 24 Sep 2023 21:06:49 +0000 (17:06 -0400)]
LU-13814 clio: use osc_dio_completion everywhere
The conversion to osc_dio_completion was incomplete because
some other code wasn't ready. Finish that conversion.
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Id6500bfb55dc27e783a91f58498f9a13906056b8
Patrick Farrell [Wed, 30 Aug 2023 15:10:26 +0000 (11:10 -0400)]
LU-13814 osc: simplify osc_dio_page_submit
Remove more code from osc_dio_page_submit, with the goal of
eliminating it entirely.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Id408aad01c2e81a126dd04807819aed9d29fca6b
Patrick Farrell [Mon, 4 Sep 2023 23:45:25 +0000 (19:45 -0400)]
LU-13814 osc: add osc_dio_completion
Specialize osc_completion to add the osc_dio_completion
version.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I4d202165ee6c4f50338b6a9381aa476b969d3593
Patrick Farrell [Sun, 24 Sep 2023 21:05:14 +0000 (17:05 -0400)]
LU-13814 osc: add osc_dio_page_submit
This patch adds the osc_dio_page_submit and specializes
osc_page_submit to take only BIO pages.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I048bbcd25ffee28fbbd8fee26017796b5e3586bd
Patrick Farrell [Sun, 24 Sep 2023 21:02:02 +0000 (17:02 -0400)]
LU-13814 clio: add osc_dio_pages_init
We need the OSC offset of the cl_dio_pages array, so we
add the osc_dio_pages_init function to collect it.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I97a0355b1cb39e9106363577c71b9fde64879cdd
Patrick Farrell [Tue, 29 Aug 2023 21:01:45 +0000 (17:01 -0400)]
LU-13814 osc: reduce usage of cdp_cl_pages array
The cdp_cl_pages array is how we are still keeping cl_pages
available, so reducing use of it is the next step.
This is the start of that process, in this case in
osc_queue_dio_pages.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ifd2f5ad63b20df8475f83447a6d9b3d56be3ca70
Patrick Farrell [Sun, 24 Sep 2023 21:00:52 +0000 (17:00 -0400)]
LU-13814 clio: remove last DIO queue usage
With cleanup of osc_dio_submit, we can remove the horrid
'convert to queue' function which converts the cl_dio_pages
array to a cl2queue. There's still a lot left to do to
remove cl_pages for DIO, but this is a nice step.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I4a67cbe8c5e5a3961f5afc593ad0261cb903eabe
Patrick Farrell [Tue, 29 Aug 2023 20:42:38 +0000 (16:42 -0400)]
LU-13814 osc: reduce queue use in __osc_dio_submit
This patch removes another queue use in __osc_dio_submit.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I6f15d4110291cdca3d3c894916ed3c894f2ad9ce
Patrick Farrell [Mon, 4 Sep 2023 23:38:31 +0000 (19:38 -0400)]
LU-13814 osc: begin converting queue_dio_pages
This patch begins the lengthy process of converting
osc_queue_dio_pages to use the page array rather than the
lists. This will be a lengthy process because this ties in
to the OSC extent and BRW code.
Test-Parameters: trivial
Test-Parameters: testlist=sanity-sec env=ONLY=52,59a,59b
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I788faa0748a88045d838fb530107938a639407d0
Patrick Farrell [Tue, 29 Aug 2023 15:37:33 +0000 (11:37 -0400)]
LU-13814 osc: convert osc_dio_submit to array
This is a trivial first step, converting from list to the
array of cl_pages, which is not a very useful step but
does have to be done as we proceed here.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ib208bc2ed2ad95602a5fde79a4b39652ff73d9bf
Patrick Farrell [Mon, 28 Aug 2023 16:12:32 +0000 (12:12 -0400)]
LU-13814 osc: cleanup osc_completion
Removing oap_cmd usage in osc_completion makes it easier to
remove osc page usage for DIO.
There's also no need for the osc_ap_completion wrapper, so
remove it.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ie72eecded73b0444a51b0273bf8913bfaee001bd
Patrick Farrell [Mon, 28 Aug 2023 15:57:48 +0000 (11:57 -0400)]
LU-13814 osc: add osc_queue_dio_pages
For dio, we need to replace osc_queue_sync_pages with a
specialized function. This adds that function and starts
specializing it to DIO only.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I1eb996578cf2fc4d758d959ffc7f9b48225374ce
Patrick Farrell [Sun, 24 Sep 2023 20:59:41 +0000 (16:59 -0400)]
LU-13814 osc: clarify osc_transfer_pinned usage
osc_transfer_pinned is not used for transient pages, so
don't clear it for them either.
This is part of a process of cleaning up DIO page
submission by truly specializing it for DIO pages.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I48f0da5f3cede5af04770c64077bbe7475623881
Patrick Farrell [Sun, 24 Sep 2023 20:59:23 +0000 (16:59 -0400)]
LU-13814 osc: add osc_dio_submit
This function is a coyp of osc_io_submit, specialized to
handle only dio pages. Converting this to handle batch
submission will come later.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Icb6923d22502bbb72af7e8b24b4c9afaba49380b
Patrick Farrell [Sun, 24 Sep 2023 20:59:02 +0000 (16:59 -0400)]
LU-13814 clio: convert lov submit to cl_dio_pages
This patch converts lov layer of IO submit to use the
cl_dio_pages array. This simplifies that code since the
page array is all on one stripe and can only succeed or
fail completely. (No partial successes.)
This pushes the array-to-queue conversion down to the OSC
layer.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ic414218be2be07bb7d74b150faebf5f404760d80
Patrick Farrell [Sun, 24 Sep 2023 20:58:45 +0000 (16:58 -0400)]
LU-13814 clio: rename 'cl_page_completion'
All of the other cl_page operations are verbs, so let's
make this one a verb as well rather than the slightly odd
'completion'.
Test-Parameters: trivial
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I3f52372b4e9ae1805fdfddb31c908553a381c0aa
Patrick Farrell [Sun, 24 Sep 2023 20:58:22 +0000 (16:58 -0400)]
LU-13814 clio: add cl_dio_pages_complete
This is the equivalent of cl_page_complete for regular
cl_pages. This is used in a patch to come shortly.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I9fc993e57d1607a6ad5cba27f896e0eb2a838218
Patrick Farrell [Sun, 24 Sep 2023 20:57:35 +0000 (16:57 -0400)]
LU-13814 clio: add cl_sync_io_note batch
Batched cl_sync_io_note will be needed shortly in the
series, so add the functionality here. This will be used
in an upcoming patch, but is separate to keep the patches
short.
Test-Parameters: trivial
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I5191424a88376e8e9a69e99ad2e785357f35fa0c
Patrick Farrell [Sun, 24 Sep 2023 20:55:56 +0000 (16:55 -0400)]
LU-13814 lov: add lov dio_pages_init
Implement dio_pages_init for the lov layer.
Note this has a place to call to the OSC layer, but there
is no corresponding function there. The OSC function will
be added when the corresponding functionality is needed by
the main conversion process.
This patch allows us to continue the conversion of the
io_submit path.
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ib8db45e89fd699febce4978f2ea95c7500b0325d
Patrick Farrell [Sun, 24 Sep 2023 20:53:37 +0000 (16:53 -0400)]
LU-13814 clio: add cl_dio_pages_init
Just like the cl_page it's replacing, the cl_dio_pages
struct needs various pieces of information from the
different layers of the cl_object in order to do the IO.
This means we need a cl_dio_pages init, analogous to
cl_page_alloc and coo_pages_init.
Note this does not implement coo_pages_init for any layers,
it just moves parts of the existing init code and lays out
the function structure.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I1fcf407b16d4077d94c7ba5afbc63bdd3fb3dfb4
Patrick Farrell [Sun, 24 Sep 2023 20:10:55 +0000 (16:10 -0400)]
LU-13814 lov: remove non-dio handling in lov_dio_submit
lov_dio_submit is now specialized to dio. Cleanup
accordingly.
Test-Parameters: fortestonly
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I4e293011c77eb1b7b8f2c0ec78565458a6fe047c
Patrick Farrell [Sun, 24 Sep 2023 20:10:41 +0000 (16:10 -0400)]
LU-13814 clio: add cio_dio_submit
Add cio_dio_submit method and push it down in to the lov
layer.
Note this part of the series relies on the slightly clever
fact that there is no vvp method for this and the LOV
method calls the OSC method. This lets us do setup and
teardown of the queue in the lov layer.
The next step is to push the queue handling through this
function to the cl_io_submit_rw call here, which may be
non-trivial.
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I3a2a7b7921391be3cc9a157d4351228226016a51
Patrick Farrell [Sun, 17 Sep 2023 20:56:34 +0000 (16:56 -0400)]
LU-13814 clio: Add cl_dio_submit_rw
This pushes the queue handling to the cl_io_submit_rw
level.
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I38ca5ff2f497147e8246ebc43ff109eecbd127bb
Patrick Farrell [Sun, 17 Sep 2023 20:51:33 +0000 (16:51 -0400)]
LU-13814 clio: add cdp_cl_pages tracking
This enables us start the lengthy process of converting the
DIO path from using lists of cl_pages to using a single
cl_dio_pages array.
We add a temporary cdp_cl_pages array tracking the cl_pages
and a function to convert it to a queue. Subsequent
patches will push this through the DIO path, gradually
converting each step to use cl_dio_pages until there are no
steps left which use lists of cl_page.
This moves the queue usage to the cl_dio_pages queue for
ll_direct_rw_pages, the first step.
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: If7cb5cbb01afd96939b05124358345a0d518c180
Patrick Farrell [Sun, 17 Sep 2023 20:38:23 +0000 (16:38 -0400)]
LU-13814 clio: rename pvec to cdp
Pvec is an older naming convention that sort of matches a
convention used in the kernel, but is really starting to
feel odd as a local variable name for cl_dio_pages.
Rename accordingly.
Test-Parameters: trivial
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I375b80e4e50dcb2aa7c640df32a8fe896e5fc082
Patrick Farrell [Sun, 17 Sep 2023 20:36:59 +0000 (16:36 -0400)]
LU-13814 clio: rename ll_dio_pages to cl_dio_pages
ll_dio_pages was originally mostly used in the llite layer,
but its usage is now expanding throughout clio.
Rename it accordingly.
Test-Parameters: trivial
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Id52ded6231510bb16eaf98b08f2eb2146b7ac582
Patrick Farrell [Sun, 17 Sep 2023 20:31:31 +0000 (16:31 -0400)]
LU-13814 llite: add to/from to ll_dio_pages
ll_dio_pages is the main way we package pages for a DIO, so
as we work on removing cl_page for DIO, we need to put some
of the info in this struct.
This puts the 'to, from' page offsets for incomplete pages
in the package. This works because all pages except the
first and the last must be complete (the first and last can
be the same page).
Test-Parameters: trivial
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I0ae4d617cb2f1310ae5981a9f9eb1480d13e1054
Patrick Farrell [Thu, 24 Aug 2023 16:24:54 +0000 (12:24 -0400)]
LU-13814 clio: transient page related cleanups
Add a few more asserts that transient pages are never
passed to certain functions and remove one unused
declaration.
This series has removed a decent bit of code and
bookkeeping activity from the DIO path, even without
getting to the much bigger wins related to removing
cl_page for DIO.
This gets us about a 17% improvement in DIO
performance - note this is a very different node
from the one which gave us 18 GiB/s two years ago.
ior --posix.odirect -w -r -t 256M -b 64G
Without series:
8.5 GiB/s DIO write
9.6 GiB/s DIO read
With series:
10.1 GiB/s DIO write
11 GiB/s DIO read
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ic90bd349a1ded8c1484dc8be29418adb8f229026
Patrick Farrell [Sun, 17 Sep 2023 19:02:00 +0000 (15:02 -0400)]
LU-13814 osc: do not call osc_lru_use for transient
Transient pages are never added to the LRU, because they
can't be cached. osc_lru_use already skips them because
they don't have the flag set, but make it explicit that
this is not called for transient pages.
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I2c92ccb52380faefbcba3bfa35508dac2b601bd4
Patrick Farrell [Sun, 17 Sep 2023 19:01:38 +0000 (15:01 -0400)]
LU-13814 osc: remove "osc_page_transfer_add" wrapper
osc_page_transfer_add is just a wrapped around osc_lru_use,
let's remove it to be more explicit about what we're
actually doing.
Test-Parameters: trivial
Test-Parameters: testlist=sanity-sec env=ONLY=52,59a,59b
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I495a90bee7dc8f8c9d823fa47f9303d2fac2a829
Patrick Farrell [Sun, 17 Sep 2023 19:01:14 +0000 (15:01 -0400)]
LU-13814 clio: remove cl_page_prep for transients
cl_page_prep no longer does anything for transient pages,
finish cleaning that up and make it explicit with asserts.
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I3285a69f971530ba0407de128430bb2497900d11
Patrick Farrell [Sun, 17 Sep 2023 19:00:52 +0000 (15:00 -0400)]
LU-13814 clio: cleanup cl_page_completion
Clean up cl_page_completion and make very explicit which
parts of the function do not apply to transient pages.
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ic288717c8487ff963f0fa7f63a943e72d05d129a
Patrick Farrell [Sun, 17 Sep 2023 18:12:03 +0000 (14:12 -0400)]
LU-13814 llite: note references in direct_rw_pages
Add a comment denoting the function of cl_2queue_fini in
ll_direct_rw_pages. This will eventually be removed as
part of this process, but this comment serves as a remidner
for when we get to that step.
Test-Parameters: trivial
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I98a03c7ee0d97665d77a321bc21b4fab6448b2d7
Patrick Farrell [Sun, 17 Sep 2023 18:11:30 +0000 (14:11 -0400)]
LU-13814 clio: further transient own/disown removal
This patch goes a bit further in removing own/disown for
transient pages, including adding asserts that the code is
not called for transient pages.
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ibad555e50088adc53a66d0e4aeba5558ac2b6a06
Patrick Farrell [Sun, 17 Sep 2023 18:10:23 +0000 (14:10 -0400)]
LU-13814 clio: remove discard for transient pages
With cl_page_delete removed for transient pages, now
cl_page_discard doesn't do anything for transient pages.
Remove it.
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ib62e8a5aea71669428b7b61ba989867702bf1758
Patrick Farrell [Thu, 24 Aug 2023 15:43:53 +0000 (11:43 -0400)]
LU-13814 clio: remove cl_page_delete for transient
cl_page_delete no longer does anything for transient pages,
so do not call it for them.
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I666cceaa1f05bf7e86fb60782eb573d5714051ee
Patrick Farrell [Thu, 24 Aug 2023 15:25:08 +0000 (11:25 -0400)]
LU-13814 osc: Remove osc delete for transient pages
Transient pages do not need an extra reference for being
part of a transfer, because they are referenced throughout
by cl_io. This requires a tweak to the page completion
behavior.
This allows us to remove osc_page_delete for transient
pages.
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I96539731f972b19830b2e08bf0f1d1f1e9674241
Patrick Farrell [Thu, 24 Aug 2023 14:36:57 +0000 (10:36 -0400)]
LU-13814 osc: specialize osc_page_delete
Nearly all of osc_page_delete is only done for cacheable
pages, so make that explicit.
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I998fc196c276aa97829f5b368e23aa4b7a797294
Patrick Farrell [Sun, 17 Sep 2023 18:09:01 +0000 (14:09 -0400)]
LU-13814 osc: add osc transient page ops
As part of gradually removing cl_pages for transient pages,
create a special set of OSC page operations for them.
This makes it easier to see what's left for transient pages
and focus on removing that.
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I20bbe7535e8df223ec1fff9b940b3063fcc3f8d7
Patrick Farrell [Sun, 17 Sep 2023 18:08:11 +0000 (14:08 -0400)]
LU-13814 clio: Remove owner for transient pages
Removing cl_page ownership is another step in removing
cl_page for transient/DIO pages. This disables all of the
ownership related functionality for transient pages.
Test-Parameters: forjanitoronly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I7f1776284d7cd14bdab89290adcc27e3c73416ec
Patrick Farrell [Thu, 24 Aug 2023 02:42:39 +0000 (22:42 -0400)]
LU-13814 clio: add io to cio_submit
cp_owner is going away for transient pages, so we need to
remove its usage to find the top level IO. Here that means
passing the IO down through a few layers.
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I7fac0e53a7831247b846261c1c734c9d6e43a7d2
Patrick Farrell [Thu, 24 Aug 2023 02:28:30 +0000 (22:28 -0400)]
LU-13814 clio: remove cp_state usage for DIO pages
DIO pages don't really go through the state transitions of
cached pages, so don't do it for them. This is a step
towards removing cl_pages for DIO/transient pages.
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I5b4cd52ec19c97d0a5b815fcd68958a108a3444a
Patrick Farrell [Wed, 23 Aug 2023 22:46:53 +0000 (18:46 -0400)]
LU-13814 clio: Improve cl_io_submit_sync comments
Add notes on what cl_io_submit_sync is for.
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I5e32f1a7e6893b63d82f14848a865f90d30fb079
Patrick Farrell [Wed, 23 Aug 2023 22:19:08 +0000 (18:19 -0400)]
LU-13814 osc: Remove most uses of oap_obj
Removing most uses of oap_obj makes it easier to do the
upcoming transient page cl_page removal.
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ic8acaed2ce3c6831f9a0d2bd13d859b9c564efdd
Patrick Farrell [Wed, 23 Aug 2023 19:59:43 +0000 (15:59 -0400)]
LU-13814 osc: Remove oap_request
oap_request isn't actually per-page, it's per requet, so
move it up and associate it with the request async args.
The goal is to shift away from page lists at the RPC level
for DIO.
The first step of this is to move everything that can be
moved from osc_async_page to the osc_brw_async_args level.
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I188039b0abd4b639755dbebfab02597da13d5ddf
Patrick Farrell [Wed, 23 Aug 2023 18:53:53 +0000 (14:53 -0400)]
LU-13814 osc: skip lru_add for transient pages
Transient pages do not go in the LRU, so don't bother
trying to add them.
Test-Parameters: forjanitoronly
Test-Parameters: fortestonly
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I90e3cca2229e1ae7d769c0534b5b6e0be2357ad9
Patrick Farrell [Sun, 17 Sep 2023 17:57:15 +0000 (13:57 -0400)]
LU-13814 llite: refactor ll_direct_rw_pages
ll_direct_rw_pages has some oddities in the control flow,
which make it a little harder to understand. Clean those
up so it's easier to modify.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I64b4639df948556da03824a71b4b30806deced0d
Patrick Farrell [Sun, 17 Sep 2023 18:07:16 +0000 (14:07 -0400)]
LU-17063 osc: remove duplicate info
The from/to in osc_page replicate the info provided by
oap_page_off and oap_count in osc_async_page, so just use
those.
Getting the full benefit of this requires removing the
padding, but with that done, this gets us a full 8 bytes
of size reduction in osc_page (and therefore cl_page)
size.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: If9f2ea5abfe1da6e586072f22c5e0758988b7760
Patrick Farrell [Sun, 17 Sep 2023 18:05:33 +0000 (14:05 -0400)]
LU-13805 llite: make page_list_{add,del} symmetric
An earlier patch created the slightly frightening situation
where we use cl_page_list_del to remove references which
were not taken by cl_page_list_add.
This assymetry is scary, so let's not do it. Instead, DIO
now explicitly puts the only cl_page reference it takes.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I832d8ca7dc7f2f99dc30f972197bebc83b8b5977
Patrick Farrell [Thu, 11 May 2023 20:55:37 +0000 (16:55 -0400)]
LU-13805 llite: wait for partially successful aio
For various reasons (notably conflicting buffered IO), we
may need to fall back from DIO to buffered IO. This also
affects AIO, and if it happens, we will sometimes submit
only part of an AIO with the AIO path, completing the rest
with the buffered path.
Userspace doesn't expect this, expecting us to either do
all or none of the IO with AIO, so it doesn't wait for
completion in this case.
To meet this expectation, we must recognize this case and
wait for AIO to complete before returning to userspace.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Iac7abac3bd01f027c353120483932a62c6475277
Patrick Farrell [Fri, 15 Sep 2023 15:31:24 +0000 (11:31 -0400)]
LU-13805 llite: fail unaligned DIO for RDMA pages
Unaligned DIO needs to directly access the page contents in
order to copy to the buffer. This means it can't work with
RDMA only (non-CPU accessible) pages.
Implement that limitation.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I52bd1d4cc143e1018ddf6942403142f26be4430f
Patrick Farrell [Sun, 21 May 2023 21:37:57 +0000 (17:37 -0400)]
LU-13805 llite: Implement unaligned DIO connect flag
Unupgraded ZFS servers may crash if they received unaligned
DIO, so we need a compat flag and a test to recognize those
servers.
This patch implements that logic.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I5d6ee3fa5dca989c671417f35a981767ee55d6e2
Patrick Farrell [Sun, 21 May 2023 16:48:25 +0000 (12:48 -0400)]
LU-13805 llite: add flag to disable unaligned DIO
As with any new IO feature, it's a good idea to have the
option to turn off unaligned DIO support if needed.
It would be reasonable to merge this patch with the core
patch implementing the feature; I have kept it separate for
ease of review.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ibc86d84704151a7f30afcc538d9c03e3fdf1c38a
Patrick Farrell [Thu, 27 Jul 2023 18:36:30 +0000 (14:36 -0400)]
LU-13805 clio: bounce buffer for unaligned DIO
Direct I/O must normally be page aligned both in terms of
I/O size and memory alignment. This is because the I/O
must be page aligned before being written to disk. This
is also true for buffered I/O, but the I/O is aligned
using the page cache.
In recent versions of Lustre, direct I/O is significantly
faster than buffered I/O, due to lower overhead for page
management. Thus it is desirable to be able to do more
I/O as direct I/O.
This patch allows unaligned direct I/O by creating a buffer
inside the kernel and aligning the I/O by copying in to
this aligned buffer. Because the main cost of buffered I/O
is locking in the page cache rather than memcopy(), this is
still significantly faster than buffered I/O (though slower
than normal direct I/O).
This will eventually allow us to convert buffered I/O to
direct I/O when doing so would increase performance.
Here are some comparative benchmarks using IOR, all single
process.
UDIO is unaligned DIO.
io size 1M 4M 16M 64M
----------------------------------------------------------
BIO Write | 1502 MiB/s | 1382 MiB/s | 1683 MiB/s | 1677 MiB/s
BIO Read | 2169 MiB/s | 1902 MiB/s | 2131 MiB.s | 1955 MiB/s
DIO Write | 1010 MiB/s | 2778 MiB/s | 5905 MiB/s | 7917 MiB/s
DIO Read | 893 MiB/s | 2657 MiB/s | 4724 MiB/s | 7579 MiB/s
UDIO Write | 848 MiB/s | 1666 MiB/s | 2117 MiB/s | 2243 MiB/s
UDIO Read | 933 MIB/s | 2412 MiB/s | 3690 MiB/s | 5370 MiB/s
Unaligned DIO offers benefits vs buffered write and
buffered read, but is of course slower than DIO.
Notice on this node the best case DIO performance is
~8 GiB/s. On a node with 12 GiB/s best case DIO, best case
UDIO read is 8 GiB/s and best case UDIO write is 2.5 GiB/s.
This is because UDIO read is fully parallel, UDIO write is
not.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I7eeebf9a608f006c8095b95f0677adb99f19d640
Patrick Farrell [Tue, 14 Feb 2023 18:47:54 +0000 (13:47 -0500)]
LU-13805 llite: unaligned direct_rw_pages
Add support for ll_direct_rw_pages to handle unaligned
IO by allowing both the first and last page to be partial
pages.
This has been broken off from the main unaligned DIO patch
to make it more reviewable.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I055105589d5416fe6aa82fb6a087db7b8b38c8d1
Timothy Day [Wed, 30 Aug 2023 17:59:20 +0000 (17:59 +0000)]
LU-930 doc: add static analysis documentation
Add more documentation about Clang/LLVM and other
static analysis tools for Lustre. This will make it
easier for other developers to try out various tools.
It will also serve as a place to record best practices
and experiences. Hopefully, this will increase awareness
and usage of these various tools and improve the Lustre
codebase as a result.
This patch also has a few other small doc updates.
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I4bd860775729aaa4ef1ae1cc2cceb6435f3affdd
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52186
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Arshad Hussain [Tue, 29 Aug 2023 07:00:35 +0000 (12:30 +0530)]
LU-16796 mdt: Change struct cdt_agent_req to use kref
This patch changes struct cdt_agent_req to use
kref(refcount_t) instead of atomic_t
Signed-off-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Change-Id: I0a99002504ff453b8b748391f08bd1020c545321
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52148
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Etienne AUJAMES <eaujames@ddn.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Mon, 28 Aug 2023 19:52:04 +0000 (19:52 +0000)]
LU-17058 build: add help and checkpatch as make targets
Add `make help` to print out available make targets. The
output is styled after the Linux kernel `make help`.
Add `make checkpatch` to run checkpatch.pl script
against most recent commit.
Update README to mention `make help`.
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I65ce84040502994ae7caa0c8b72d808442f6b79e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52142
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Thomas Bertschinger [Fri, 25 Aug 2023 16:28:16 +0000 (12:28 -0400)]
LU-13031 tests: skip sanity/test_205h,205i in interop
Skip sanity tests 205h and 205i when the MDS version is too old
to have the jobid xattr changes. Fix test 103a to not try to set
the job_xattr parameter when it does not exist.
Fixes:
23a2db28dcf1 ("LU-13031 jobstats: store jobid in xattr when files are created")
Test-Parameters: trivial testlist=sanity env=ONLY="103a 205h 205i"
Signed-off-by: Thomas Bertschinger <bertschinger@lanl.gov>
Change-Id: Iaa5d0c1a7f3fa6769fab4340ade315e7a49df009
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52095
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Jian Yu [Wed, 23 Aug 2023 19:05:22 +0000 (12:05 -0700)]
LU-16831 lod: replace (__u16)-1 with LOV_ALL_STRIPES
This patch replaces "(__u16)-1" with constant LOV_ALL_STRIPES
and replaces "(__u64)-1" with OBD_OBJECT_EOF.
Test-Parameters: trivial
Change-Id: I2345c5e67da20328e7173c9add8da27015df9d13
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52055
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Zhenyu Xu <bobijam@hotmail.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Bobi Jam [Thu, 17 Aug 2023 17:00:03 +0000 (01:00 +0800)]
LU-10026 csdc: DoM pattern could be a combined value
DoM pattern is LOV_PATTERN_MDT for now, and in the future it could
be combined with LOV_PATTERN_COMPRESS to represent a compressed
DoM component.
Fix a minor glitch for lov_getstripe_old code path (in
ll_lov_getstripe_ea_info), which intends to return the last component
stripe info but the commit
abf04e7ea3 omits to correctly set the
last component stripe info before using it.
Fixes:
abf04e7ea3 ("LU-14337 lov: return valid stripe_count/size for PFL files")
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Id0779c30c004b6979f88bf96b7b7b74a8b8c26e4
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51978
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Sebastien Buisson [Thu, 10 Aug 2023 11:05:52 +0000 (13:05 +0200)]
LU-17023 krb: use a Kerberos realm different from default
It makes sense to give the ability to specify a Kerberos realm that is
different from the default realm as returned by
krb5_get_default_realm().
On client side, the desired realm needs to be specified via the new
'-R' option to lgss_keyring. This can be specified in the config file
/etc/request-key.d/lgssc.conf to replace the default domain, e.g.:
create lgssc * * /usr/sbin/lgss_keyring -R DOMAIN.COM %o %k %t %d %c %u %g %T %P %S
On server side, the desired realm can be specified via the new '-R'
parameter of the lsvcgssd daemon, replacing the default realm.
This patch adds sanity-krb5 test_1b to exercise the new realm options,
by just re-using the same realm as the test system is configured to
use. And former test_1 is renamed test_1a.
Test-Parameters: kerberos=true testlist=sanity-krb5
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I9c91d5cb9904781d546e77b1e46115fed433618f
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51914
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Thu, 24 Aug 2023 09:40:46 +0000 (11:40 +0200)]
LU-17050 tests: test Kerberos env in sanity-krb5
Test Kerberos environnement is sane before trying to launch
sanity-krb5 tests.
Test-Parameters: trivial kerberos=true testlist=sanity-krb5
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: I1675ba7db8c62687c69359a15cc931b5dfd40018
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52068
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Sat, 5 Aug 2023 19:12:15 +0000 (19:12 +0000)]
LU-17000 lnet: fix various bugs in lib-move.c
In lnet_select_peer_ni, best_lpni_is_preferred is often written
to before being immediately overwritten.
Addresses-Coverity-ID: 397646 ("Unused value")
Addresses-Coverity-ID: 397434 ("Unused value")
Both LNetPut and LNetGet were not freeing msg under certain failure
conditions. This leaks a small amount of memory each time it occurs.
Addresses-Coverity-ID: 397644 ("Resource leak")
Addresses-Coverity-ID: 397133 ("Resource leak")
Fix potential null dereference in lnet_find_best_ni_on_local_net
when best_lp gets defined by best_net doesn't.
Addresses-Coverity-ID: 397568 ("Explicit null dereferenced")
lnet_post_send_locked has an un-needed null check, since every path
leading to that block of code must dereference ni anyway.
Addresses-Coverity-ID: 397278 ("Dereference before null check")
In the other usage of msg_peerrtrcredit, it is accessed under
lpni_lock. Change the second usage to also be accessed under this
lock.
Addresses-Coverity-ID: 397606 ("Data race condition")
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I4012f407b10d0c9644535d49cce83a6c95d3d22d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51876
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Hongchao Zhang [Mon, 14 Aug 2023 07:28:17 +0000 (15:28 +0800)]
LU-16988 mdd: update projid when merging layout
When creating mirrors by the special directory ".lustre/fid",
the project ID could not be set correctly, which causes
wrong quota calculation for the projid.
Signed-off-by: Hongchao Zhang <hongchao@whamcloud.com>
Change-Id: Ia4c3a8973b8c467642e12629d36fa42d64162084
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51859
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Sergey Cheremencev <scherementsev@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Alexandre Ioffe [Tue, 25 Jul 2023 02:08:26 +0000 (19:08 -0700)]
LU-16977 utils: access_log_reader accesses beyond batch array
Fixed access_log_reader accesses sorted batch array beyond upper
boundary when batch-fraction 100%: consider fraction = 100% as a
special case, which requires no sorting and filtering.
Use a separate thread function to process 100% fraction case.
Made some minor changes using enum type to nicefy the code.
Test-Parameters: trivial testlist=sanity
Signed-off-by: Alexandre Ioffe <aioffe@ddn.com>
Change-Id: Iba1734b17dc901875f343c793688aec17b9f7a93
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51754
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Fri, 12 May 2023 04:32:53 +0000 (04:32 +0000)]
LU-16763 obdclass: add unit tests for OBD life cycle
Add some simple OBD life cycle tests. These tests
consist of a kernel module which defines a simple OBD
device, and a few sanity tests. The new OBD device
print logs validating that it has been loaded
correctly. Unlike other OBD devices, this one has
minimal side-effects. The new test OBD device has
been added to the test rpm and dkms.
sanity/55a aims to test that a device can loaded
properly and found by the various OBD device search
functions.
sanity/55b aims to load the maximum number of allowed
OBD devices, which is currently 8192. It also times how
long it takes to perform the loading and unloading. In
the future, this could be used to test for performance
regression.
The tests avoid using any userspace function, like lctl
or lfs, since I noticed bugs when using them with a large
number of devices. Follow-up patches will include fixes
and more testing.
I used a variation of these tests when debugging
sanity/60a failures, and when debugging removing
MAX_OBD_DEVICES.
This test (obd_test.c) and the llog test (llog_test.c)
should probably be moved to a different directory in a
follow-up patch.
Test-Parameters: trivial testlist=sanity env=ONLY=55,ONLY_REPEAT=25
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: Ibc347ac962c59a4bbc26410c30f9cc5529e6c84d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51103
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Fri, 25 Aug 2023 22:42:23 +0000 (18:42 -0400)]
LU-14111 tests: only support recovery-small test 146 for 2.15.54+
If you running newer clients with older servers (2.15.3) then
recovery-small test 146 will fail since the old servers lack
the new sysfs file eviction_count.
Fixes:
3c69d46e176 ("LU-14111 obdclass: count eviction per obd_device")
Test-Parameters: trivial testlist=recovery-small env=ONLY=146 mdsversion=2.15.3
Change-Id: I53f6dabd305ec920e8de1d9fde407b2f2c15ba69
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52098
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Shaun Tancheff [Thu, 24 Aug 2023 09:14:09 +0000 (04:14 -0500)]
LU-16314 llite: Prefer %pK with seq_printf
Update procfs and sysfs users to prefer %pK to when
printing pointers so that when kptr_restrict is set to 1
a real pointer value is provided.
To enable printing non-hashed pointer values:
sysctl -w kernel/kptr_restrict=1
This change also sets kptr_restrict to 1 for all clients
and servers under test by test framework.
Test-Parameters: trivial
HPE-bug-id: LUS-10945
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Iccfce1399648e752cb7b78afc75aacbfb0bde390
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51212
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Vladimir Saveliev [Wed, 26 Jul 2023 13:09:18 +0000 (16:09 +0300)]
LU-16043 osc: allow error for write on CL_FSYNC_DISCARD
If case of CL_FSYNC_DISCARD error is allowed for write of osc object.
Otherwise, the included test fails in rm with:
(osc_page.c:174:osc_page_delete()) Trying to teardown failed: -16
(osc_page.c:175:osc_page_delete()) ASSERTION( 0 ) failed:
(osc_page.c:175:osc_page_delete()) LBUG
Test-Parameters: trivial testlist=sanity env=ONLY=907
HPE-bug-id: LUS-10410
Signed-off-by: Vladimir Saveliev <vladimir.saveliev@hpe.com>
Change-Id: I0aae0dc470ba0371964e7643a6d84b19a1b4e106
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48032
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Patrick Farrell [Wed, 30 Aug 2023 20:05:07 +0000 (16:05 -0400)]
LU-15619 osc: Rename brw_page members
The brw_page members have generic names - add a structure
related tag to the names.
test-parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I98e6f874902074934eb01476a9595f502526bc38
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/46715
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
James Simmons [Mon, 28 Aug 2023 14:03:03 +0000 (10:03 -0400)]
LU-8130 osc: convert osc_quota hash to xarray
The cl_quota_hash originally had 3 hashes, one for each type of quota
(USR, GRP, PRJ) that just stored on the client whether a particular
quota ID was over its limit. This was overkill since cl_quota_hash
only needs one bit to check if a particular ID has exceeded quota
with IO from this client, and there will usually be only a few IDs
that are actually exceeding their limit where a client is involved.
Instead, use the quota ID as the index into an Xarray, and store
a value with the quota TYPE(s) that are over the limit for that ID.
We only need to test the presence/absence of an ID and a quota type
without the need to store any additional values (the clients do not
track the actual quota usage or limits).
To test if a quota is exceeded for particular ID is a two-step
process. First check if there is any entry for the particular ID,
and if it exists then check which quota type (USR, GRP, PRJ) is
over the limit for that ID value. The same is done when setting
a particular quota ID/TYPE is over its limit - first lookup the
ID and then add the TYPE flag to the value if not already set.
The Xarray implementation does offer using "marks" (up to 3 bits
per index) but in this case there is no other value that needs to
be stored into the Xarray other than one bit for any exceeded type,
so they are not used here.
Change-Id: I9355ed2a7158f0d5cc0d600ad51ea1a1434f3e98
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/32038
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Xinliang Liu [Fri, 25 Aug 2023 03:24:12 +0000 (03:24 +0000)]
LU-17052 libcfs: fix build for old kernel
Fix build for kernel v4.17 to v4.19.
These old kernels already have xarray.h and #include by fs.h but
don't have full xarray support. It is needed to #include libcfs's
xarray.h also to contain xarray support.
Rename the header define macro to ensure libcfs's xarray.h will be
included。
Test-Parameters: trivial
Test-Parameters: testlist=sanityn envdefinitions=ONLY=77,ONLY_REPEAT=20
Fixes:
778791dd7da1 ("LU-8130 libcfs: don't use radix tree for xarray")
Change-Id: I760c394cc1d885c2de79d1770243ab7f292b9b3a
Signed-off-by: Xinliang Liu <xinliang.liu@linaro.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52090
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Alexander Boyko [Fri, 7 Jul 2023 19:35:51 +0000 (15:35 -0400)]
LU-16973 osd: adds SB_KERNMOUNT flag
During umount mntput() is called. It uses delayed_mntput()
function, and it could take much time to finish. A block
device is occupied during delayed work.
[ 8753.941980] Lustre: server umount XXX complete
[ 8800.129136] sysrq: SysRq : Trigger a crash
PID: 319306 TASK:XXXX CPU: 2 COMMAND: "kworker/2:0"
#0 __schedule at
ffffffff9754e1d4
#1 preempt_schedule_common at
ffffffff9754e6fa
#2 _cond_resched at
ffffffff9754e72d
#3 invalidate_mapping_pages at
ffffffff96e72da5
#4 invalidate_bdev at
ffffffff96f5d13c
#5 ldiskfs_put_super at
ffffffffc1c82e34 [ldiskfs]
#6 generic_shutdown_super at
ffffffff96f1bdcc
#7 kill_block_super at
ffffffff96f1bed1
#8 deactivate_locked_super at
ffffffff96f1b784
#9 cleanup_mnt at
ffffffff96f3b86b
Let's use SB_KERNMOUNT flag during mount, it leads to
synchronous mntput().
It also calls flush_delayed_fput during umount to finish
delayed fput.
HPE-bug-id: LUS-11629
Signed-off-by: Alexander Boyko <alexander.boyko@hpe.com>
Change-Id: Ia6729f6cbac85c3626562e946a4b96665a143714
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51731
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andrew Perepechko <andrew.perepechko@hpe.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Sat, 18 Mar 2023 00:34:06 +0000 (18:34 -0600)]
LU-12064 ptlrpc: set at_min=5 by default
Having at_min=0 as the default value can result in clients timing
out and/or being evicted too easily when there is a sudden spike
in server load. Increase at_min to 5s by default.
For large clusters, at_min=15 is more reasonable, but distributing
a variable at_min value to clients will need more complex changes.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I3463cbc642458f6dd5977fe34478b135d1cd0219
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50609
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Aurelien Degremont <adegremont@nvidia.com>
Patrick Farrell [Wed, 8 Feb 2023 19:01:24 +0000 (14:01 -0500)]
LU-13805 llite: add mm to dio struct
When copying to or from userspace, we must use the mm from
the userspace thread. This can be done either by running
in that thread or borrowing its mm. Unaligned DIO does
some memory movement to userspace in ptlrpcd threads, so it
requires the user mm be stored in the sub dio.
This will be used by the main unaligned DIO patch and has
been split out for reviewability.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I419cb9f1899b8c8f9790ce25b3aba1d6f07397aa
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49947
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Zhenyu Xu <bobijam@hotmail.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Patrick Farrell [Mon, 6 Feb 2023 18:10:10 +0000 (13:10 -0500)]
LU-13805 clio: Add csi_complete
The next patch will make end_io potentially sleep, so we
need to modify how completion works to avoid holding a
spinlock over the end_io() call.
This patch is strictly supporting work for the next patch
and has been pulled out so it can be tested by itself.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Iba3388a0e09fdd0ab2f4a95f1cde96908a485cfa
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49913
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Oleg Drokin [Fri, 1 Sep 2023 20:38:40 +0000 (16:38 -0400)]
New tag 2.15.58
Change-Id: I6d58a43d5904c24d32575b4790bcaabd9ebdfb6f
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Wed, 23 Aug 2023 16:26:41 +0000 (16:26 +0000)]
LU-17038 tests: remove unused compile.sh script
This script just runs make automatically. It doesn't
appear to be called by any other Lustre sanity
test script. I doubt it has been used in many
years. This patch removes it.
Checked for usage using:
`git grep -i "compile.sh"`
Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: If1615196bc8d004a63ad8baddd1d3fe3e360dc74
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52054
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Timothy Day [Wed, 23 Aug 2023 02:52:32 +0000 (02:52 +0000)]
LU-17038 tests: remove mlink utility
The mlink utility is nearly identical to the link utility
provided by coreutils. They only differ by some GNU
boilerplate. All tests using mlink are replaced with link.
Luckily, mlink is only used in a few places.
Used the following command:
`git grep -i mlink | grep -i -v symlink`
to track down all uses of mlink.
Test-Parameters: trivial testlist=recovery-small
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I197235572d2cb267ee68930c64058e4f5ffe5be1
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52051
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Mr NeilBrown [Wed, 23 Aug 2023 00:18:41 +0000 (20:18 -0400)]
LU-12678 lnet: discard lnet_kvaddr_to_page
This function is not needed, so discard it.
Change-Id: Iffe9745adf477a5f4b78d8ef191849179426cb07
Signed-off-by: Mr NeilBrown <neilb@suse.de>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52041
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Sebastien Buisson [Mon, 21 Aug 2023 09:44:32 +0000 (11:44 +0200)]
LU-17043 enc: fix osd lookup cache for long encrypted names
Fix osd lookup cache to support files with long encrypted names.
Those encrypted names can be up to 256 bytes, not NUL terminated.
Fixes:
29f8eb2a67 ("LU-16405 osd: lookup cache")
Signed-off-by: Sebastien Buisson <sbuisson@ddn.com>
Change-Id: Ica2329c8a0990395307a14fe9bb9d43db3b364ed
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52016
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Patrick Farrell [Fri, 18 Aug 2023 18:31:32 +0000 (14:31 -0400)]
LU-15367 llite: iotrace standardization
Clean up and standardize some of the iotrace messages for
easier parsing.
Add a clear 'START' indicator.
Remove a now-redundant debug message in the mmap code.
Test-Parameters: trivial
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: Ia620cc8c783509cbc3f47b21a274d67d860b80e7
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52002
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Shaun Tancheff [Fri, 18 Aug 2023 04:50:56 +0000 (23:50 -0500)]
LU-17039 build: cleanup ib_dma_map_sg
CONFIG_INFINIBAND_VIRT_DMA is a kernel configuration option
that in some cases conflicts with the configuration of the
externally provided OFED stack.
During configure when ib_dma_map_sg fails to build correctly
we can simply #undef CONFIG_INFINIBAND_VIRT_DMA to resolve
the inconsistent configuration that breaks ib_dma_map_sg
Test-Parameters: trivial
HPE-bug-id: LUS-11771
Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Change-Id: Id0849464d3ffbd573cac13016191d80c6ea991af
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51979
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Andreas Dilger [Thu, 17 Aug 2023 22:06:36 +0000 (16:06 -0600)]
LU-17038 tests: remove munlink utility
The munlink utility is obsoleted by the unlink command added in
the coreutils package many moons ago, and can be removed. All
tests using munlink are replaced with unlink.
Test-Parameters: trivial testlist=recovery-small,replay-dual,replay-single
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I984406525ed958814bd8af74a2d81c4920e320b0
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51977
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Jian Yu [Thu, 17 Aug 2023 20:46:38 +0000 (13:46 -0700)]
LU-16510 build: check if CONFIG_FORTIFY_SOURCE is defined
The linux/fortify-string.h header file should not be
included while the kernel config option CONFIG_FORTIFY_SOURCE
is not defined.
Change-Id: I2e1905406e892b182f143d512a2d3722b141e52d
Fixes:
919b93b951d4 ("LU-16510 build: fortified memcpy from linux 6.1")
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51973
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Li Dongyang [Thu, 17 Aug 2023 13:27:00 +0000 (23:27 +1000)]
LU-17036 utils: make sure resize option is legit
To align the metadata on 1MB boundaries we manually
set the resize blocks to 16368G for 4K block size,
however mke2fs expects the resize blocks is bigger
than device size.
For devices between 16368G and 16384G the mke2fs
will fail with:
The resize maximum must be greater than the filesystem size.
Change-Id: I4567a79c1405e9527d7f0f9bec4c8a7aae0eba6c
Test-Parameters: trivial
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51970
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Emoly Liu <emoly@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Qian Yingjin [Wed, 16 Aug 2023 02:11:39 +0000 (22:11 -0400)]
LU-17031 build: fix refefine __compiletime_strlen error
Lustre build failed on Ubuntu 2204 kernel v5.17 with "redefine
__compiletime_strlen".
This patch fixes this build error.
Fixes:
919b93b951 ("LU-16510 build: fortified memcpy from linux 6.1")
Signed-off-by: Qian Yingjin <qian@ddn.com>
Change-Id: Ic26daecd6b91614e01b5b0030f40eede205a21f7
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51953
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <jsimmons@infradead.org>
Reviewed-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Patrick Farrell [Tue, 15 Aug 2023 23:08:12 +0000 (19:08 -0400)]
LU-17030 llite: allow setting max_cached_mb to a %
Lustre's max_cached_mb parameter is hard to use because it
must be set to a specific numeric value, so in effect it
cannot be set on the server side unless all clients are
guaranteed identical.
Let's add the ability to set that to a % of memory to make
it more useful.
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I1f9f5a8a5d671ab00b7ab6133bb9b1d1214ca59e
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51952
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Qian Yingjin <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Laura Hild [Tue, 15 Aug 2023 17:04:37 +0000 (13:04 -0400)]
LU-10885 docs: note flock now being enabled by default
mount -o flock was made the default, but the mount.lustre(8) man-page
still said noflock is default. Text based on comments in LU-10885 and
http://wiki.lustre.org/Mounting_a_Lustre_File_System_on_Client_Nodes.
Signed-off-by: Laura Hild <lsh@jlab.org>
Change-Id: I48bfc0260fb948771f5cf4fb8cbc6ee9588e2217
Test-Parameters: trivial
Fixes:
16fb13eb3863 ("LU-10885 llite: enable flock mount option by default")
Fixes:
3613af3e15cb ("LU-10885 llite: enable flock mount option by default")
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51948
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Aurelien Degremont [Tue, 15 Aug 2023 14:03:07 +0000 (16:03 +0200)]
LU-17015 gss: support large kerberos token on client
If the current Kerberos setup is using large token, like
when PAC feature is enabled for Kerberos, client can crash.
Return an error instead of asserting to avoid the crash
and increase the default buffer size to 4kB instead of 1kB.
This will only increase the SEC_CTX_INIT request size, and
the buffer is shrunk before being sent over the wire.
This will allow security token up to 2kB to be properly
handled by Lustre. Above that size, a different issue will
happen on server side that will require another patch.
Test-Parameters: trivial kerberos=true testlist=sanity-krb5
Signed-off-by: Aurelien Degremont <adegremont@nvidia.com>
Change-Id: I9ce30ee7f8c95bfe41525c49986ffac45ffac97c
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51946
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Serguei Smirnov [Fri, 11 Aug 2023 00:58:11 +0000 (17:58 -0700)]
LU-17006 lnet: set up routes for going across subnets
Modify ksocklnd-config to set up route which features
default gateway for the subnet in case if default gateway
is defined, for example:
ip route add default via <gw_for_eth0> dev eth0 table eth0
which results in a route similar to the following added to
the eth0 route table:
default via <gw_for_eth0> dev eth0
If there's no gateway found for the eth0 subnet, keep the old
behaviour which results in the following added to eth0
route table:
<eth0_subnet> dev eth0 proto kernel scope link src <eth0_ip>
This makes sure that MR traffic goes out the intended interface
as selected by LNet no matter whether going across subnets or not.
Test-Parameters: trivial testlist=sanity-lnet
Signed-off-by: Serguei Smirnov <ssmirnov@whamcloud.com>
Change-Id: I84a299c8b7eb4cdb4fc24408a1e42ad0283d9219
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51921
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Cyril Bordage <cbordage@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Thomas Bertschinger [Thu, 13 Jul 2023 22:32:52 +0000 (18:32 -0400)]
LU-16766 obdclass: trim kernel thread names in jobids
When collecting jobstats on operations coming from kernel threads, it
is more useful and reduces the noisiness of the data if the names of
kernel threads are trimmed so that all "kworker/CPU:ID" threads are
collected under "kworker", all "ll_sa_PID" threads under ll_sa, etc.
Signed-off-by: Thomas Bertschinger <bertschinger@lanl.gov>
Change-Id: Icd82a99c1153de0277ea5ed3f4b1d92535809921
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51919
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
git://git.whamcloud.com / fs / lustre-release.git / log