Whamcloud - gitweb
Andreas Dilger [Tue, 9 Feb 2021 18:06:45 +0000 (11:06 -0700)]
build: update version to 1.45.6.wc5
4aea203f LU-5949 e2fsck: call delete_inode() properly
Change-Id: Iac0043985f099248f6a53ea0df2cfe9c22ba18f6
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Tue, 9 Feb 2021 07:46:23 +0000 (00:46 -0700)]
LU-5949 e2fsck: call delete_inode() properly
Fix incorrect argument for delete_inode() in pass1b().
Fixes:
8725134d2757 ("LU-5949 e2fsck: simplify inode badness handling")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ib90d87055e2c9359005034d06d88b083423ebbe5
Reviewed-on: https://review.whamcloud.com/41450
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Artem Blagodarenko <artem.blagodarenko@hpe.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Mon, 8 Feb 2021 11:40:40 +0000 (04:40 -0700)]
build: update version to 1.45.6.wc4
8725134d LU-5949 e2fsck: simplify inode badness handling
71b74579 LU-14345 e2fsck: fix check of directories over 4GB
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Thu, 30 Jul 2020 20:36:49 +0000 (14:36 -0600)]
LU-5949 e2fsck: simplify inode badness handling
Move the badness increment into fix_problem() so that code does
not need to be spread across the code and explicitly set. Instead,
take advantage of the fact that pctx->ino is almost always pointing
at the bad inode and increment badness whenever fix_problem() is
called on the inode. This also handles new problems in the future.
That leaves only a handful of places in the code that need special
handling to either set a higher badness (e.g. inode blocks that are
directly referencing filesystem metadata or have wildly incorrect
timestamps), or should not contribute to inode badness at all.
The main exceptions are when reprocessing duplicate blocks in the
inode (badness already set), and cleaning bad entries in a directory
leaf block in check_dir_block(), which uses pctx->ino to reference
the parent directory inode number. In such cases, fix_problem_bad()
is called with 0 badness, since the leaf block was been repaired,
and clearing the directory inode in this case is unnecessary.
This patch should be merged into original badness patch on rebase.
Fixes:
92a85a3ced2c ("e2fsck: track errors/badness found for each inode")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I96cd21b5976991f0bb1c63fc99857c80e23ebbe5
Reviewed-on: https://review.whamcloud.com/41328
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Artem Blagodarenko <artem.blagodarenko@hpe.com>
Andreas Dilger [Mon, 1 Feb 2021 21:55:06 +0000 (14:55 -0700)]
LU-14345 e2fsck: fix check of directories over 4GB
If directories grow larger than 4GB in size with the large_dir
feature, e2fsck will consider them to be corrupted and clear
the high bits of the size. Previous checking appears to have
been done only beyond the prior 2GB directory size limit.
Since it isn't very common to have directories this large, and
unlike sparse files that don't have any ill effect if the size
is wrong, an overly-large directory will have all of the sparse
blocks filled in by e2fsck, so such directories should still
be viewed with suspicion. Check for consistency between two of
the three block count, inode size, and superblock large_dir flag
before deciding whether the directory should be fixed or cleared,
or if the large_dir feature should be set in the superblock.
Update the f_recnect_bad test case to match new output.
Fixes:
49f28a06b738 ("e2fsck: allow to check >2GB sized directory")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I1b898cdab95d239ba1a7b37eb96255acadce7057
Reviewed-on: https://review.whamcloud.com/41385
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Reviewed-by: Artem Blagodarenko <artem.blagodarenko@hpe.com>
Andreas Dilger [Fri, 4 Dec 2020 23:30:23 +0000 (16:30 -0700)]
LU-11848 build: update version to 1.45.6.wc3
23547cbe57df LU-14126 e2fsck: update mmp block race
0b6718f5e22d LU-11848 filefrag: support PFL and FLR file
233fe52cade6 LU-11848 filefrag: add -V option to print version
d99d208e73cd LU-11848 filefrag: rework port PFL and FLR file
3bdf8ca2e409 LU-11848 filefrag: further PFL and FLR fixes
502f2e17bd5b LU-13797 e2fsck: quiet 'extent tree could be narrower'
94318cef3508 misc: replace remaining loff_t with ext2_loff_t
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Iba4a2f2dc747d376b37bb115a6b0539a0d3ebbe5
Reviewed-on: https://review.whamcloud.com/40874
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Sat, 12 Dec 2020 09:11:07 +0000 (02:11 -0700)]
misc: replace remaining loff_t with ext2_loff_t
Replace the remaining loff_t uses with ext2_loff_t, as
was done in patch
1df6a4555, since loff_t is a GCC'ism
and is not portable.
Change-Id: I92e3643c05021a18f40a9a022bbf7e54121e453a
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40951
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Andreas Dilger [Sat, 12 Dec 2020 10:17:18 +0000 (03:17 -0700)]
LU-13797 e2fsck: quiet 'extent tree could be narrower'
The "extent tree (at level 1) could be narrower" message is
overly verbose and raises concerns by end users, even though
it is harmless. On the flip side, this may save only a few
hundred blocks in the filesystem for a short period of time,
so there is relatively little benefit to be had.
Disable the extent optimization step in e2fsck by default by
adding the "no_optimize_extents" option to e2fsck.conf.
Change-Id: I0cee9407ef0653c3b946908513f3429816b89548
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40952
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Andreas Dilger [Thu, 10 Dec 2020 11:04:56 +0000 (04:04 -0700)]
LU-11848 filefrag: further PFL and FLR fixes
Revert some of the changes from the earlier patches to bring the
filefrag functionality closer to the original behavior.
The kernel is instead changed to return the device number with
every extent. This allows the previous filefrag to work for PFL
and FLR files without any changes by just copying the fe_device
from the last returned extent to the next call. The only gap in
old filefrag is that it prints the stripe number with OST index.
Fixes:
d99d208e73cd ("LU-11848 filefrag: rework port PFL and FLR file")
Fixes:
0b6718f5e22d ("filefrag: support PFL and FLR file")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie6a8d93ebceb1e70894b21f0dadc9655083ebbe5
Reviewed-on: https://review.whamcloud.com/40932
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Bobi Jam [Mon, 7 Dec 2020 08:57:10 +0000 (16:57 +0800)]
LU-11848 filefrag: rework port PFL and FLR file
Lustre uses the high 16bits of fiemap_extent:fe_device to record
the absolute stripe number is under process, so that next fiemap
call continues from that stripe.
Fixes:
0b6718f5e22d ("filefrag: support PFL and FLR file")
Change-Id: I111bcb100307aebacbffed99109057accdc27f5b
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40889
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Fri, 4 Dec 2020 22:11:07 +0000 (15:11 -0700)]
LU-11848 filefrag: add -V option to print version
Add '-V' to filefrag to print the installed version of the tool.
If '-V' is used twice, print out the list of supported FIEMAP flags.
This can be used to check if filefrag understands a specific feature.
Change-Id: Ib126bdd70efa1775aef6db761f54e27a593ebbe5
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40873
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Bobi Jam [Thu, 26 Nov 2020 06:29:30 +0000 (14:29 +0800)]
filefrag: support PFL and FLR file
Add FIEMAP_EXTENT_DATA_MIRROR support FLR file, which is returned
from Lustre indicating that a new mirror will be processed, so we'd
reset the fm_start and fe_logical.
Change-Id: Ifa68d951ac68d9f141eae915e58c13d73833c8c9
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40765
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Sun, 8 Nov 2020 09:19:06 +0000 (17:19 +0800)]
LU-14126 e2fsck: update mmp block race
Update mmp block is only expected in one thread, @mmp_update_thread
is used to get/set active thread number, however this should
be set globally shared with different threads rather than be private
per thread.
Fixes:
7284939c57618 ("LU-8465 e2fsck: update mmp block in one thread")
Change-Id: Id3c4ed80003cf9caecb8814ac7b03fabeb83c8bd
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/40572
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Andreas Dilger [Tue, 29 Sep 2020 03:57:13 +0000 (21:57 -0600)]
LU-8465 build: update version to 1.45.6-wc2
Pfsck is supported since this release
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: If28f68206e97b3eddbbc493104968347733ebbe5
Reviewed-on: https://review.whamcloud.com/40081
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Mon, 28 Sep 2020 19:21:16 +0000 (13:21 -0600)]
LU-8465 e2fsck: misc cleanups for pfsck
Add -m option description to e2fsck.8 man page.
Rename e2fsck_struct fs_num_threads to pfs_num_threads to avoid
confusion with the ext2_filsys fs_num_threads field, and move
thread_info to be together with the other CONFIG_PFSCK fields.
Move ext2_filsys fs_num_threads to fit into the __u16 "pad" field
to avoid consuming one of the few remaining __u32 reserved fields.
Fix a few print format warnings.
Change-Id: Ie69fb51642e71ca5d0d242d9cdc12d29ff181f52
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40075
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Mon, 28 Sep 2020 08:04:57 +0000 (16:04 +0800)]
LU-8465 ext2fs: fix to set tail flags with pfsck enabled
If any of block/inode bitmap block checksum error happen,
tail flag should be set properly.
However, we firstly set tail flags in each thread, after
threads finish we clear those tail problem wrongly.
This will make fsck miss bitmap checksum erors later,
patch try to fix the problem by move all this kind of
logic in read_bitmaps_range_end()
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Ie605f4475d35f9ce7d040036d97988c6512cb8c9
Reviewed-on: https://review.whamcloud.com/40071
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Mon, 28 Sep 2020 02:27:12 +0000 (10:27 +0800)]
LU-8465 e2fsck: fix memory leaks with pfsck enabled
valgrind detected few memory leaks:
1) quota context is not released after merging.
2) three block bufs are not freed in read_bitmaps_range_start()
3) @refcount_orig should be released
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I13e51c31cacba9ed14bc9f4d4ce82ec0a6cfe610
Reviewed-on: https://review.whamcloud.com/40069
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Sun, 27 Sep 2020 13:25:07 +0000 (21:25 +0800)]
LU-8465 e2fsck: fix readahead for pass1 without pfsck
If admin try fsck without -m option, codes try old
behavior, thread information won't be inited either.
@et_group_end is 0 thus readahead for pass1 will be
totally disabled.
With the patch applied, we could get same performance
number without pfsck as before.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I3f6158937fb83490243f34693f631a2d7a8f9e4c
Reviewed-on: https://review.whamcloud.com/40066
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Sun, 27 Sep 2020 05:01:13 +0000 (13:01 +0800)]
LU-8465 e2fsck: fix race in ext2fs_read_bitmaps()
During corruption testing hiting following segfault:
Multiple threads triggered to read bitmaps
Signal (11) SIGSEGV si_code=SEGV_MAPERR fault addr=0x200
./e2fsck[0x4382ae]
/lib64/libpthread.so.0(+0x14b20)[0x7f5854d2fb20]
./e2fsck(ext2fs_rb_insert_color+0xc)[0x46ac0c]
./e2fsck[0x467bb4]
./e2fsck[0x467e6d]
./e2fsck[0x45ba95]
./e2fsck[0x45c124]
/lib64/libpthread.so.0(+0x94e2)[0x7f5854d244e2]
/lib64/libc.so.6(clone+0x43)[0x7f5854beb6c3]
Problem is @block_map might be set NULL if one of
thread exit, move such kind of cleanup operation
to main thread after all threads exit.
Another potential problem is e2fsck_read_bitmap()
could be called during pass1, this need be serialized,
serialize it in the pass1.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I6cf2f4a11f3cf04bfa70d1e1f8b97fbcc33b79dc
Reviewed-on: https://review.whamcloud.com/40065
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Sun, 27 Sep 2020 17:26:42 +0000 (11:26 -0600)]
LU-8465 e2fsck: fix f_multithread_ok test
Don't use $OUT for both the input amd output of a pipeline,
as the output file is truncated befor the input is read.
Fix the handling in the failure case to generate the
*.failed file, and print the actual $test_name instead
of "test_name".
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I66513a0422a0e70fd7a887b215e3a4e6c9c250e5
Reviewed-on: https://review.whamcloud.com/40068
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Wang Shilong [Wed, 23 Sep 2020 15:58:23 +0000 (23:58 +0800)]
LU-8465 test: add pfsck test
pfsck run on a clean fs should not return any errors.
Generate an image with possible features enabled,
especially EA shared blocks etc.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Ie61a76729338370a630b5bccf279074286b02b90
Reviewed-on: https://review.whamcloud.com/40017
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Thu, 24 Sep 2020 00:16:13 +0000 (08:16 +0800)]
LU-8465 configure: enable pfsck by default
Since most of work has been done, compile e2fsprogs
with pfsck enabled by default.
So it could testing widely now.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I7fe31e0d8161132ea1717f0d7e5f4fe0db432fec
Reviewed-on: https://review.whamcloud.com/40021
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Sat, 26 Sep 2020 07:41:58 +0000 (15:41 +0800)]
LU-8465 e2fsck: fix build for make rpm
Two problems addressed by this patch:
1)link e2fsck with -lpthread properly to make rpm pass.
2)a bounch of ext2fs_get_avg_group() unused warning messages are
annoying, move it to rw_bitmaps.c to make gcc happy.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Ia7d372b7b5aadcbf5d94916f6f67363a2a9f0bfa
Reviewed-on: https://review.whamcloud.com/40060
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Thu, 24 Sep 2020 02:00:06 +0000 (10:00 +0800)]
LU-8465 e2fsck: reset @inodes_to_rebuild if restart
Verify multiple thread on a corrupted images hit following bug:
pass1.c:2902: e2fsck_pass1_thread_prepare:
Assertion `global_ctx->inodes_to_rebuild == NULL' failed.
Signal (6) SIGABRT si_code=SI_TKILL
./e2fsck/e2fsck[0x43829e]
/lib64/libpthread.so.0(+0x14b20)[0x7f3b45135b20]
/lib64/libc.so.6(gsignal+0x145)[0x7f3b44f2c625]
/lib64/libc.so.6(abort+0x12b)[0x7f3b44f158d9]
/lib64/libc.so.6(+0x257a9)[0x7f3b44f157a9]
/lib64/libc.so.6(+0x34a66)[0x7f3b44f24a66]
./e2fsck/e2fsck(e2fsck_pass1+0x1662)[0x423572]
./e2fsck/e2fsck(e2fsck_run+0x5a)[0x41611a]
./e2fsck/e2fsck(main+0x1608)[0x4121b8]
/lib64/libc.so.6(__libc_start_main+0xf3)[0x7f3b44f171a3]
./e2fsck/e2fsck(_start+0x2e)[0x413dde]
@inodes_to_rebuild could be not NULL after we restart pass1
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I63255570e6b7405066f14b334b689fca5fbf4f8b
Reviewed-on: https://review.whamcloud.com/40024
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Wed, 23 Sep 2020 14:36:41 +0000 (22:36 +0800)]
LU-8465 e2fsck: update mmp block in one thread
For multiple threads, different threads will try to
update mmp block at the same time, only allow one
thread to update it.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Iee824dc7473b8af79f4c385204e109644fcc738c
Reviewed-on: https://review.whamcloud.com/39874
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Thu, 3 Sep 2020 02:51:49 +0000 (10:51 +0800)]
LU-8465 ext2fs: parallel bitmap loading
In our benchmarking for PiB size filesystem, pass5 takes
10446s to finish and 99.5% of time takes on reading bitmaps.
It makes sense to reading bitmaps using multiple threads,
a quickly benchmark show 10446s to 626s with 64 threads.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I8d7389413a09bf262d0ae657cb485e8862385d0c
Reviewed-on: https://review.whamcloud.com/39851
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Tue, 15 Sep 2020 07:06:11 +0000 (15:06 +0800)]
LU-8465 e2fsck: make default smallest RA size to 1M
If we have a smaller inodes per group, default ra size could
be very small(etc 128KiB), this hurts performances.
Tune above 128K to 1M, i see pass1 time drop down from
677.12 seconds to 246 secons with 32 threads.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Id54e156d3a5c65beee23533aa2ae71d34d65ca20
Reviewed-on: https://review.whamcloud.com/39914
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Fri, 19 Jun 2020 02:20:39 +0000 (10:20 +0800)]
LU-8465 ext2fs: fix to avoid invalid memory access
Dir block might be corrupted and cause the next dirent is out
of block size boundary, even though we have the check to avoid
problem, memory check tools like valgrind still complains it.
Patch try to fix the problem by checking if offset exceed max
offset firstly before getting the pointer.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I40c4587bb35b7f22d47f37b4ffa52ef822f11b41
Reviewed-on: https://review.whamcloud.com/39850
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Wed, 27 May 2020 07:14:23 +0000 (07:14 +0000)]
LU-8465 e2fsck: avoid too much memory allocation for pfsck
e2fsck init memory according to filesystem inodes/dir numbers
recorded in the superblock, this should be aware of filesystem
number of threads, otherwise, oom happen.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I99b7415231e0eb3670acec82a8a7b85b20dd36d5
Reviewed-on: https://review.whamcloud.com/39849
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Wed, 23 Sep 2020 10:05:34 +0000 (18:05 +0800)]
LU-8465 e2fsck: cleanup e2fsck_pass1_thread_join()
Use e2fsck_reset_context() to free memory to simpify
codes.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Iddb7942d5bcb8fbd44ec15cb9e7de58d466a4430
Reviewed-on: https://review.whamcloud.com/40016
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Thu, 14 May 2020 14:16:50 +0000 (22:16 +0800)]
LU-8465 e2fsck: wait fix thread finish before checking
Before proceeding next inodes, waitting existed
fixing finished.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I9ef3b8d43aee226af4c14ad02b76551f2d55ceb2
Reviewed-on: https://review.whamcloud.com/39846
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Thu, 16 Apr 2020 07:08:31 +0000 (15:08 +0800)]
LU-8465 e2fsck: set E2F_FLAG_ALLOC_OK after threads
Only flag ALLOC OK after all threads finished without problem.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I9127cca33f982100ed62a8ae0230f5eae6c8f7c1
Reviewed-on: https://review.whamcloud.com/39844
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Fri, 13 Mar 2020 10:46:24 +0000 (18:46 +0800)]
LU-8465 e2fsck: simplify e2fsck context merging codes
We tried to copy thread context to global context directly
and then copy back some saved variables before merging.
Since we have finished almost all necessary variables
in the e2fsck context, we could simplify codes, and
this could help us understand what is missing rather
than hide problems.
Change-Id: I853f386f0e61ed9110ecdae4056553ef2709b354
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37996
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Fri, 20 Mar 2020 01:55:18 +0000 (09:55 +0800)]
LU-8465 e2fsck: merge extent depth count after threads finish
tests covered by f_extent_htree.
Change-Id: Ica5a61ce9fbf73770eb014e0e938148cccb1b357
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/38000
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Fri, 20 Mar 2020 01:50:56 +0000 (09:50 +0800)]
LU-8465 e2fsck: reset lost_and_found after threads finish
This should not be kept, the reaons is similar to what
e2fsck_pass1 has done before.
Change-Id: Ieab9f799e159047f2d94cf5ba0568c9f146785ae
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37999
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Thu, 19 Mar 2020 14:05:30 +0000 (22:05 +0800)]
LU-8465 e2fsck: merge options after threads finish
It will be possible that threads might append E2F_OPT_YES,
so we need merge options to global, test f_yesall cover this.
Change-Id: I9c73f39016f09570679c11d744ba701c0e3ad294
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37998
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Tue, 25 Aug 2020 08:31:27 +0000 (16:31 +0800)]
LU-8465 e2fsck: merge expand_eisize_map properly
Min known min_extra_isize should be updated after
threads finish.
Change-Id: Ia4b00cd9c8064c8829e5c3e9f83d034d100600a5
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37997
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Thu, 12 Mar 2020 11:40:27 +0000 (19:40 +0800)]
LU-8465 e2fsck: merge encrypted dir lists after thread finish
This is missed and should be done after threads finish.
Change-Id: Ib8b05f6d0d236c7a50e1517563b46cd34efeb2a9
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37995
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Wed, 11 Mar 2020 12:48:00 +0000 (20:48 +0800)]
LU-8465 e2fsck: fix readahead for pfsck of pass1
Several improvments for this patch:
1) move readahead_kb detection to preparing phase.
2) inode readahead should be aware of thread block group
boundary.
3) make readahead_kb aware of multiple threads.
Change-Id: I4c5cde29d6e177286c7f153f26ed18e10cd954e1
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37887
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Wed, 9 Sep 2020 07:07:10 +0000 (15:07 +0800)]
LU-8465 e2fsck: adjust number of threads
number of threads should not exceed flex bg numbers,
and output messages if we adjust threads number.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I0db5441156e947693427cf6940f0cc2b3737e75c
Reviewed-on: https://review.whamcloud.com/39852
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Mon, 9 Mar 2020 07:11:10 +0000 (15:11 +0800)]
LU-8465 e2fsck: allow admin specify number of threads
-m option is introduced to specify number of threads for pfsck.
Change-Id: I1fc9ed4ef1726c699c111931aa96ad5a4ab52dff
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37859
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Wed, 11 Mar 2020 02:12:18 +0000 (10:12 +0800)]
LU-8465 e2fsck: kickoff mutex lock for block found map
Now @block_found_map is no longer shared by multiple threads,
and @block_dup_map need be checked again after threads finish.
Change-Id: I03e812e24cc36e2896e68cf8d6d8d022f7d7c283
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37884
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Thu, 12 Mar 2020 08:05:51 +0000 (16:05 +0800)]
LU-8465 e2fsck: merge EA blocks properly
EA blocks might be shared, merge them carefully.
Change-Id: Idb80a667eb364035448044a4a6e3c7a7f940ac81
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37905
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Sat, 14 Dec 2019 05:12:36 +0000 (13:12 +0800)]
LU-8465 e2fsck: split and merge invalid bitmaps
Invalid bitmaps are splitted per thread, and we
should merge them after thread finish.
Change-Id: I4b8f132dc970ba4ae63ed7b6a4f52e2ee9ba455e
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37885
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Sat, 7 Mar 2020 14:00:45 +0000 (22:00 +0800)]
LU-8465 e2fsck: move some fixes out of parallel pthreads
We could only use @found_map_block to find free blocks
after we have collectd all used blocks, so something like
handle_fs_bad_blocks(), ext2fs_create_resize_inode(),
e2fsck_pass1_dupblocks() really should be handled after
all threads has been finished.
Change-Id: I9b29782f4b055ee4cb52248a28381ed601891ebd
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37856
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Fri, 6 Mar 2020 08:46:47 +0000 (16:46 +0800)]
LU-8465 e2fsck: serialize fix operations
Allow different threads to fix at the same time could
be dangerous and error-prone now, and most of time
parallel scanning and checking is important.
So this patch adds a mutex to serialize
fix operations during pass1.
And the good benefit of this, we don't need block
allocations and free, superblock updates protection
any more, since only fix operations during pass1
could touch them.
Change-Id: Id19a54db86820d7c50e4b84ad10b2e3cb9d1ff2b
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37826
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Fri, 6 Mar 2020 15:08:07 +0000 (23:08 +0800)]
LU-8465 e2fsck: merge quota context after threads finish
Every threads calculate its own quota accounting,
merge them after threads finish.
Change-Id: I8c2022458b3b8eaad7b3b2d3fceb8d2abf562066
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37825
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Mon, 24 Feb 2020 05:08:58 +0000 (13:08 +0800)]
LU-8465 e2fsck: merge context flags properly
e2fsck might restart after pass1, so we should keep
flags if possible, this patch try to fix f_illitable_flexbg failure
Change-Id: I11229f2ad6225f53fed3a646058526bf5c853985
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37783
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Mon, 24 Feb 2020 04:44:06 +0000 (12:44 +0800)]
LU-8465 e2fsck: merge dirs_to_hash when threads finish
@dirs_to_hash list need be merged after threads finish,
test covered by t_dangerous.
Change-Id: Iad691e551c7e193f0e3efb8ddef7981c2b1035fd
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37782
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Fri, 6 Dec 2019 15:37:53 +0000 (23:37 +0800)]
LU-8465 e2fsck: merge dx_dir_info after threads finish
Merge properly.
Change-Id: I250de2d510e3c71974f6c853d5f6ff01229d5573
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/37024
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Sun, 8 Sep 2019 08:15:35 +0000 (16:15 +0800)]
LU-8465 e2fsck: merge fs flags when threads finish
merge fs flags properly.
Change-Id: Ia8a3731d2710a725938a927ec3464cb028dd271b
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/36098
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Tue, 22 Sep 2020 13:14:39 +0000 (21:14 +0800)]
LU-8465 e2fsck: merge counts after threads finish
Merge counts properly.
Change-Id: I1228b392ca500dfa64be64773f7e1f16870a6672
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/36097
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Thu, 5 Sep 2019 11:40:36 +0000 (19:40 +0800)]
LU-8465 e2fsck: add debug codes for multiple threads
These debug codes are added to run the multiple pass1 check
thread one by one in order. If all the codes are correct,
fsck of multiple threads should have exactly the same outcome
with single thread.
Change-Id: Ieb78a26611b3d5a4f21cc239e8616d215032c646
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/36054
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Li Xi [Thu, 5 Sep 2019 08:30:40 +0000 (16:30 +0800)]
LU-8465 e2fsck: merge dblist after thread finishes
Merge dblist properly.
Change-Id: I76f43dfb1ee60304662f4fc428c0243481a4f81e
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/36044
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Tue, 25 Aug 2020 08:12:37 +0000 (16:12 +0800)]
LU-8465 e2fsck: merge inode_badness after threads finish
inode_badness is only specific for master-lustre branch,
seperate it and fix a bug that inode_badness is not freed properly
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I75a54adc088cb5b636171e4ee7929496858fedf9
Reviewed-on: https://review.whamcloud.com/39843
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Li Xi [Wed, 4 Sep 2019 10:47:14 +0000 (18:47 +0800)]
LU-8465 e2fsck: merge icounts after thread finishes
Merge inode_count and inode_link_info properly after
threads finish.
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I45fbbfb161b72f83aa9e2b26cc5158ea63693f16
Reviewed-on: https://review.whamcloud.com/39841
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Tue, 22 Sep 2020 10:00:39 +0000 (18:00 +0800)]
LU-8465 e2fsck: merge badblocks after thread finishes
Badblocks should be merged properly after threads finish.
Change-Id: I3ac8144a36059be9000aa2f87b677c5de58d7288
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/36026
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Mon, 21 Sep 2020 11:02:33 +0000 (19:02 +0800)]
LU-8465 e2fsck: rbtree bitmap for dir
Only rbtree support merge operation now, use it for bitmaps.
Change-Id: I863687ce275f9c891cd2d18c115cb75c6e24f4e4
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/36020
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Li Xi [Mon, 2 Sep 2019 08:14:38 +0000 (16:14 +0800)]
LU-8465 e2fsck: merge dir_info after thread finishes
dir_info need be merged after thread finish.
Change-Id: I86eff1b91ef03e63ae5d1b883413923793aca981
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/36018
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Li Xi [Fri, 30 Aug 2019 09:56:10 +0000 (17:56 +0800)]
LU-8465 e2fsck: optimize the inserting of dir_info_db
Binary search is now used when inserting an dir info to the array.
Memmove is now used when moving array. Both of them improves
the performance of inserting.
This patch is also a prepartion for the merging of two dir db
arrays.
Change-Id: Iefcf64f195ae6c2f85ac3d550863cfd06815037e
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/36004
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Li Xi [Fri, 30 Aug 2019 04:20:11 +0000 (12:20 +0800)]
LU-8465 e2fsck: do not change global variables
Global variables used in pass1 check are changed to local variables
in this patch. This will avoid conflict between threads.
Change-Id: I7f8dea6281c8c7f19b0b8d4b3c8f211ab8aa6924
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/36001
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Wang Shilong [Wed, 23 Sep 2020 03:17:29 +0000 (11:17 +0800)]
LU-8465 e2fsck: merge bitmaps after thread completes
A new method merge_bmap has been added to bitmap operations. But
only red-black bitmap has that operation now.
Change-Id: I571af964a31f2d08fa6802b682b538f60ed271dc
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35961
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Mon, 26 Aug 2019 14:34:06 +0000 (22:34 +0800)]
LU-8465 e2fsck: print thread log properly
When multi-thread fsck is enabled, logs printed from multiple
threads could overlap with each other. The overlap sometimes
makes the logs unreadable because log_out() is used multiple times
for a single line.
This patch adds leading [Thread XXX] to each logs if multi-thread
is enabed by -m option.
This patch also adds message to show the group ranges and inode
numbers for each thread, which is useful for debuging multi-thread
check.
Change-Id: I5bde34263d737ad5ffe177eba5520111baa71e54
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35922
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Thu, 22 Aug 2019 07:35:12 +0000 (15:35 +0800)]
LU-8465 e2fsck: split groups to different threads
The start/end groups of a thread is calculated according to the
thread number. But still, only one thread is used to check.
Change-Id: I29a8de724f6f7bd0984aabae8314ec65ff03851e
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35855
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Li Xi [Wed, 21 Aug 2019 13:58:31 +0000 (21:58 +0800)]
LU-8465 e2fsck: add start/end group for thread
When multi-threads are used for check, each thread needs to jump
to different group in pass1 check. This patch adds the group
jumping support. But still, only one thread is used to check.
Change-Id: I3c299f4383f0406782fbedf77cd2641a8131db49
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35846
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Li Xi [Thu, 15 Aug 2019 08:33:19 +0000 (16:33 +0800)]
LU-8465 e2fsck: optionally configure one pfsck thread
This patch creates only one thread to do pass1 check. The same
codes can be used to create multiple threads, but other functions
need to be modified to get ready for that.
pfsck support will be enabled with if configured with
--enable-pfsck option.
Change-Id: Ic1c704d357aa323b19211fbefb9ab6dbd798b9aa
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35793
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Tue, 13 Aug 2019 03:55:27 +0000 (11:55 +0800)]
LU-8465 e2fsck: create logs for mult-threads
When multi-threads are used, different logs should be created
for different threads. Each thread has log files with suffix
of ".$THREAD_INDEX".
And this patch adds f_multithread_logfile test case.
Change-Id: I9d108f385a1edf06eb0fa67a311856ca9ab0d004
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35780
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Sat, 10 Aug 2019 07:59:09 +0000 (15:59 +0800)]
LU-8465 e2fsck: open io-channel when copying fs
This patch also add writethrough flag to the thread io-channel.
When multiple threads write the same disk, we don't want the
data being saved in memory cache. This will be useful in the
future, but even without that flag, the tests can be passed too.
This patch also cleanup the io channel cache of the global
context. Otherwise, after pass1 step, the next steps would use
old data saved in the cache. And the cached data might have
already been overwritten in pass1.
Change-Id: Ie9d93a2193394ab12550da6141d8b08b9e519ef0
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35763
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Wed, 9 Sep 2020 07:07:10 +0000 (15:07 +0800)]
LU-8465 e2fsck: copy badblocks when copying fs
This patch copies badblocks when the copying fs.
Change-Id: I1fd600cdb0f01a1dc6f03cd3637f83d64b586698
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35726
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Wed, 7 Aug 2019 08:54:41 +0000 (16:54 +0800)]
LU-8465 e2fsck: copy bitmaps when copying context
This patch copies bitmap when the copying context. In the
multi-thread fsck, each thread use different bitmap that copied
from the glboal bitmap. And Bitmaps from multiple threads will
be merged into a global one after the pass1 finishes.
Change-Id: Ic5e181712630814d12b2fa6e3836ce0acc1832a7
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35712
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Wed, 7 Aug 2019 07:34:00 +0000 (15:34 +0800)]
LU-8465 e2fsck: add assert when copying context
Adding the assert would simplify the copying of context.
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I5270884f5c5cb3c8c51ca34f7e37fed4fc0b30f1
Reviewed-on: https://review.whamcloud.com/40015
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Fri, 25 Sep 2020 02:52:03 +0000 (20:52 -0600)]
ATM-1754 tests: skip f_large_dir on RHEL6
Skip f_large_dir test on RHEL6 since it runs very slowly, and
has enough coverage on other branches.
Change-Id: I7e001f734d84fc7927bed1da861a33283577d490
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40042
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Wang Shilong [Thu, 17 Sep 2020 02:58:56 +0000 (10:58 +0800)]
LU-8465 e2fsck: clear icache when using multi-thread fsck
icache of fs will be rebuilt when needed, so after copying
fs, icache can be inited to NULL.
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: If153d20c4fb10163a7759facb14d1c065c7037fa
Reviewed-on: https://review.whamcloud.com/40014
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Thu, 8 Aug 2019 08:59:59 +0000 (16:59 +0800)]
LU-12641 libext2fs: memory leak of check_if_lustre_mounted
check_if_lustre_mounted() doesn't free the allocated memory
if failure.
Change-Id: I36b5c1e981ca4f1db8c9515be29dd98c074d14dc
Signed-off-by: Li Xi <lixi@ddn.com>
Reviewed-on: https://review.whamcloud.com/35729
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Li Xi [Tue, 6 Aug 2019 03:19:15 +0000 (11:19 +0800)]
LU-8465 e2fsck: copy fs when using multi-thread fsck
This patch only copy the fs to a new one when -m is enabled.
It doesn't actually start any thread. When pass1 test finishes,
the new fs is copied back to the original context.
This patch handles the fs fields in dblist, inode_map and block_map
properly.
Change-Id: I1cc80bb1d762692a6bde3a57d1498d2842884c41
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35696
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Xi [Mon, 5 Aug 2019 04:01:17 +0000 (12:01 +0800)]
LU-8465 e2fsck: copy context when using multi-thread fsck
This patch only copy the context to a new one when -m is enabled.
It doesn't actually start any thread. When pass1 test finishes,
the new context is copied back to the original context.
Since the signal handler only changes the original context, so
add global_ctx in "struct e2fsck_struct" and use that to check
whether there is any signal of canceling.
This patch handles the long jump properly so that all the existing
tests can be passed even the context has been copied. Otherwise,
test f_expisize_ea_del would fail when aborting.
Change-Id: I99bc621165ac7470bbd54550f8198b0a0b35feee
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35690
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Li Xi [Sun, 4 Aug 2019 15:28:59 +0000 (23:28 +0800)]
LU-8465 e2fsck: add -m option for multithread
-m option is added but no actual functionality is added. This
patch only adds the logic that when -m is specified, one of
-p/-y/-n options should be specified. And when -m is specified,
-C shouldn't be specified and the completion progress report won't
be triggered by sending SIGUSR1/SIGUSR2 signals. This simplifies
the implementation of multi-thread fsck in the future.
Completion progress support with multi-thread fsck will be added
back after multi-thread fsck implementation is finished. Right
now, disable it to simplify the implementation of multi-thread fsck.
Change-Id: I34c6ee2263bd398699acefe88fb4526f19148e39
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35689
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Thu, 16 Apr 2020 06:44:44 +0000 (14:44 +0800)]
LU-8465 e2fsck: remove unused fs_ext_attr_inodes/blocks
Only define but not used, remove them.
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Ia885a56f574a7f817c532cb0efcf249a42143276
Reviewed-on: https://review.whamcloud.com/39840
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Fri, 27 Mar 2020 09:20:24 +0000 (17:20 +0800)]
LU-8465 e2fsck: cleanup struct e2fsck_struct
Fields of "struct e2fsck_struct" are seperated into
different types according to how these fields will be
used when parallel fsck is enabled.
remove unused @abort_code
Change-Id: Ic89398cca70e0833ecb8fe6aae24c1e99e5a9e90
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/35684
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Fri, 12 Jun 2020 21:21:48 +0000 (15:21 -0600)]
LU-13672 tune2fs: reset MMP state on error exit
If tune2fs cannot perform the requested change, ensure that the MMP
block is reset to the unused state before exiting. Otherwise, the
filesystem will be left with mmp_seq = EXT4_MMP_SEQ_FSCK set, which
prevents it from being mounted afterward:
LDISKFS-fs warning (device dm-9): ldiskfs_multi_mount_protect:311:
fsck is running on the filesystem
Add a test to try some failed tune2fs operations and verify that the
MMP block is left in a clean state afterward.
Change-Id: I3a5acc5875fa2d082a66e751ec4b326ffe3ebbe5
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/38926
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Li Dongyang [Mon, 23 Mar 2020 01:56:00 +0000 (12:56 +1100)]
LU-13241 libext2fs: add quota to libext2fs
add quota related interface to libext2fs and install the
relevant headers.
Change-Id: I17e6b5aa74e0f1bb1465168a1cf4e03184e003b0
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/38027
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Thu, 21 May 2020 20:57:49 +0000 (14:57 -0600)]
ATM-1754 tests: skip f_large_dir on aarch64
Skip the f_large_dir test on the aarch64 builders because this is
extremely slow (90 minutes) and clogs up the build system when
there are many patches in the queue.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8c3a588c34caae5332b4c643df940b0e855cab07
Reviewed-on: https://review.whamcloud.com/38693
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Minh Diep [Tue, 28 Jan 2020 22:35:46 +0000 (14:35 -0800)]
LU-13177 build: add support for SLES15SP1
Add SUSE 15SP1 to distro and spec file
Change-Id: Ica12ae0246db4755746ada6647122e8853fc6258
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37357
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Artem Blagodarenko [Thu, 4 Jul 2019 10:54:17 +0000 (13:54 +0300)]
LU-11915 deubugfs: add support for xattrs in external inodes
There was "e2fsck: add support for xattrs in external inodes" before
that added xattr in external inode support to utils. But it was
not applied to the last e2fsprogs version for some reasons.
This patch adds external inode output to debugfs that is required
for conf-sanity 115 test.
Change-Id: I3cfca85e2d34b54030d21fafa12be3dbf30e5b5c
Signed-off-by: Artem Blagodarenko <c17828@cray.com>
Reviewed-on: https://review.whamcloud.com/35417
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Li Dongyang [Tue, 11 Jun 2019 05:05:03 +0000 (15:05 +1000)]
LU-12403 build: add support for RHEL/CentOS 8
Add RHEL/CentOS 8 to Distro, reuse the RHEL 7 spec.
Change-Id: Ic0fa915d1145ffc04bd5f6f7e3531b892a3d004a
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/35168
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Dongyang [Fri, 26 Apr 2019 05:23:28 +0000 (15:23 +1000)]
LU-12226 debian: fix debhelper level requirement
Requiring debhelper >= 11 will break the build on
Ubuntu 16.04 LTS, it only has debhelper 9 and 10
from backports.
Change-Id: I40838a4de5ee7a841ba970d4f7531f8fe9dc5ff4
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/34765
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Wang Shilong [Tue, 21 May 2019 04:06:39 +0000 (12:06 +0800)]
LU-12158 mke2fs: avoid too large stride and stripe_width
According to benchmarks numbers, too large stripe and stripe_width
will make preassure with ext4 mballoc allocater and hurts performances
finally.
2MB Chunk size 256K Chunk size
stripe_width,stride Write(MB/s) Read(MB/s) Write(MB/s) Read(MB/s)
512,512 10,810 10,124 10,492 6,923
1024,1024 10,793 10,064 10,431 6,921
2048,2048 8,047 10,080 6,629 7,381
4096,4096 7,350 10,089 6,505 7,282
Performance number comes from Shuichi Ihara.
This patch try to avoid use too large stride and stripe_width when mkfs.
If users really want large value they could do it by specify mkfs options
or run tune2fs later.
Change-Id: I768f1ecb39837338e08842b21b4fca8b98165d2a
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/34767
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Andreas Dilger [Sat, 3 Nov 2018 09:10:22 +0000 (03:10 -0600)]
LU-11609 mke2fs: allow 64bit feature without extents
The 64bit feature should be allowed without extents to for 32-bit
metadata_csum checksums to be stored in the group descriptor.
Change the extents check to check for more than 2^32 blocks instead
of the 64bit feature flag. This also avoids warnings later if the
metadata_csum feature is enabled on a filesystem without 64bit.
Consolidate repeated 64bit feature checks into one check.
The fs_blocks_count value is always set when parse_fs_type() is
called, so no need to check if it is zero.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie73ff0ed50cfed5d6a9596260c6b6dc32d3ebbe5
Reviewed-on: https://review.whamcloud.com/33897
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Li Dongyang [Mon, 6 Aug 2018 01:33:44 +0000 (11:33 +1000)]
LU-11215 libe2p: add large_xattr as an alias of ea_inode
The upstream version of large xattr suppport doesn't
add "large_xattr" as a feature name, but it's still
used by our test scripts.
e2fsprogs commit:
6a081f6d2a5cff0f5a077065aab39901d54bfb61
Change-Id: I48d7ac54b2b2ad252c404050bc4c3d9527865107
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/32932
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Alexander Boyko [Thu, 8 Feb 2018 06:43:40 +0000 (01:43 -0500)]
LU-10638 build: add support for Scientific
Add Scientific to Distro with a reference to RHEL.
Signed-off-by: Alexander Boyko <c17825@cray.com>
Change-Id: I05001ee9233e4cac6b2c6b523a374ffdc243394d
Cray-bug-id: MRP-3882
Reviewed-on: https://review.whamcloud.com/31220
Reviewed-by: Artem Blagodarenko <c17828@cray.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Andreas Dilger [Sat, 18 Jun 2016 10:50:41 +0000 (04:50 -0600)]
LU-8300 e2fsck: print out FID with dirent inode number
When printing out the dirent inode number in e2fsck, also print
out the FID, if available, for reference in the future if needed.
Entry 'lustre.conf' in /ROOT/etc/yum/protected.d (50132) has
deleted/unused inode=1964 fid=[0x200000401:0x844:0x0]. Clear? yes
This can be useful if LFSCK is able to rebuild the MDT layout from
the OST object xattrs, but the filename isn't stored on the OST.
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Id0ec22c47d83f16c392e93328d2869e7a53ebbe5
Reviewed-on: https://review.whamcloud.com/20866
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Artem Blagodarenko <c17828@cray.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Alexey Lyashkov [Mon, 15 May 2017 14:45:38 +0000 (17:45 +0300)]
LU-9501 mke2fs: avoid inode number error with large FS
Limit inodes count instead of error return, to use all inodes in
the filesystem.
Change-Id: I76ee0a6c11b099b8c17d90b0778e3c0f0cf1f60a
Signed-off-by: Alexey Lyashkov <alexey.lyashkov@seagate.com>
Reviewed-on: https://review.whamcloud.com/27111
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Fan Yong [Fri, 26 Jun 2015 15:29:59 +0000 (23:29 +0800)]
LU-6722 jbd: double minimum journal size for RHEL7
In RHEL7 (kernel version >= 3.10.0), the maximum journal transaction
size is reduced to half of the RHEL6 case. That may cause Lustre to
complain that the declared transaction credits too large for very
small journal device. This patch increases the minimum journal size
from 4MB to 8MB for RHEL7 case, then counteract the above limitation
about the journal transaction size.
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iec8a2c561416cb7b5acce342c8ebcb845c8d7a19
Reviewed-on: http://review.whamcloud.com/15401
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Bob Glossman [Thu, 16 Oct 2014 18:41:07 +0000 (11:41 -0700)]
LU-5845 build: add support for SUSE_LINUX-12 .spec file
Add the SLES 12 .spec file.
For now SLES 12 and SLES 11 can share the same .spec file.
LU-8390 build: use correct .spec file for SLES 12
Changes in output from the lsb_release command in SLES 12
resulted in using the wrong .spec file in builds.
This patch corrects the problem for old and new versions.
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I41d47c305eba7d3dd0d313be5ad8192592291fad
Reviewed-on: http://review.whamcloud.com/21237
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I3b716c09ea8632d2333aaf61b8a6c881ccd18a88
Reviewed-on: http://review.whamcloud.com/12316
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Andreas Dilger [Fri, 18 Jul 2014 20:47:02 +0000 (14:47 -0600)]
LU-5307 build: add support for RHEL-7 .spec file
Add the RHEL 7 .spec file.
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I00278b50c3669f56c5297a75b89775fec9476f43
Andreas Dilger [Sat, 14 Jul 2012 02:33:01 +0000 (20:33 -0600)]
LU-1540 e2fsck: add missing symlink NUL terminator
If a long symbolic link target is written into an external block
without a NUL terminator, its length is decided by the inode's size.
Make symlink check add a NUL termination in such cases if needed.
Such faulty symlinks were generated by osd-ldiskfs on the MDS until
Lustre 2.1.3 and Lustre 2.3. The in-kernel code would handle such
unterminated symlinks correctly, since it used the inode size to
determine the symlink length, but e2fsck would assume the symlink
is broken if there wasn't a trailing NUL.
LU-2627 e2fsck: check_symlink() SIGSEGV
Since e2fsck_pass1_check_symlink() calls into check_symlink()
with pctx == NULL, we should use 'ino' instead of 'pctx->ino'
in check_symlink().
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: If9c16f96d0655d5a886ef607f1f47ced6176f8d8
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I4419b30f1adb4a7d273796a936427aa351510213
Andreas Dilger [Fri, 13 Apr 2012 08:16:24 +0000 (02:16 -0600)]
ext2fs: check if Lustre filesystem is mounted
Add a check to ext2fs_check_mount_point() to loo in /proc/fs/lustre/*
to see if Lustre is mounted, since st_rdev of the mountpoint does not
match st_rdev of the block device itself, which confuses libext2fs.
LU-6093 ext2fs: don't confuse mke2fs by Lustre zfs mount
check_if_lustre_mounted() by efb735 will mistakenly report the
device is mounted when there is a ZFS-based Lustre mount.
This patch fix it by parsing the real device name from the pool
name exported in Lustre proc file with "zpool" command.
It also includes a typo fix in check_if_lustre_mounted() and a
few other code cleanups.
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I43dabbcd6fa721c569cd9cc0f4a6de93ab761392
Reviewed-on: http://review.whamcloud.com/14223
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Andreas Dilger [Fri, 13 Apr 2012 18:55:45 +0000 (12:55 -0600)]
debugfs: dump "fid" and "lma" xattrs on inode stat
Print out the Lustre "fid" and "lma" object xattr contents,
if present, with debugfs stat to simplify debugging.
LU-8998 debugfs: handle combined LMA EA and PFID EA
We need to store the stripe size and stripe count information in
the OST object's PFID EA for the layout LFSCK to recover the MDT
object or its (lost or corrupted) LOV EA. On the other hand, for
PFL file, we also need to store its PFL ID and extent information
in its OST object's PFID EA.
Consider performance, we will store the PFID EA inside the inode
body to avoid extra IO when read/write the PFID EA. Unfortunately,
the space inside current 256-bytes inode for EA is very limited,
and above enlarged PFID EA exceeds such limitation. So we have to
make some hack with the PFID EA and LMA EA combined together. The
patch makes the debugfs tool to handle such case properly.
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I1380d16d94d6d05272e22fdfff07ada038685669
Reviewed-on: https://review.whamcloud.com/19436
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Andreas Dilger [Fri, 18 May 2018 04:52:58 +0000 (14:52 +1000)]
tests: add basic tests for dirdata feature
Signed-off-by: Pravin Shelar <pravin@clusterfs.org>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Artem Blagodarenko <artem.blagodarenko@gmail.com>
Andreas Dilger [Thu, 17 May 2018 03:43:16 +0000 (13:43 +1000)]
e2fsck: add support for dirdata feature
Add support for the INCOMPAT_DIRDATA feature, which allows
storing extra data in the directory entry beyond the name.
This allows the Lustre File IDentifier to be accessed in
an efficient manner, and would be useful for expanding a
filesystem to allow more than 2^32 inodes in the future.
Include this patches:
e2fsck: e2fsck -D does not change dirdata content
Fix dir optimization to preserve dirdata content for dot
and dotdot entries.
Lustre-bug: https://jira.hpdd.intel.com/browse/LU-1774
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Iae190794da75a2080a8e5cc5b95a49e0c894f72f
e2fsprogs: Consider DIRENT_LUFID flag in link_proc().
While adding the new file entry in directory block, link_proc()
calculates minimum record length of the existing directory entry
without considering the dirent data size and which leads to
corruption. Changed the code to use EXT2_DIR_REC_LEN() which will
return correct record length including dirent data size.
Lustre-bug: https://jira.hpdd.intel.com/browse/LU-2462
Signed-off-by: Manisha Salve <msalve@ddn.com>
Change-Id: Ic593c558c47a78183143ec8e99d8385ac94d06f7
libext2fs, e2fsck: don't use ext2_dir_entry_2
Due to endian issues, do not use ext2_dir_entry_2 because it will
have the wrong byte order on directory entries that are swabbed.
Instead, use the standard practice of mask-and-shift to access the
file_type and dirdata flags.
Lustre-bug: https://jira.hpdd.intel.com/browse/LU-4677
Signed-off-by: Pravin Shelar <pravin@clusterfs.com>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Artem Blagodarenko <artem.blagodarenko@gmail.com>
git://git.whamcloud.com / tools / e2fsprogs.git / log