Whamcloud - gitweb
Andreas Dilger [Fri, 13 Apr 2012 00:03:37 +0000 (18:03 -0600)]
e2fsck: add support for expanding the inode size
This patch adds a "-E expand_extra_isize" feature which makes sure
that _every_ used inode has i_extra_isize >= s_min_extra_isize if
s_min_extra_isize is set. Else it makes sure that i_extra_isize
of every inode is equal to sizeof(ext2_inode_large) - 128.
This is useful for the case where nanosecond timestamps or 64-bit
inode version fields are required for all inodes in the filesystem.
There is also a fix for test f_itable_collision, the original
E2FSCK_TIME would overflow on a 32bit system when adding with
ctx->time_fudge in EXT4_XTIME_FUTURE, making us increase the
inode badness incorrectly.
LU-10205 libext2fs: fix buffer overrun in ext2fs_expand_extra_isize
In ext2fs_expand_extra_isize, we size buffer using 'size' but then
do the memcpy with the rounded-up size, which can overflow the buffer.
With MALLOC_CHECK_=2, I see:
Error in `../e2fsck/e2fsck': free(): invalid pointer: <addr>
Change-Id: I31be58de12d4d50646c7aa96959de0efc5c279c3
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Reviewed-on: https://review.whamcloud.com/29975
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Change-Id: I306ff4f81d8bd6bdf0446c76d6772951043fead4
Signed-off-by: Kalpak Shah <kalpak@clusterfs.com>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Andreas Dilger [Thu, 12 Apr 2012 23:32:53 +0000 (17:32 -0600)]
e2fsck: allow deleting or zeroing shared blocks
E2fsck fixes files that are found to be sharing blocks by cloning
the shared blocks and giving each file a private copy in pass 1D.
Allowing all files claiming the shared blocks to have copies can
inadvertantly bypass access restrictions. Deleting all the files,
zeroing the cloned blocks, or placing the files in the /lost+found
directory after cloning may be preferable in some secure environments.
The following patches implement config file and command line options
in e2fsck that allow pass 1D behavior to be tuned according to site
policy. It adds two extended options and config file counterparts.
On the command line:
-E clone=dup|zero
Select the block cloning method. "dup" is old behavior,
and is the default. "zero" is a new method that substitutes
zero-filled blocks for the shared blocks in all the files
that claim them.
-E shared=preserve|lost+found|delete
Select the disposition of files containing shared blocks.
"preserve" is the old behavior which remains the default.
"lost+found" causes files to be unlinked after cloning so
they will be reconnected to /lost+found in pass 3.
"delete" skips cloning entirely and simply deletes the files.
In the config file:
[options]
clone=dup|zero
shared=preserve|lost+found|delete
Change-Id: I132a3122076b5c68c4f9ea8c1316cb48f8d94020
Signed-off-by: Jim Garlick <garlick@llnl.gov>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Andreas Dilger [Thu, 12 Apr 2012 23:24:55 +0000 (17:24 -0600)]
e2fsck: parse config file before command-line opts
The patch changes the order that the config file and command line
are parsed so that command line has precedence. It also parses
the -E option for every occurrence, otherwise the -E option is
not cumulative.
Signed-off-by: Jim Garlick <garlick@llnl.gov>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Andreas Dilger [Thu, 12 Apr 2012 21:31:35 +0000 (15:31 -0600)]
filefrag: Lustre changes to filefrag FIEMAP handling
Add support for multiple-device filesystems by defining a new
fe_device field in the fiemap_extent structure. This allows
printing the filesystem-relative or linux block device number
associated with each extent of a file. If a single filesystem
extent is mirrored to multiple block devices, the fe_device
field can be used to disambiguate the multiple copies.
If the "-l" (device-logical) option is given to filefrag, then
all extents for a particular device of a file are returned
before returning extents for the next device. This makes it
easier to see if extent allocation within a single device is
contiguous, instead of returning all of the blocks of a file
interleaved in file-logical-offset order.
filefrag: support PFL and FLR file
Add FIEMAP_EXTENT_DATA_MIRROR support FLR file, which is returned
from Lustre indicating that a new mirror will be processed, so we'd
reset the fm_start and fe_logical.
Change-Id: Ifa68d951ac68d9f141eae915e58c13d73833c8c9
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40765
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
LU-11848 filefrag: add -V option to print version
Add '-V' to filefrag to print the installed version of the tool.
If '-V' is used twice, print out the list of supported FIEMAP flags.
This can be used to check if filefrag understands a specific feature.
Change-Id: Ib126bdd70efa1775aef6db761f54e27a593ebbe5
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40873
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
LU-11848 filefrag: rework port PFL and FLR file
Lustre uses the high 16bits of fiemap_extent:fe_device to record
the absolute stripe number is under process, so that next fiemap
call continues from that stripe.
Fixes:
0b6718f5e22d ("filefrag: support PFL and FLR file")
Change-Id: I111bcb100307aebacbffed99109057accdc27f5b
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40889
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-11848 filefrag: further PFL and FLR fixes
Revert some of the changes from the earlier patches to bring the
filefrag functionality closer to the original behavior.
The kernel is instead changed to return the device number with
every extent. This allows the previous filefrag to work for PFL
and FLR files without any changes by just copying the fe_device
from the last returned extent to the next call. The only gap in
old filefrag is that it prints the stripe number with OST index.
Fixes:
d99d208e73cd ("LU-11848 filefrag: rework port PFL and FLR file")
Fixes:
0b6718f5e22d ("filefrag: support PFL and FLR file")
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie6a8d93ebceb1e70894b21f0dadc9655083ebbe5
Reviewed-on: https://review.whamcloud.com/40932
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Change-Id: Ifb40cc159ddc61d2296c494f0e899ac11bf88b60
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Li Dongyang [Mon, 23 Mar 2020 01:56:00 +0000 (12:56 +1100)]
LU-13241 libext2fs: add quota to libext2fs
add quota related interface to libext2fs and install the
relevant headers.
Change-Id: I17e6b5aa74e0f1bb1465168a1cf4e03184e003b0
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/38027
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Andreas Dilger [Fri, 25 Sep 2020 02:52:03 +0000 (20:52 -0600)]
ATM-1754 tests: skip f_large_dir on RHEL6
Skip f_large_dir test on RHEL6 since it runs very slowly, and
has enough coverage on other branches.
Change-Id: I7e001f734d84fc7927bed1da861a33283577d490
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40042
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Andreas Dilger [Thu, 21 May 2020 20:57:49 +0000 (14:57 -0600)]
ATM-1754 tests: skip f_large_dir(_csum) on aarch64
Skip the f_large_dir(_csum) tests on the aarch64 builders because this is
extremely slow (90 minutes) and clogs up the build system when
there are many patches in the queue.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I8c3a588c34caae5332b4c643df940b0e855cab07
Reviewed-on: https://review.whamcloud.com/38693
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
Minh Diep [Tue, 28 Jan 2020 22:35:46 +0000 (14:35 -0800)]
LU-13177 build: add support for SLES15SP1
Add SUSE 15SP1 to distro and spec file
Change-Id: Ica12ae0246db4755746ada6647122e8853fc6258
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/37357
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Andreas Dilger <adilger@whamcloud.com>
Li Dongyang [Tue, 11 Jun 2019 05:05:03 +0000 (15:05 +1000)]
LU-12403 build: add support for RHEL/CentOS 8
Add RHEL/CentOS 8 to Distro, reuse the RHEL 7 spec.
Change-Id: Ic0fa915d1145ffc04bd5f6f7e3531b892a3d004a
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/35168
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Li Dongyang [Fri, 26 Apr 2019 05:23:28 +0000 (15:23 +1000)]
LU-12226 debian: fix debhelper level requirement
Requiring debhelper >= 11 will break the build on
Ubuntu 16.04 LTS, it only has debhelper 9 and 10
from backports.
As we dropped the build for 16.04, change the required
debhelper to 11, which is the version shipping with
18.04
Change-Id: I40838a4de5ee7a841ba970d4f7531f8fe9dc5ff4
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/34765
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Alexander Boyko [Thu, 8 Feb 2018 06:43:40 +0000 (01:43 -0500)]
LU-10638 build: add support for Scientific
Add Scientific to Distro with a reference to RHEL.
Signed-off-by: Alexander Boyko <c17825@cray.com>
Change-Id: I05001ee9233e4cac6b2c6b523a374ffdc243394d
Cray-bug-id: MRP-3882
Reviewed-on: https://review.whamcloud.com/31220
Reviewed-by: Artem Blagodarenko <c17828@cray.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Bob Glossman [Thu, 16 Oct 2014 18:41:07 +0000 (11:41 -0700)]
LU-5845 build: add support for SUSE_LINUX-12 .spec file
Add the SLES 12 .spec file.
For now SLES 12 and SLES 11 can share the same .spec file.
LU-8390 build: use correct .spec file for SLES 12
Changes in output from the lsb_release command in SLES 12
resulted in using the wrong .spec file in builds.
This patch corrects the problem for old and new versions.
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I41d47c305eba7d3dd0d313be5ad8192592291fad
Reviewed-on: http://review.whamcloud.com/21237
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I3b716c09ea8632d2333aaf61b8a6c881ccd18a88
Reviewed-on: http://review.whamcloud.com/12316
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Andreas Dilger [Fri, 18 Jul 2014 20:47:02 +0000 (14:47 -0600)]
LU-5307 build: add support for RHEL-7 .spec file
Add the RHEL 7 .spec file.
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I00278b50c3669f56c5297a75b89775fec9476f43
Andreas Dilger [Fri, 13 Apr 2012 08:23:12 +0000 (02:23 -0600)]
TT-177 build: add .spec file for SLES11 packaging
Include the upstream SLES11 .spec file to ensure the packages we
build match the upstream packages. Any later patches that change
the packaging should patch the .spec file appropriately.
Add in the SLES-specific patches, excluding the replacement de.po
file, since the original SLES11 de.po file is only against 1.41.4,
and is missing a large number of changes to the translated messages
related to 64-bit format specifiers.
Always skip m_hugefile and t_replay_and_set tests, building rpms in
an ext3 environment will make the tests fail.
LU-4284 build: add missing Provides line in SLES spec file
Need to add a line in the SUSE spec file for Provides: ldiskfsprogs.
This is present in the RHEL spec file and is needed to resolve
dependencies in lustre server rpms at rpm install time.
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ib4821004d27c9a7271ffdbd7403990e586d6c9ca
LU-9713 build: Require uptodate libcom_err, libss
Ensure that the libcom_err and libext2fs2 packages installed match the
other packages built from e2fsprogs so that modifications to those
libraries (e.g. project quota) are available.
Test-Parameters: clientdistro=sles12sp2 mdsdistro=sles12sp2 ossdistro=sles12sp2
Change-Id: Idc22a029a6950cc1ffb9b6e44181d345db3ebbe5
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-on: https://review.whamcloud.com/27834
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I783d58bd78d7c4c66cc85ec5557ae1aaf64016ba
Andreas Dilger [Fri, 13 Apr 2012 08:19:19 +0000 (02:19 -0600)]
build: add RHEL6 .spec file for packaging
Include the upstream RHEL6 .spec file to ensure the packages we
build match the upstream packages. Any later patches that change
the packaging should patch the .spec file appropriately.
Change-Id: Ia12d193ff1a9c5a40cee2585a9bc47d5f9f63f4d
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Andreas Dilger [Thu, 12 Apr 2012 21:39:04 +0000 (15:39 -0600)]
build: update e2fsprogs.spec for distro builds
Add the distro version to the RPM release number, so that it the
RPM names do not conflict.
Allow the RPM built from upstream to replace the split packages
provided by the distros. At some point in the future it may be
desirable to also split the RPM built by this spec file, but this
bs complicated by the fact that SLES and RHEL have different splits.
Change-Id: Iab8f75a7f7bd9790899a2024d91079d55501ef65
Signed-off-by: Girish Shilamkar <girish.shilamkar@sun.com>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Andreas Dilger [Thu, 12 Apr 2012 20:00:07 +0000 (14:00 -0600)]
build: update version for Lustre build
Add Lustre-specific build version to distinguish packages from
upstream packages.
Since e2fsprogs release 1.44.4 the Version in e2fsprogs.spec
is not generated anymore. However contrib/build-rpm still relies
on the Version from the spec, maintain the version in the spec
as well.
e2fsprogs commit:
e5f0f3eebef7b483d5cc8b37023d6e0256776d94
Change-Id: I4a31e659ff4d98f9bb00fea731bd1ee59a55fd38
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Li Dongyang [Sun, 14 Feb 2021 23:30:11 +0000 (10:30 +1100)]
LU-14432 configure.ac: fix AC_PREREQ
AS_HELP_STRING was introduced in autoconf 2.58
requiring autoconf 2.69 is not necessary and it breaks
several builds in our Jenkins.
Change required autoconf to 2.59 just to be safe.
Change-Id: I30af873879866858d32b715ac4ce78d095e10f7d
Fixes:
8c9e2921 ("configure.ac: convert all help strings to use AS_HELP_STRING")
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/41665
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Saranya Muruganandam [Tue, 17 Nov 2020 17:54:30 +0000 (09:54 -0800)]
e2fsck: Annotating fields in e2fsck_struct
Adding information on fields in e2fsck_struct
on how they are used when running parallel fsck.
E2fsprogs-commit:
c2f04f7da367ffc4cd64be5b2285878ad23b3e7f
Change-Id: I5b97921f0191b1d2bf4bf6bbedefc7ae401e9e33
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Saranya Muruganandam [Fri, 13 Nov 2020 23:33:05 +0000 (15:33 -0800)]
e2fsck: propagate number of threads
Sometimes, such as in orphan_inode case, e2fsck_pass1
is called after reading the block bitmaps. This results in
reading the block bitmap sequentially and multithreading
only gets kicked in later. Fix the thread count earlier
while setting up the file system.
E2fsprogs-commit:
3f56676f4daa7379dbd868a4114910f20d49527d
Change-Id: I51ab36ad8094c605a32e4b7929fadb326d77169e
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Mon, 28 Sep 2020 19:21:16 +0000 (13:21 -0600)]
e2fsck: misc cleanups for pfsck
Add -m option description to e2fsck.8 man page.
Rename e2fsck_struct fs_num_threads to pfs_num_threads to avoid
confusion with the ext2_filsys fs_num_threads field, and move
thread_info to be together with the other HAVE_PTHREAD fields.
Move ext2_filsys fs_num_threads to fit into the __u16 "pad" field
to avoid consuming one of the few remaining __u32 reserved fields.
Fix a few print format warnings.
E2fsprogs-commit:
afad25acf8c6694c8cea08820fe5af84388a76f3
Change-Id: I5ab49a3ad41f1ebb374669c5d640afa1f3fc35fd
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Mon, 28 Sep 2020 02:27:12 +0000 (10:27 +0800)]
e2fsck: fix memory leaks with pfsck enabled
valgrind detected two memory leaks:
1) quota context is not released after merging.
2) @refcount_orig should be released
E2fsprogs-commit:
d9aa090f713ecf3393c4e68d951a03ccba4a7073
Change-Id: I8b8379505524d25b5c18a0d67bb4dd8ede3494ef
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Wed, 23 Sep 2020 15:58:23 +0000 (23:58 +0800)]
tests: add pfsck test
pfsck run on a clean fs should not return any errors.
Generate an image with possible features enabled,
especially EA shared blocks etc.
E2fsprogs-commit:
59ae5e5a627aaa24d663e8ce41955e1ded46633a
Change-Id: Ifa8d725993cf8bec08e847e2734bb44b0ddbfd04
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Thu, 24 Sep 2020 02:00:06 +0000 (10:00 +0800)]
e2fsck: reset @inodes_to_rebuild if restart
Verify multiple thread on a corrupted images hit following bug:
pass1.c:2902: e2fsck_pass1_thread_prepare:
Assertion `global_ctx->inodes_to_rebuild == NULL' failed.
Signal (6) SIGABRT si_code=SI_TKILL
./e2fsck/e2fsck[0x43829e]
/lib64/libpthread.so.0(+0x14b20)[0x7f3b45135b20]
/lib64/libc.so.6(gsignal+0x145)[0x7f3b44f2c625]
/lib64/libc.so.6(abort+0x12b)[0x7f3b44f158d9]
/lib64/libc.so.6(+0x257a9)[0x7f3b44f157a9]
/lib64/libc.so.6(+0x34a66)[0x7f3b44f24a66]
./e2fsck/e2fsck(e2fsck_pass1+0x1662)[0x423572]
./e2fsck/e2fsck(e2fsck_run+0x5a)[0x41611a]
./e2fsck/e2fsck(main+0x1608)[0x4121b8]
/lib64/libc.so.6(__libc_start_main+0xf3)[0x7f3b44f171a3]
./e2fsck/e2fsck(_start+0x2e)[0x413dde]
@inodes_to_rebuild could be not NULL after we restart pass1
E2fsprogs-commit:
3b7879e310d30d5f4c518c607aec830b63b2315b
Change-Id: I8ca6a37d41788333fe9494bd84816932f39fcb3c
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Wed, 23 Sep 2020 14:36:41 +0000 (22:36 +0800)]
e2fsck: update mmp block in one thread
For multiple threads, different threads will try to
update mmp block at the same time, only allow one
thread to update it.
E2fsprogs-commit:
1806140014cbce395d0ca4a51d46fa21e2833241
Change-Id: Id78b949d338c384a64828b955ed7477aa816f8ee
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Tue, 15 Sep 2020 07:06:11 +0000 (15:06 +0800)]
e2fsck: make default smallest RA size to 1M
If we have a smaller inodes per group, default ra size could
be very small(etc 128KiB), this hurts performances.
Tune above 128K to 1M, i see pass1 time drop down from
677.12 seconds to 246 secons with 32 threads.
E2fsprogs-commit:
9226b3264b74044013f4933280b1989c33be42f3
Change-Id: Ia503b4f73dc8f3d5e6be705c7311bdacc5b874a9
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Wed, 27 May 2020 07:14:23 +0000 (07:14 +0000)]
e2fsck: avoid too much memory allocation for pfsck
e2fsck init memory according to filesystem inodes/dir numbers
recorded in the superblock, this should be aware of filesystem
number of threads, otherwise, oom happen.
E2fsprogs-commit:
ed2cec54da604efe1af2734182985d1730674138
Change-Id: Ia66cd782a330f3f0215c7119c25ef701edc8b143
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Wed, 23 Sep 2020 10:05:34 +0000 (18:05 +0800)]
e2fsck: cleanup e2fsck_pass1_thread_join()
Use e2fsck_reset_context() to free memory to simpify
codes.
E2fsprogs-commit:
41f40f4215fd9a357a3d250a34745eb953982bda
Change-Id: I1fdf12eb520f738d90453bbc886dbe89bc9a2b17
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Thu, 14 May 2020 14:16:50 +0000 (22:16 +0800)]
e2fsck: wait fix thread finish before checking
Before proceeding next inodes, waitting existed
fixing finished.
E2fsprogs-commit:
ae4a78493b16d01d7dbe628ec9064216f960bd16
Change-Id: I159082381e66bd51b3558a65ef45e936aa741370
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Thu, 16 Apr 2020 07:08:31 +0000 (15:08 +0800)]
e2fsck: set E2F_FLAG_ALLOC_OK after threads
Only flag ALLOC OK after all threads finished without problem.
E2fsprogs-commit:
2c1e9ced674f0d8e634c3b6299ce9bdfbe4b8739
Change-Id: I764d08bab5299ec0aec9d7d5e9b21320c085dee9
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Fri, 13 Mar 2020 10:46:24 +0000 (18:46 +0800)]
e2fsck: simplify e2fsck context merging codes
We tried to copy thread context to global context directly
and then copy back some saved variables before merging.
Since we have finished almost all necessary variables
in the e2fsck context, we could simplify codes, and
this could help us understand what is missing rather
than hide problems.
E2fsprogs-commit:
41c991af7ed7244da0fd799f63e4e6fd0d6c5b25
Change-Id: I450e3f5fb26c5c6fe8a075373bf538965f443d02
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Fri, 20 Mar 2020 01:55:18 +0000 (09:55 +0800)]
e2fsck: merge extent depth count after threads finish
tests covered by f_extent_htree.
E2fsprogs-commit:
291bc9b7fc5280166e58b0e586d5ec93738f31a7
Change-Id: I5b4f629e1031f719a2d032a9dd5edb1cfac90975
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Fri, 20 Mar 2020 01:50:56 +0000 (09:50 +0800)]
e2fsck: reset lost_and_found after threads finish
This should not be kept, the reaons is similar to what
e2fsck_pass1 has done before.
E2fsprogs-commit:
10abbf8fd534d61c40fbf9699bf605ac6456b2e2
Change-Id: I0e431be7619720e31927ab1df7034aee1e3eb747
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Thu, 19 Mar 2020 14:05:30 +0000 (22:05 +0800)]
e2fsck: merge options after threads finish
It will be possible that threads might append E2F_OPT_YES,
so we need merge options to global, test f_yesall cover this.
E2fsprogs-commit:
a68a5e890793eda2e8d7b4f7d27d23a7eeb474c5
Change-Id: I0e8bbc078fc538bd7d948ced1161506710401ab3
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Wed, 11 Mar 2020 12:48:00 +0000 (20:48 +0800)]
e2fsck: fix readahead for pfsck of pass1
Several improvments for this patch:
1) move readahead_kb detection to preparing phase.
2) inode readahead should be aware of thread block group
boundary.
3) make readahead_kb aware of multiple threads.
E2fsprogs-commit:
02618b8f69c1ed453b092476cbd0d735206c939d
Change-Id: I2f9d7b8ae5039c0551bb3994f1c7814af257cee5
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Wed, 9 Sep 2020 07:07:10 +0000 (15:07 +0800)]
e2fsck: adjust number of threads
number of threads should not exceed flex bg numbers,
and output messages if we adjust threads number.
E2fsprogs-commit:
56070582ad951aa55b6eeecd7e8e2a19d53bbff6
Change-Id: I8bc87e6ca99205d7fc9e0c9f35d0ba9c3eed38ca
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Mon, 9 Mar 2020 07:11:10 +0000 (15:11 +0800)]
e2fsck: allow admin specify number of threads
-m option is introduced to specify number of threads for pfsck.
E2fsprogs-commit:
e3bb22efe1c62f8c81686e1d8d065a4943e14bd3
Change-Id: I856d06e0c62f23bdd84e7b6bf53cdc4a46dd58da
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Wed, 11 Mar 2020 02:12:18 +0000 (10:12 +0800)]
e2fsck: kickoff mutex lock for block found map
Now @block_found_map is no longer shared by multiple threads,
and @block_dup_map need be checked again after threads finish.
E2fsprogs-commit:
17df3e9f85594eca5515630753542ab5a5c92ee1
Change-Id: I4fb9ae84b70b7ee7d7ce2c0f7a8876ab89c31b5c
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Thu, 12 Mar 2020 08:05:51 +0000 (16:05 +0800)]
e2fsck: merge EA blocks properly
EA blocks might be shared, merge them carefully.
E2fsprogs-commit:
b9c38a69ff1af2e0c2b9ddce96de965ce62b0de6
Change-Id: Ib01b7cf9c32f7e119bfd00972488b731b13f1ed9
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Sat, 14 Dec 2019 05:12:36 +0000 (13:12 +0800)]
e2fsck: split and merge invalid bitmaps
Invalid bitmaps are splitted per thread, and we
should merge them after thread finish.
E2fsprogs-commit:
e3e15f5c8f72f95d65d602929f59c1c78e72c5a0
Change-Id: I004f595c107adf390f3b16b90cb4b6aa3650f9de
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Sat, 7 Mar 2020 14:00:45 +0000 (22:00 +0800)]
e2fsck: move some fixes out of parallel pthreads
We could only use @found_map_block to find free blocks
after we have collectd all used blocks, so something like
handle_fs_bad_blocks(), ext2fs_create_resize_inode(),
e2fsck_pass1_dupblocks() really should be handled after
all threads has been finished.
E2fsprogs-commit:
6b1d459bf6018eea7ff0a0ae6b555276718dcda3
Change-Id: Ibdbaed3d218623daa0baec31837037d773799262
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Fri, 6 Mar 2020 08:46:47 +0000 (16:46 +0800)]
e2fsck: serialize fix operations
Allow different threads to fix at the same time could
be dangerous and error-prone now, and most of time
parallel scanning and checking is important.
So this patch adds a mutex to serialize
fix operations during pass1.
And the good benefit of this, we don't need block
allocations and free, superblock updates protection
any more, since only fix operations during pass1
could touch them.
E2fsprogs-commit:
5485120b668c952b6e83f24a1e1a74eaa7bf362e
Change-Id: Iedb64545240cc9325dc0c4ce737a970b27f4f3f6
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Fri, 6 Mar 2020 15:08:07 +0000 (23:08 +0800)]
e2fsck: merge quota context after threads finish
Every threads calculate its own quota accounting,
merge them after threads finish.
E2fsprogs-commit:
a0af18577fbb960eb20695afbb4af4e23b864909
Change-Id: If9f0e17e560715ed55183f85c443f21ebbce7d40
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Mon, 24 Feb 2020 05:08:58 +0000 (13:08 +0800)]
e2fsck: merge context flags properly
e2fsck might restart after pass1, so we should keep
flags if possible, this patch try to fix f_illitable_flexbg failure
E2fsprogs-commit:
c36fa1b28bd4171f53ab4f89a90f6b3b8bf80fd2
Change-Id: I6dda1520a2ff6255d244461af75d7609f47ba92b
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Mon, 24 Feb 2020 04:44:06 +0000 (12:44 +0800)]
e2fsck: merge dirs_to_hash when threads finish
@dirs_to_hash list need be merged after threads finish,
test covered by t_dangerous.
E2fsprogs-commit:
e2db6c19943065909c11af3b03c2904c35e5e4f8
Change-Id: I99b44c40afa22ce8e4530c874b077493cf969e03
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Fri, 6 Dec 2019 15:37:53 +0000 (23:37 +0800)]
e2fsck: merge dx_dir_info after threads finish
Merge properly.
E2fsprogs-commit:
45519b8167a00979e86d5a5aa95d0bfb07f72520
Change-Id: Ifaed7a0022b716c42df908327c41aaaaa39aa365
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Sun, 8 Sep 2019 08:15:35 +0000 (16:15 +0800)]
e2fsck: merge fs flags when threads finish
merge fs flags properly.
E2fsprogs-commit:
d4c0519725d59c9380acc828d24bc070c5bc2c03
Change-Id: I80c773bebc0e321bf2fe3b436c3f0e601b1f1b79
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Tue, 22 Sep 2020 13:14:39 +0000 (21:14 +0800)]
e2fsck: merge counts after threads finish
Merge counts properly.
E2fsprogs-commit:
057060ccabf9b5b47afe435d90a1f7f68c03a231
Change-Id: Ieace889f0d1a20a1990424113b0e8fc81f945266
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Thu, 5 Sep 2019 11:40:36 +0000 (19:40 +0800)]
e2fsck: add debug codes for multiple threads
These debug codes are added to run the multiple pass1 check
thread one by one in order. If all the codes are correct,
fsck of multiple threads should have exactly the same outcome
with single thread.
E2fsprogs-commit:
a8b483a1e7987a6dddc32bc82db06f649d245630
Change-Id: I41b0493a0cd830b76434ff0f86a87c98d8d3ad22
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Thu, 5 Sep 2019 08:30:40 +0000 (16:30 +0800)]
e2fsck: merge dblist after thread finishes
Merge dblist properly.
E2fsprogs-commit:
f3c2256490297201c63139ba0fea5c6b09bca3cf
Change-Id: Ifd907035ed9762c446c688ccc5af3f3fcaa52d24
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Wed, 4 Sep 2019 10:47:14 +0000 (18:47 +0800)]
e2fsck: merge icounts after thread finishes
Merge inode_count and inode_link_info properly after
threads finish.
E2fsprogs-commit:
bb78742b5e2e95c3e7e9a76755144fdf8ec12285
Change-Id: If40930a87595010963fd59a54709a4d2b27d223b
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Tue, 22 Sep 2020 10:00:39 +0000 (18:00 +0800)]
e2fsck: merge badblocks after thread finishes
Badblocks should be merged properly after threads finish.
E2fsprogs-commit:
f39197f53ef5d90cd6952d7e75f2d2e3a02bf7ef
Change-Id: Ieb9a96a5e4438270ba6e939155e2666bb53177e4
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Mon, 21 Sep 2020 11:02:33 +0000 (19:02 +0800)]
e2fsck: rbtree bitmap for dir
Only rbtree support merge operation now, use it for bitmaps.
E2fsprogs-commit:
9bab3284e30752bccc36be3e07b9ca4dfd218041
Change-Id: I42aeb009ece78a4baf4369aff3355fc9df51089a
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Mon, 2 Sep 2019 08:14:38 +0000 (16:14 +0800)]
e2fsck: merge dir_info after thread finishes
dir_info need be merged after thread finish.
E2fsprogs-commit:
fdbf5e226cf931a5713414001b95e4697997a464
Change-Id: I7e33c0035ec8c6bfa7c13d759e3edb468634eae3
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Fri, 30 Aug 2019 09:56:10 +0000 (17:56 +0800)]
e2fsck: optimize the inserting of dir_info_db
Binary search is now used when inserting an dir info to the array.
Memmove is now used when moving array. Both of them improves
the performance of inserting.
This patch is also a prepartion for the merging of two dir db
arrays.
E2fsprogs-commit:
13d3c76475d050941d244c85cbb9256ffd980f88
Change-Id: I69041084dbd7e36eefa44744c3fb3737af8e906e
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Fri, 30 Aug 2019 04:20:11 +0000 (12:20 +0800)]
e2fsck: do not change global variables
Global variables used in pass1 check are changed to local variables
in this patch. This will avoid conflict between threads.
E2fsprogs-commit:
5a1b22da5ca3e90728cee1dbe6bc1920b245dded
Change-Id: Ifc682874801af15e8e13764832ca569b7da4ac57
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Wed, 23 Sep 2020 03:17:29 +0000 (11:17 +0800)]
e2fsck: merge bitmaps after thread completes
A new method merge_bmap has been added to bitmap operations. But
only red-black bitmap has that operation now.
E2fsprogs-commit:
ecf85fb474fb7537f14bddd8f85d8372b808d5e9
Change-Id: Id5c387dd813d633adff3afb9208213086c33b6d2
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Mon, 26 Aug 2019 14:34:06 +0000 (22:34 +0800)]
e2fsck: print thread log properly
When multi-thread fsck is enabled, logs printed from multiple
threads could overlap with each other. The overlap sometimes
makes the logs unreadable because log_out() is used multiple times
for a single line.
This patch adds leading [Thread XXX] to each logs if multi-thread
is enabed by -m option.
This patch also adds message to show the group ranges and inode
numbers for each thread, which is useful for debuging multi-thread
check.
E2fsprogs-commit:
5b071ed80deca2cdc87585a673076529dd47ca6e
Change-Id: I4bd08bc48813a399ab3a79130ff159b12d22f3a0
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Thu, 22 Aug 2019 07:35:12 +0000 (15:35 +0800)]
e2fsck: split groups to different threads
The start/end groups of a thread is calculated according to the
thread number. But still, only one thread is used to check.
E2fsprogs-commit:
562df7468d97042689896cff4b48c6cebc3b8508
Change-Id: I3175b4e6f0423ecd69ed55afea45f1c5919f5c79
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Wed, 21 Aug 2019 13:58:31 +0000 (21:58 +0800)]
e2fsck: add start/end group for thread
When multi-threads are used for check, each thread needs to jump
to different group in pass1 check. This patch adds the group
jumping support. But still, only one thread is used to check.
E2fsprogs-commit:
f64ff7f53c6fb22c7e4bebe86749b1be31439424
Change-Id: I904bedcbd02bca591665943cff5f2d7a6fa50fbe
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Thu, 15 Aug 2019 08:33:19 +0000 (16:33 +0800)]
e2fsck: configure one pfsck thread
This patch creates only one thread to do pass1 check if pthreads are
enabled. The same codes can be used to create multiple threads, but
other functions need to be modified to get ready for that.
E2fsprogs-commit:
b2a9a40831dfbb6dacf8bbc819acac2c25ab6980
Change-Id: I3df998a8ecc00d2dc5e959f5a9991a6b65182572
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Tue, 13 Aug 2019 03:55:27 +0000 (11:55 +0800)]
e2fsck: create logs for mult-threads
When multi-threads are used, different logs should be created
for different threads. Each thread has log files with suffix
of ".$THREAD_INDEX".
And this patch adds f_multithread_logfile test case.
E2fsprogs-commit:
2643de7d00dc8da9f5be79fab768fca7a27f3745
Change-Id: I86982866b9e8e2d2cc5ac2ec9ac44a3b802c21e0
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Sat, 10 Aug 2019 07:59:09 +0000 (15:59 +0800)]
e2fsck: open io-channel when copying fs
This patch also add writethrough flag to the thread io-channel.
When multiple threads write the same disk, we don't want the
data being saved in memory cache. This will be useful in the
future, but even without that flag, the tests can be passed too.
This patch also cleanup the io channel cache of the global
context. Otherwise, after pass1 step, the next steps would use
old data saved in the cache. And the cached data might have
already been overwritten in pass1.
E2fsprogs-commit:
f15100003400e5fc01b34c25a7a4b61f5ed76148
Change-Id: I7fcf4038b6cb4eb52808b9a5bafdf909ca2c0c8e
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Wed, 9 Sep 2020 07:07:10 +0000 (15:07 +0800)]
e2fsck: copy badblocks when copying fs
This patch copies badblocks when the copying fs.
E2fsprogs-commit:
8095b716e15bb15a47925ae02cb1c06e91fdb651
Change-Id: Ibf5718ad5851a9ccd853a2a1f928bce447aa5de9
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Wed, 7 Aug 2019 08:54:41 +0000 (16:54 +0800)]
e2fsck: copy bitmaps when copying context
This patch copies bitmap when the copying context. In the
multi-thread fsck, each thread use different bitmap that copied
from the glboal bitmap. And Bitmaps from multiple threads will
be merged into a global one after the pass1 finishes.
E2fsprogs-commit:
4d47d0e84388bdf2242e09f9ba592ea778c1ba8e
Change-Id: I6a5a2b49b845215a7a735a828709396586520536
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Wed, 7 Aug 2019 07:34:00 +0000 (15:34 +0800)]
e2fsck: add assert when copying context
Adding the assert would simplify the copying of context.
E2fsprogs-commit:
464abd60f5b68716d8ac5063d90562f9ecd2b004
Change-Id: I8276c2e90f40b08dea28b157e04df3c0f7099211
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Wang Shilong [Thu, 17 Sep 2020 02:58:56 +0000 (10:58 +0800)]
e2fsck: clear icache when using multi-thread fsck
icache of fs will be rebuilt when needed, so after copying
fs, icache can be inited to NULL.
E2fsprogs-commit:
5d4f09fa148e41b4e0ea01fc364de5bbedf02ecf
Change-Id: I21c58b3f126fd85008d6c732da71b298b2a8b4ff
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Tue, 6 Aug 2019 03:19:15 +0000 (11:19 +0800)]
e2fsck: copy fs when using multi-thread fsck
This patch only copy the fs to a new one when -m is enabled.
It doesn't actually start any thread. When pass1 test finishes,
the new fs is copied back to the original context.
This patch handles the fs fields in dblist, inode_map and block_map
properly.
E2fsprogs-commit:
d88f9ae76882fc1210158ca3558c0bbad6676a79
Change-Id: Iab763de64e63366bc413d954dd7c8a8af45cb6a2
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Mon, 5 Aug 2019 04:01:17 +0000 (12:01 +0800)]
e2fsck: copy context when using multi-thread fsck
This patch only copy the context to a new one when -m is enabled.
It doesn't actually start any thread. When pass1 test finishes,
the new context is copied back to the original context.
Since the signal handler only changes the original context, so
add global_ctx in "struct e2fsck_struct" and use that to check
whether there is any signal of canceling.
This patch handles the long jump properly so that all the existing
tests can be passed even the context has been copied. Otherwise,
test f_expisize_ea_del would fail when aborting.
E2fsprogs-commit:
05a637a9e68d3f7e15323deeab00981d4b7df7e8
Change-Id: I4994e62bdf27c385b02e55de82a9dafcc6a12139
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Li Xi [Sun, 4 Aug 2019 15:28:59 +0000 (23:28 +0800)]
e2fsck: add -m option for multithread
-m option is added but no actual functionality is added. This
patch only adds the logic that when -m is specified, one of
-p/-y/-n options should be specified. And when -m is specified,
-C shouldn't be specified and the completion progress report won't
be triggered by sending SIGUSR1/SIGUSR2 signals. This simplifies
the implementation of multi-thread fsck in the future.
Completion progress support with multi-thread fsck will be added
back after multi-thread fsck implementation is finished. Right
now, disable it to simplify the implementation of multi-thread fsck.
E2fsprogs-commit:
842ae5f528f9797e4b2c739daa76884e4349e8f3
Change-Id: I428f4f3b10974b769100bf00169bdd7d8cf86deb
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 28 Feb 2021 14:57:25 +0000 (09:57 -0500)]
Update release notes, etc., for the 1.46.2 release
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 28 Feb 2021 23:52:20 +0000 (18:52 -0500)]
libext2fs: fix potential races in unix_io
When unix_io does not use pread/pread64 (which is the case the bounce
buffer is in use, either when Direct I/O is in use or the
IO_FLAG_FORCE_BOUNCE in enabled), there are races between the llseek
and and read or write system calls. Fix this by using the BOUNCE_MTX
so only one thread is using the file descriptor at a time.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 28 Feb 2021 14:12:47 +0000 (09:12 -0500)]
libext2s: fix unix_io with IO_FLAG_FORCE_BOUNCE flag set
The bounce read/write code would crash with a floating point exception
if alignment is set to 0.
Fixes:
c001596110e8 ("libext2fs: fix unix_io's Direct I/O support")
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Antonio Ceballos [Sat, 27 Feb 2021 15:24:06 +0000 (10:24 -0500)]
po: update es.po (from translationproject.org)
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 26 Feb 2021 22:41:06 +0000 (17:41 -0500)]
libext2fs: fix unix_io's Direct I/O support
The previous Direct I/O support worked on HDD's with 512 byte logical
sector sizes, and on FreeBSD which required 4k aligned memory buffers.
However, it was incomplete and was not correctly working on HDD's with
4k logical sector sizes (aka Advanced Format Disks).
Based on a patch from Alexey Lyashkov <alexey.lyashkov@hpe.com> but
rewritten to work with the latest e2fsprogs and to minimize changes to
make it easier to review.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reported-by: Artem Blagodarenko <artem.blagodarenko@gmail.com>
Theodore Ts'o [Fri, 26 Feb 2021 22:18:48 +0000 (17:18 -0500)]
iscan: fix the test program iscan so it builds again
The iscan program program isn't built by default, and was relying on
e2fsck's util.c, so it had suffered bitrot as e2fsck/util.c had
evolved. Fix it so that iscan builds correct.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Lukas Czerner [Thu, 18 Feb 2021 09:51:46 +0000 (10:51 +0100)]
mmp: do not use O_DIRECT when working with regular file
Currently the mmp block is read using O_DIRECT to avoid any caching that
may be done by the VM. However when working with regular files this
creates alignment issues when the device of the host file system has
sector size larger than the blocksize of the file system in the file
we're working with.
This can be reproduced with t_mmp_fail test when run on the device with
4k sector size because the mke2fs fails when trying to read the mmp
block.
Fix it by disabling O_DIRECT when working with regular files. I don't
think there is any risk of doing so since the file system layer, unlike
shared block device, should guarantee cache consistency.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 26 Feb 2021 02:40:27 +0000 (21:40 -0500)]
resize2fs: prevent block bitmap warnings when doing extreme fs expansions
This commit fixes a bug where if a small file system is resized to
ridiculous sizes, such that the size of the resized block group
descriptor blocks exceed the original file system, and this would
result in resize2fs triggering a large number of scary warning
messages:
Illegal block number passed to ext2fs_test_block_bitmap #12440
for block bitmap for broken.img
This can be replicated via:
./misc/mke2fs broken.img -b 1024 -T default
truncate -s +1500G broken.img
./resize/resize2fs broken.img
Fortunately, aside from triggering these warning messages, the bug had
no other bad effects.
https://github.com/tytso/e2fsprogs/issues/60
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Thu, 25 Feb 2021 22:26:07 +0000 (17:26 -0500)]
debugfs: fix rdump and ls to handle uids and gids > 65536 correctly
https://github.com/tytso/e2fsprogs/issues/63
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Thu, 25 Feb 2021 17:39:01 +0000 (12:39 -0500)]
Remove superfluous break statements
https://github.com/tytso/e2fsprogs/pull/45
https://github.com/tytso/e2fsprogs/pull/46
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Lennart Poettering [Thu, 25 Feb 2021 17:33:35 +0000 (12:33 -0500)]
chattr/lsattr: expose FS_NOCOMP_FL (aka EXT2_NOCOMPR_FL)
This adds support for setting/querying the FS_NOCOMP_FL/EXT2_NOCOMPR_FL
file flag to chattr/lsattr. I picked the character "m" because it was
so far unused and all other characters that were more obvious candidates
were already taken.
The flag is available on btrfs, and with this patch it is possible to
manage it correctly.
Signed-off-by: Lennart Poettering <lennart@poettering.net>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Zhiqiang Liu [Sat, 20 Feb 2021 08:41:29 +0000 (16:41 +0800)]
debugfs: fix memory leak problem in read_list()
In read_list func, if strtoull() fails in while loop,
we will return the error code directly. Then, memory of
variable lst will be leaked without setting to *list.
Signed-off-by: Zhiqiang Liu <liuzhiqiang26@huawei.com>
Signed-off-by: linfeilong <linfeilong@huawei.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 24 Feb 2021 05:08:07 +0000 (00:08 -0500)]
libe2p: remove nested #ifdefs in fgetflags.c and fsetflags.c
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 24 Feb 2021 04:57:10 +0000 (23:57 -0500)]
debugfs: fix file descriptor leak on error path in do_logdump()
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 24 Feb 2021 03:03:59 +0000 (22:03 -0500)]
misc/Makefile.in: add rules to build static versions of lsattr and chattr
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 24 Feb 2021 02:25:46 +0000 (21:25 -0500)]
libe2p: skip unneeded lstat(2) in fgetflags() and fsetflags()
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 24 Feb 2021 01:58:31 +0000 (20:58 -0500)]
debugfs: fix uninitialized variable es in do_logdump()
Fixes:
d96064e9821a ("debugfs: simplify the do_logdump() function")
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Tue, 23 Feb 2021 21:19:00 +0000 (16:19 -0500)]
debugfs: simplify the do_logdump() function
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Tue, 23 Feb 2021 21:02:42 +0000 (16:02 -0500)]
Add checks for fs->blocksize == 0 which could cause some crashes
This should never happeb, but some checks is useful, and also fixes
some Coverity warnings.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Tue, 23 Feb 2021 18:03:37 +0000 (13:03 -0500)]
e2fsck: fix miscellaneous clang warnings
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Harshad Shirwadkar [Tue, 23 Feb 2021 17:41:56 +0000 (09:41 -0800)]
e2fsck: initialize variable before first use in fast commit replay
Initialize ext2fs_ex variable in ext4_fc_replay_scan() before first
use. Also make sure ext2fs_decode_extent() completely overwrites the
extent structure passed to it as argument to prevent potential future
bugs for the users of the function.
Signed-off-by: Harshad Shirwadkar <harshadshirwadkar@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Harshad Shirwadkar [Tue, 23 Feb 2021 17:41:55 +0000 (09:41 -0800)]
e2fsck: add fallthrough comment in fc replay switch case
During fast commit replay scan phase, in ext4_fc_replay_scan(), we
want to fallthrough in switch case for EXT4_FC_TAG_ADD_RANGE case. Add
a comment for that.
Signed-off-by: Harshad Shirwadkar <harshadshirwadkar@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Harshad Shirwadkar [Tue, 23 Feb 2021 17:41:54 +0000 (09:41 -0800)]
ext2fs: don't ignore return value in ext2fs_count_blocks
Don't ignore return value of ext2fs_get_array() in
ext2fs_count_blocks().
Signed-off-by: Harshad Shirwadkar <harshadshirwadkar@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Harshad Shirwadkar [Tue, 23 Feb 2021 17:41:53 +0000 (09:41 -0800)]
e2fsck: don't ignore return values in e2fsck_rewrite_extent_tree
Don't ignore return values of library function calls in
e2fsck_rewrite_extent_tree.
Signed-off-by: Harshad Shirwadkar <harshadshirwadkar@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Tue, 23 Feb 2021 17:56:01 +0000 (12:56 -0500)]
create_inode: fix potential file descriptor leak on an error path
Addresses-Coverity-Bug: 1473307
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Tue, 23 Feb 2021 02:53:53 +0000 (21:53 -0500)]
misc: replace remaining loff_t with ext2_loff_t
Replace the remaining loff_t uses with ext2_loff_t, as
was done in patch
1df6a4555, since loff_t is a GCC'ism
and is not portable.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Akatsuki Rui [Mon, 22 Feb 2021 16:42:38 +0000 (11:42 -0500)]
chattr.1: improve attributes description to btrfs
- c: for btrfs, compression are conflicts with nodatacow and nodatasum
- C: for btrfs, nodatacow is conflict with compression
ref:
- https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs(5)#MOUNT_OPTIONS
- https://www.spinics.net/lists/linux-btrfs/msg103219.html
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Earl Chew [Sat, 20 Feb 2021 16:47:46 +0000 (08:47 -0800)]
create_inode: Find subdirectory in do_write_internal
Follow the example in do_mkdir_internal, and do_symlink_internal,
and find the correct parent directory if the destination
is not just a plain basename.
https://github.com/tytso/e2fsprogs/issues/61
Signed-off-by: Earl Chew <earl_chew@yahoo.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Tue, 16 Feb 2021 05:30:24 +0000 (00:30 -0500)]
debugfs: fix dump_metadata_block() for block sizes > 8192
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 15 Feb 2021 04:51:45 +0000 (23:51 -0500)]
libext2fs: fix crash when ext2fs_mmp_stop() is called before MMP is initialized
The fatal_error() function in e2fsck can call ext2fs_mmp_stop() on a
file system where MMP hasn't yet been initialized. When that happens,
instead of crashing, have ext2fs_mmp_stop() return success, since mmp
doesn't need to be stopped if it hasn't even been initialized yet.
Addresses-Debian-Bug: #696609
Signed-off-by: Theodore Ts'o <tytso@mit.edu>