Whamcloud - gitweb
tools/e2fsprogs.git
12 months agoe2fsck: wait fix thread finish before checking
Wang Shilong [Thu, 14 May 2020 14:16:50 +0000 (22:16 +0800)]
e2fsck: wait fix thread finish before checking

Before proceeding next inodes, waitting existed
fixing finished.

E2fsprogs-commit: ae4a78493b16d01d7dbe628ec9064216f960bd16

Change-Id: I159082381e66bd51b3558a65ef45e936aa741370
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: set E2F_FLAG_ALLOC_OK after threads
Wang Shilong [Thu, 16 Apr 2020 07:08:31 +0000 (15:08 +0800)]
e2fsck: set E2F_FLAG_ALLOC_OK after threads

Only flag ALLOC OK after all threads finished without problem.

E2fsprogs-commit: 2c1e9ced674f0d8e634c3b6299ce9bdfbe4b8739

Change-Id: I764d08bab5299ec0aec9d7d5e9b21320c085dee9
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoLU-14768 e2fsck: merge casefolded dir lists after thread finish
Wang Shilong [Thu, 12 Mar 2020 11:40:27 +0000 (19:40 +0800)]
LU-14768 e2fsck: merge casefolded dir lists after thread finish

This is missed and should be done after threads finish.

Original commit:
18538b27 ("LU-8465 e2fsck: merge encrypted dir lists after thread finish")
Upstream is now using e2fsck_struct->casefolded_dirs instead of
encrypted_dirs.

Only for the correctness of pfsck, we won't use casefolded feature on
ldiskfs.

Change-Id: I6f3a28501886c2ed4bec26dc5c5e5a1914fece04
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/44010
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
12 months agoe2fsck: simplify e2fsck context merging codes
Wang Shilong [Fri, 13 Mar 2020 10:46:24 +0000 (18:46 +0800)]
e2fsck: simplify e2fsck context merging codes

We tried to copy thread context to global context directly
and then copy back some saved variables before merging.

Since we have finished almost all necessary variables
in the e2fsck context, we could simplify codes, and
this could help us understand what is missing rather
than hide problems.

E2fsprogs-commit: 41c991af7ed7244da0fd799f63e4e6fd0d6c5b25

Change-Id: I450e3f5fb26c5c6fe8a075373bf538965f443d02
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: merge extent depth count after threads finish
Wang Shilong [Fri, 20 Mar 2020 01:55:18 +0000 (09:55 +0800)]
e2fsck: merge extent depth count after threads finish

tests covered by f_extent_htree.

E2fsprogs-commit: 291bc9b7fc5280166e58b0e586d5ec93738f31a7

Change-Id: I5b4f629e1031f719a2d032a9dd5edb1cfac90975
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: reset lost_and_found after threads finish
Wang Shilong [Fri, 20 Mar 2020 01:50:56 +0000 (09:50 +0800)]
e2fsck: reset lost_and_found after threads finish

This should not be kept, the reaons is similar to what
e2fsck_pass1 has done before.

E2fsprogs-commit: 10abbf8fd534d61c40fbf9699bf605ac6456b2e2

Change-Id: I0e431be7619720e31927ab1df7034aee1e3eb747
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: merge options after threads finish
Wang Shilong [Thu, 19 Mar 2020 14:05:30 +0000 (22:05 +0800)]
e2fsck: merge options after threads finish

It will be possible that threads might append E2F_OPT_YES,
so we need merge options to global, test f_yesall cover this.

E2fsprogs-commit: a68a5e890793eda2e8d7b4f7d27d23a7eeb474c5

Change-Id: I0e8bbc078fc538bd7d948ced1161506710401ab3
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: fix readahead for pfsck of pass1
Wang Shilong [Wed, 11 Mar 2020 12:48:00 +0000 (20:48 +0800)]
e2fsck: fix readahead for pfsck of pass1

Several improvments for this patch:

1) move readahead_kb detection to preparing phase.
2) inode readahead should be aware of thread block group
boundary.
3) make readahead_kb aware of multiple threads.

E2fsprogs-commit: 02618b8f69c1ed453b092476cbd0d735206c939d

Change-Id: I2f9d7b8ae5039c0551bb3994f1c7814af257cee5
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoLU-14432 ext2fs: fix ext2fs_get_avg_group() warning
Andreas Dilger [Mon, 31 May 2021 21:08:13 +0000 (15:08 -0600)]
LU-14432 ext2fs: fix ext2fs_get_avg_group() warning

Move ext2fs_get_avg_group() into e2fsck/pass1.c instead of the
ext2fs.h header, since it is currently only used in that one
function.  It is too large to be a static inline function, and
it is currently causing many compile warnings because it is
not marked with "_INLINE_".

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: Ie948bc10f44dccf91558099a5217ea1fb5ce7057
Reviewed-on: https://review.whamcloud.com/43881
Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
12 months agoe2fsck: adjust number of threads
Wang Shilong [Wed, 9 Sep 2020 07:07:10 +0000 (15:07 +0800)]
e2fsck: adjust number of threads

number of threads should not exceed flex bg numbers,
and output messages if we adjust threads number.

E2fsprogs-commit: 56070582ad951aa55b6eeecd7e8e2a19d53bbff6

Change-Id: I8bc87e6ca99205d7fc9e0c9f35d0ba9c3eed38ca
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: allow admin specify number of threads
Wang Shilong [Mon, 9 Mar 2020 07:11:10 +0000 (15:11 +0800)]
e2fsck: allow admin specify number of threads

-m option is introduced to specify number of threads for pfsck.

E2fsprogs-commit: e3bb22efe1c62f8c81686e1d8d065a4943e14bd3

Change-Id: I856d06e0c62f23bdd84e7b6bf53cdc4a46dd58da
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: kickoff mutex lock for block found map
Wang Shilong [Wed, 11 Mar 2020 02:12:18 +0000 (10:12 +0800)]
e2fsck: kickoff mutex lock for block found map

Now @block_found_map is no longer shared by multiple threads,
and @block_dup_map need be checked again after threads finish.

E2fsprogs-commit: 17df3e9f85594eca5515630753542ab5a5c92ee1

Change-Id: I4fb9ae84b70b7ee7d7ce2c0f7a8876ab89c31b5c
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoLU-16972 e2fsck: fix merging ea_inode_refs
Li Dongyang [Wed, 26 Jul 2023 05:10:11 +0000 (15:10 +1000)]
LU-16972 e2fsck: fix merging ea_inode_refs

ea_inode_refs is used to track refs to ea inodes,
which has nothing to do with blocks.

Change-Id: Icd965a8e3a92edc51754c132b55044af9abbb9e4
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/c/tools/e2fsprogs/+/51763
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
12 months agoe2fsck: merge EA blocks properly
Wang Shilong [Thu, 12 Mar 2020 08:05:51 +0000 (16:05 +0800)]
e2fsck: merge EA blocks properly

EA blocks might be shared, merge them carefully.

E2fsprogs-commit: b9c38a69ff1af2e0c2b9ddce96de965ce62b0de6

Change-Id: Ib01b7cf9c32f7e119bfd00972488b731b13f1ed9
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: split and merge invalid bitmaps
Wang Shilong [Sat, 14 Dec 2019 05:12:36 +0000 (13:12 +0800)]
e2fsck: split and merge invalid bitmaps

Invalid bitmaps are splitted per thread, and we
should merge them after thread finish.

E2fsprogs-commit: e3e15f5c8f72f95d65d602929f59c1c78e72c5a0

Change-Id: I004f595c107adf390f3b16b90cb4b6aa3650f9de
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: move some fixes out of parallel pthreads
Wang Shilong [Sat, 7 Mar 2020 14:00:45 +0000 (22:00 +0800)]
e2fsck: move some fixes out of parallel pthreads

We could only use @found_map_block to find free blocks
after we have collectd all used blocks, so something like
handle_fs_bad_blocks(), ext2fs_create_resize_inode(),
e2fsck_pass1_dupblocks() really should be handled after
all threads has been finished.

E2fsprogs-commit: 6b1d459bf6018eea7ff0a0ae6b555276718dcda3

Change-Id: Ibdbaed3d218623daa0baec31837037d773799262
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: serialize fix operations
Wang Shilong [Fri, 6 Mar 2020 08:46:47 +0000 (16:46 +0800)]
e2fsck: serialize fix operations

Allow different threads to fix at the same time could
be dangerous and error-prone now, and most of time
parallel scanning and checking is important.

So this patch adds a mutex to serialize
fix operations during pass1.

And the good benefit of this, we don't need block
allocations and free, superblock updates protection
any more, since only fix operations during pass1
could touch them.

E2fsprogs-commit: 5485120b668c952b6e83f24a1e1a74eaa7bf362e

Change-Id: Iedb64545240cc9325dc0c4ce737a970b27f4f3f6
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: merge quota context after threads finish
Wang Shilong [Fri, 6 Mar 2020 15:08:07 +0000 (23:08 +0800)]
e2fsck: merge quota context after threads finish

Every threads calculate its own quota accounting,
merge them after threads finish.

E2fsprogs-commit: a0af18577fbb960eb20695afbb4af4e23b864909

Change-Id: If9f0e17e560715ed55183f85c443f21ebbce7d40
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: merge context flags properly
Wang Shilong [Mon, 24 Feb 2020 05:08:58 +0000 (13:08 +0800)]
e2fsck: merge context flags properly

e2fsck might restart after pass1, so we should keep
flags if possible, this patch try to fix f_illitable_flexbg failure

E2fsprogs-commit: c36fa1b28bd4171f53ab4f89a90f6b3b8bf80fd2

Change-Id: I6dda1520a2ff6255d244461af75d7609f47ba92b
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: merge dirs_to_hash when threads finish
Wang Shilong [Mon, 24 Feb 2020 04:44:06 +0000 (12:44 +0800)]
e2fsck: merge dirs_to_hash when threads finish

@dirs_to_hash list need be merged after threads finish,
test covered by t_dangerous.

E2fsprogs-commit: e2db6c19943065909c11af3b03c2904c35e5e4f8

Change-Id: I99b44c40afa22ce8e4530c874b077493cf969e03
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: merge dx_dir_info after threads finish
Wang Shilong [Fri, 6 Dec 2019 15:37:53 +0000 (23:37 +0800)]
e2fsck: merge dx_dir_info after threads finish

Merge properly.

E2fsprogs-commit: 45519b8167a00979e86d5a5aa95d0bfb07f72520

Change-Id: Ifaed7a0022b716c42df908327c41aaaaa39aa365
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: merge fs flags when threads finish
Li Xi [Sun, 8 Sep 2019 08:15:35 +0000 (16:15 +0800)]
e2fsck: merge fs flags when threads finish

merge fs flags properly.

E2fsprogs-commit: d4c0519725d59c9380acc828d24bc070c5bc2c03

Change-Id: I80c773bebc0e321bf2fe3b436c3f0e601b1f1b79
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: merge counts after threads finish
Wang Shilong [Tue, 22 Sep 2020 13:14:39 +0000 (21:14 +0800)]
e2fsck: merge counts after threads finish

Merge counts properly.

E2fsprogs-commit: 057060ccabf9b5b47afe435d90a1f7f68c03a231

Change-Id: Ieace889f0d1a20a1990424113b0e8fc81f945266
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: add debug codes for multiple threads
Li Xi [Thu, 5 Sep 2019 11:40:36 +0000 (19:40 +0800)]
e2fsck: add debug codes for multiple threads

These debug codes are added to run the multiple pass1 check
thread one by one in order. If all the codes are correct,
fsck of multiple threads should have exactly the same outcome
with single thread.

E2fsprogs-commit: a8b483a1e7987a6dddc32bc82db06f649d245630

Change-Id: I41b0493a0cd830b76434ff0f86a87c98d8d3ad22
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: merge dblist after thread finishes
Li Xi [Thu, 5 Sep 2019 08:30:40 +0000 (16:30 +0800)]
e2fsck: merge dblist after thread finishes

Merge dblist properly.

E2fsprogs-commit: f3c2256490297201c63139ba0fea5c6b09bca3cf

Change-Id: Ifd907035ed9762c446c688ccc5af3f3fcaa52d24
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: merge icounts after thread finishes
Li Xi [Wed, 4 Sep 2019 10:47:14 +0000 (18:47 +0800)]
e2fsck: merge icounts after thread finishes

Merge inode_count and inode_link_info properly after
threads finish.

E2fsprogs-commit: bb78742b5e2e95c3e7e9a76755144fdf8ec12285

Change-Id: If40930a87595010963fd59a54709a4d2b27d223b
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: merge badblocks after thread finishes
Wang Shilong [Tue, 22 Sep 2020 10:00:39 +0000 (18:00 +0800)]
e2fsck: merge badblocks after thread finishes

Badblocks should be merged properly after threads finish.

E2fsprogs-commit: f39197f53ef5d90cd6952d7e75f2d2e3a02bf7ef

Change-Id: Ieb9a96a5e4438270ba6e939155e2666bb53177e4
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: rbtree bitmap for dir
Wang Shilong [Mon, 21 Sep 2020 11:02:33 +0000 (19:02 +0800)]
e2fsck: rbtree bitmap for dir

Only rbtree support merge operation now, use it for bitmaps.

E2fsprogs-commit: 9bab3284e30752bccc36be3e07b9ca4dfd218041

Change-Id: I42aeb009ece78a4baf4369aff3355fc9df51089a
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: merge dir_info after thread finishes
Li Xi [Mon, 2 Sep 2019 08:14:38 +0000 (16:14 +0800)]
e2fsck: merge dir_info after thread finishes

dir_info need be merged after thread finish.

E2fsprogs-commit: fdbf5e226cf931a5713414001b95e4697997a464

Change-Id: I7e33c0035ec8c6bfa7c13d759e3edb468634eae3
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: optimize the inserting of dir_info_db
Li Xi [Fri, 30 Aug 2019 09:56:10 +0000 (17:56 +0800)]
e2fsck: optimize the inserting of dir_info_db

Binary search is now used when inserting an dir info to the array.
Memmove is now used when moving array. Both of them improves
the performance of inserting.

This patch is also a prepartion for the merging of two dir db
arrays.

E2fsprogs-commit: 13d3c76475d050941d244c85cbb9256ffd980f88

Change-Id: I69041084dbd7e36eefa44744c3fb3737af8e906e
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: do not change global variables
Li Xi [Fri, 30 Aug 2019 04:20:11 +0000 (12:20 +0800)]
e2fsck: do not change global variables

Global variables used in pass1 check are changed to local variables
in this patch. This will avoid conflict between threads.

E2fsprogs-commit: 5a1b22da5ca3e90728cee1dbe6bc1920b245dded

Change-Id: Ifc682874801af15e8e13764832ca569b7da4ac57
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: merge bitmaps after thread completes
Wang Shilong [Wed, 23 Sep 2020 03:17:29 +0000 (11:17 +0800)]
e2fsck: merge bitmaps after thread completes

A new method merge_bmap has been added to bitmap operations. But
only red-black bitmap has that operation now.

E2fsprogs-commit: ecf85fb474fb7537f14bddd8f85d8372b808d5e9

Change-Id: Id5c387dd813d633adff3afb9208213086c33b6d2
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: print thread log properly
Li Xi [Mon, 26 Aug 2019 14:34:06 +0000 (22:34 +0800)]
e2fsck: print thread log properly

When multi-thread fsck is enabled, logs printed from multiple
threads could overlap with each other. The overlap sometimes
makes the logs unreadable because log_out() is used multiple times
for a single line.

This patch adds leading [Thread XXX] to each logs if multi-thread
is enabed by -m option.

This patch also adds message to show the group ranges and inode
numbers for each thread, which is useful for debuging multi-thread
check.

E2fsprogs-commit: 5b071ed80deca2cdc87585a673076529dd47ca6e

Change-Id: I4bd08bc48813a399ab3a79130ff159b12d22f3a0
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: split groups to different threads
Li Xi [Thu, 22 Aug 2019 07:35:12 +0000 (15:35 +0800)]
e2fsck: split groups to different threads

The start/end groups of a thread is calculated according to the
thread number. But still, only one thread is used to check.

E2fsprogs-commit: 562df7468d97042689896cff4b48c6cebc3b8508

Change-Id: I3175b4e6f0423ecd69ed55afea45f1c5919f5c79
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: add start/end group for thread
Li Xi [Wed, 21 Aug 2019 13:58:31 +0000 (21:58 +0800)]
e2fsck: add start/end group for thread

When multi-threads are used for check, each thread needs to jump
to different group in pass1 check. This patch adds the group
jumping support. But still, only one thread is used to check.

E2fsprogs-commit: f64ff7f53c6fb22c7e4bebe86749b1be31439424

Change-Id: I904bedcbd02bca591665943cff5f2d7a6fa50fbe
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: configure one pfsck thread
Li Xi [Thu, 15 Aug 2019 08:33:19 +0000 (16:33 +0800)]
e2fsck: configure one pfsck thread

This patch creates only one thread to do pass1 check if pthreads are
enabled. The same codes can be used to create multiple threads, but
other functions need to be modified to get ready for that.

E2fsprogs-commit: b2a9a40831dfbb6dacf8bbc819acac2c25ab6980

Change-Id: I3df998a8ecc00d2dc5e959f5a9991a6b65182572
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: create logs for mult-threads
Li Xi [Tue, 13 Aug 2019 03:55:27 +0000 (11:55 +0800)]
e2fsck: create logs for mult-threads

When multi-threads are used, different logs should be created
for different threads. Each thread has log files with suffix
of ".$THREAD_INDEX".

And this patch adds f_multithread_logfile test case.

E2fsprogs-commit: 2643de7d00dc8da9f5be79fab768fca7a27f3745

Change-Id: I86982866b9e8e2d2cc5ac2ec9ac44a3b802c21e0
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: open io-channel when copying fs
Li Xi [Sat, 10 Aug 2019 07:59:09 +0000 (15:59 +0800)]
e2fsck: open io-channel when copying fs

This patch also add writethrough flag to the thread io-channel.
When multiple threads write the same disk, we don't want the
data being saved in memory cache. This will be useful in the
future, but even without that flag, the tests can be passed too.

This patch also cleanup the io channel cache of the global
context. Otherwise, after pass1 step, the next steps would use
old data saved in the cache. And the cached data might have
already been overwritten in pass1.

E2fsprogs-commit: f15100003400e5fc01b34c25a7a4b61f5ed76148

Change-Id: I7fcf4038b6cb4eb52808b9a5bafdf909ca2c0c8e
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: copy badblocks when copying fs
Wang Shilong [Wed, 9 Sep 2020 07:07:10 +0000 (15:07 +0800)]
e2fsck: copy badblocks when copying fs

This patch copies badblocks when the copying fs.

E2fsprogs-commit: 8095b716e15bb15a47925ae02cb1c06e91fdb651

Change-Id: Ibf5718ad5851a9ccd853a2a1f928bce447aa5de9
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: copy bitmaps when copying context
Li Xi [Wed, 7 Aug 2019 08:54:41 +0000 (16:54 +0800)]
e2fsck: copy bitmaps when copying context

This patch copies bitmap when the copying context. In the
multi-thread fsck, each thread use different bitmap that copied
from the glboal bitmap. And Bitmaps from multiple threads will
be merged into a global one after the pass1 finishes.

E2fsprogs-commit: 4d47d0e84388bdf2242e09f9ba592ea778c1ba8e

Change-Id: I6a5a2b49b845215a7a735a828709396586520536
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: add assert when copying context
Li Xi [Wed, 7 Aug 2019 07:34:00 +0000 (15:34 +0800)]
e2fsck: add assert when copying context

Adding the assert would simplify the copying of context.

E2fsprogs-commit: 464abd60f5b68716d8ac5063d90562f9ecd2b004

Change-Id: I8276c2e90f40b08dea28b157e04df3c0f7099211
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: clear icache when using multi-thread fsck
Wang Shilong [Thu, 17 Sep 2020 02:58:56 +0000 (10:58 +0800)]
e2fsck: clear icache when using multi-thread fsck

icache of fs will be rebuilt when needed, so after copying
fs, icache can be inited to NULL.

E2fsprogs-commit: 5d4f09fa148e41b4e0ea01fc364de5bbedf02ecf

Change-Id: I21c58b3f126fd85008d6c732da71b298b2a8b4ff
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: copy fs when using multi-thread fsck
Li Xi [Tue, 6 Aug 2019 03:19:15 +0000 (11:19 +0800)]
e2fsck: copy fs when using multi-thread fsck

This patch only copy the fs to a new one when -m is enabled.
It doesn't actually start any thread. When pass1 test finishes,
the new fs is copied back to the original context.

This patch handles the fs fields in dblist, inode_map and block_map
properly.

E2fsprogs-commit: d88f9ae76882fc1210158ca3558c0bbad6676a79

Change-Id: Iab763de64e63366bc413d954dd7c8a8af45cb6a2
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: copy context when using multi-thread fsck
Li Xi [Mon, 5 Aug 2019 04:01:17 +0000 (12:01 +0800)]
e2fsck: copy context when using multi-thread fsck

This patch only copy the context to a new one when -m is enabled.
It doesn't actually start any thread. When pass1 test finishes,
the new context is copied back to the original context.

Since the signal handler only changes the original context, so
add global_ctx in "struct e2fsck_struct" and use that to check
whether there is any signal of canceling.

This patch handles the long jump properly so that all the existing
tests can be passed even the context has been copied. Otherwise,
test f_expisize_ea_del would fail when aborting.

E2fsprogs-commit: 05a637a9e68d3f7e15323deeab00981d4b7df7e8

Change-Id: I4994e62bdf27c385b02e55de82a9dafcc6a12139
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoe2fsck: add -m option for multithread
Li Xi [Sun, 4 Aug 2019 15:28:59 +0000 (23:28 +0800)]
e2fsck: add -m option for multithread

-m option is added but no actual functionality is added. This
patch only adds the logic that when -m is specified, one of
-p/-y/-n options should be specified. And when -m is specified,
-C shouldn't be specified and the completion progress report won't
be triggered by sending SIGUSR1/SIGUSR2 signals. This simplifies
the implementation of multi-thread fsck in the future.

Completion progress support with multi-thread fsck will be added
back after multi-thread fsck implementation is finished. Right
now, disable it to simplify the implementation of multi-thread fsck.

E2fsprogs-commit: 842ae5f528f9797e4b2c739daa76884e4349e8f3

Change-Id: I428f4f3b10974b769100bf00169bdd7d8cf86deb
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoLU-17310 tests: make m_assume_storage_prezeroed more robust
Andreas Dilger [Thu, 23 Nov 2023 02:47:38 +0000 (19:47 -0700)]
LU-17310 tests: make m_assume_storage_prezeroed more robust

Don't assume that mke2fs is going to zero out an exact number
of blocks when run with/without "-E assume_storage_prezeroed",
since this depends on a number of different options that are
not specified in the test script.

Instead, check that the number of blocks zeroed in the image is
a small fraction (1/15th) of the number of blocks zeroed when
"-E assume_storage_prezeroed" is not given, which makes it more
robust when running in different environments.  This varies from
1/16 in the original test to 1/91 in my local test environment.

Avoid "losetup --sector-size 4096", use "mke2fs -b 4096" instead.
Clean up the loop device before checking "stat" so that all
blocks are flushed to the backing storage before calling sync.
Only one loop device and test file is needed for the test.

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Change-Id: I98fc3f5334cb8e2dd754c5d4994f2ca582300c13
Reviewed-on: https://review.whamcloud.com/c/tools/e2fsprogs/+/53217
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Li Dongyang <dongyangli@ddn.com>
12 months agoLU-14432 configure.ac: fix AC_PREREQ
Li Dongyang [Sun, 14 Feb 2021 23:30:11 +0000 (10:30 +1100)]
LU-14432 configure.ac: fix AC_PREREQ

AS_HELP_STRING was introduced in autoconf 2.58
requiring autoconf 2.69 is not necessary and it breaks
several builds in our Jenkins.

Change required autoconf to 2.59 just to be safe.

Change-Id: I30af873879866858d32b715ac4ce78d095e10f7d
Fixes: 8c9e2921 ("configure.ac: convert all help strings to use AS_HELP_STRING")
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Reviewed-on: https://review.whamcloud.com/41665
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Wang Shilong <wshilong@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
12 months agoext4.5: add preprocessor hint
Theodore Ts'o [Tue, 21 May 2024 02:52:47 +0000 (22:52 -0400)]
ext4.5: add preprocessor hint

This fixes a Lintian warning which is triggered by an arbtrary
MANROFFSEQ='' environment variable:

an.tmac:<standard input>:376: warning: tbl preprocessor failed, or it or soelim was not run; table(s) likely not rendered (TE macro called with TW register undefined)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoUpdate release notes, etc., for the 1.47.1 release
Theodore Ts'o [Mon, 20 May 2024 19:31:17 +0000 (15:31 -0400)]
Update release notes, etc., for the 1.47.1 release

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoUpdate makefile dependencies
Theodore Ts'o [Mon, 20 May 2024 19:46:10 +0000 (15:46 -0400)]
Update makefile dependencies

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agoAOSP: Use no_full_install: true instead of installable: false
Jiyong Park [Wed, 1 May 2024 07:17:36 +0000 (16:17 +0900)]
AOSP: Use no_full_install: true instead of installable: false

So far, we have used `instalable: false` to avoid collision with the
other modules that are installed to the same path. A typical example was
<foo> and <foo>.microdroid. The latter is a modified version of the
former for the inclusion of the microdroid image. They however both have
the same instalation path (ex: system/bin) and stem (ex: foo) so that we
can reference them using the same path regardless of whether we are in
Android or microdroid.

However, the use of `installable: false` for the purpose is actually
incorrect, because `installable: false` also means, obviously, "this
module shouldn't be installed". The only reason this incorrect way has
worked is simply because packaging modules (ex: android_filesystem)
didn't respect the property when gathering the modules.

As packaging modules are now fixed to respect `installable: false`, we
need a correct way of avoiding the collision. `no_full_install: true` is
it.

If a module has this property set to true, it is never installed to the
full instal path like out/target/product/<partition>/... It can be
installed only via packaging modules.

Bug: 338160898
Test: m
Change-Id: Idb173a7e3528c96b23f857bb3bdf5f37e698c445
From AOSP commit: 21a895548df7de83ce1e2e146e1718e5f723af7f

12 months agoAOSP: e2fsdroid: disable asan leak detection
Steven Moreland [Mon, 15 Apr 2024 23:07:31 +0000 (23:07 +0000)]
AOSP: e2fsdroid: disable asan leak detection

borked and breaks asan build

Bugs: me
Test: build with SANITIZE_HOST=address
Change-Id: I9ae15ba328081c38e31c61834e80ce10765f9e30
From AOSP commit: eff2c071b546c3d2d3ea5eb89328babcc48e4bc2

12 months agoAOSP: Make mke2fs/e2fsdroid available for vendor
Kelvin Zhang [Tue, 26 Mar 2024 17:35:02 +0000 (10:35 -0700)]
AOSP: Make mke2fs/e2fsdroid available for vendor

vendor_init needs to execute these binaries when converting partitions
to EXT4.

Test: th
Bug: 293313353
Change-Id: I1fa49c1a0f802b3c36e96112ef262bae4d5d394a
From AOSP commit: 0b54b8227815d447b52de76bb419735b21608941

12 months agoAOSP: Make badblocks host-only
Cole Faust [Fri, 13 Oct 2023 19:53:10 +0000 (12:53 -0700)]
AOSP: Make badblocks host-only

It was being installed on some products when it shouldn't be.

Bug: 205632228
Test: m installclean && m with aosp/2773149
Change-Id: I7f4642ba6fa8d97f7711b6df57c4e3fd781b40fd
From AOSP commit: ecb8d2faa7411d9de228a3bd8b883ed2d5220188

12 months agoAOSP: Compile libext2_blkid and libext2_uuid on Mac OS X for the Cuttlefish launcher
A. Cody Schuffelen [Thu, 22 Jun 2023 01:19:08 +0000 (18:19 -0700)]
AOSP: Compile libext2_blkid and libext2_uuid on Mac OS X for the Cuttlefish launcher

Test: m libext2_blkid libext2_uuid
Bug: 288342686
Change-Id: Ieaf2b73efe4b9e1ed0b52e12cf931d225cd8844d
From AOSP commit: 24d1f08c6c42a292cb49c0526d2a39e889c50683

12 months agoAOSP: ext2simg: fix same_file() with symlinks
Eric Biggers [Thu, 23 Mar 2023 02:52:09 +0000 (02:52 +0000)]
AOSP: ext2simg: fix same_file() with symlinks

Fix same_file() to use stat() instead of lstat() when checking the
paths, so that symlinks are dereferenced.  This is needed to be
consistent with how the paths are actually accessed later.  Otherwise,
not all cases where the input and output file are the same are detected.

Also just use the stat() result to check whether the output file exists,
instead of using a separate call to access().

Fixes: db6f320912cf ("AOSP: android: add the ext2simg tool")
Change-Id: Ie36981f9dbc19494732f518488a75fb92c0f0343
Signed-off-by: Eric Biggers <ebiggers@google.com>
From AOSP commit: 08c122f12fc231029a74c24b969e337203c7b6e2

12 months agoAOSP: ext2simg: fix error check of io_channel_read_blk64()
Eric Biggers [Thu, 23 Mar 2023 02:49:43 +0000 (02:49 +0000)]
AOSP: ext2simg: fix error check of io_channel_read_blk64()

Check the return value of io_channel_read_blk64() correctly, considering
that it returns an errcode_t, which can be positive.

Fixes: db6f320912cf ("AOSP: android: add the ext2simg tool")
Change-Id: Iafc6c0169bc8ac79198f285da0246ff3b841ded8
Signed-off-by: Eric Biggers <ebiggers@google.com>
From AOSP commit: 60634ff32a0d8c0d7942c6e522a74a5051d4b6e9

12 months agoAOSP: ext2simg: clean up integer types and check for too-large fs
Eric Biggers [Thu, 23 Mar 2023 00:44:22 +0000 (00:44 +0000)]
AOSP: ext2simg: clean up integer types and check for too-large fs

libsparse assumes 32-bit block numbers.  Also, ext2simg might read
nearly the entire filesystem into memory.

Therefore, make ext2simg use appropriate integer types, and explicitly
check for when the filesystem is too large or allocating memory failed.

Change-Id: Ic415d0e974dce2b4ff6e7fa9265f6e86d371a274
Signed-off-by: Eric Biggers <ebiggers@google.com>
From AOSP commit: 8fff11068c100be627745967992fb88759dea9c1

12 months agoAOSP: ext2simg: clean up add_chunk()
Eric Biggers [Thu, 23 Mar 2023 00:44:22 +0000 (00:44 +0000)]
AOSP: ext2simg: clean up add_chunk()

Remove a level of indentation, check a bool in the normal way, and
simplify the linked list handling.  No change in behavior.

Change-Id: I12589a254f155b1c40418458a666b87c7ef5c1cf
Signed-off-by: Eric Biggers <ebiggers@google.com>
From AOSP commit: 7d0f5c1aca332da22e4878f5825e0ffb5122f96b

12 months agoAOSP: ext2simg: use a standard flexible array
Eric Biggers [Thu, 23 Mar 2023 00:44:21 +0000 (00:44 +0000)]
AOSP: ext2simg: use a standard flexible array

Use a standard flexible array instead of a nonstandard zero-length
array.  No change in behavior.

Change-Id: Ifdce24f5d6e2471634bb785527def3fe8fefc202
Signed-off-by: Eric Biggers <ebiggers@google.com>
From AOSP commit: c88ea796fbf7f4c79155196ec483681b3733bbff

12 months agoAOSP: ext2simg: use bool where appropriate
Eric Biggers [Thu, 23 Mar 2023 00:44:21 +0000 (00:44 +0000)]
AOSP: ext2simg: use bool where appropriate

For the values that get used as the 'bool' parameters of
sparse_file_write(), use 'bool' in ext2simg too.  No change in behavior.

Change-Id: I05f7d6fd3027eb10231c035f9fdc8e946e2c4c90
Signed-off-by: Eric Biggers <ebiggers@google.com>
From AOSP commit: 2728c6e766976acbf442d3721f2d93960e13682e

12 months agoAOSP: ext2simg: fix same_file() to check st_dev
Eric Biggers [Thu, 23 Mar 2023 00:44:21 +0000 (00:44 +0000)]
AOSP: ext2simg: fix same_file() to check st_dev

File identity is determined by the combination of st_dev and st_ino, not
by st_ino alone.

This fixes a bug where ext2simg would needlessly make a copy of all the
data when the input and output files happened to have the same st_ino.

Fixes: db6f320912cf ("AOSP: android: add the ext2simg tool")
Change-Id: I94e4bf57d9f91b31e5438768805e9f10bec3411d
Signed-off-by: Eric Biggers <ebiggers@google.com>
From AOSP commit: 0749f83a2cf4c134a2403701ab78388500e53f76

12 months agoAOSP: ext2simg: fix off-by-one errors causing corruption
Eric Biggers [Thu, 23 Mar 2023 00:44:21 +0000 (00:44 +0000)]
AOSP: ext2simg: fix off-by-one errors causing corruption

The chunk_end parameter to add_chunk() is exclusive, but two callers
incorrectly treat it as inclusive: when the maximum chunk length of
'INT32_MAX - 12' bytes is reached, and when a chunk extends to the very
end of the filesystem.  The result is that the output simg file contains
zeroes for the last block of these chunks instead of the correct data.

A related bug is that the expanded size of the simg file is set to the
filesystem size (in blocks) minus s_first_data_block.  On filesystems
where s_first_data_block != 0, i.e. 1K blocksize filesystems without
bigalloc enabled, this truncates the last block of the filesystem.

Fix these bugs by (a) making add_chunk() take the chunk length and
passing the correct values, and (b) using the filesystem size properly.

Here is a reproducer that shows the last block of the filesystem being
truncated (bsize=1024) and being corrupted with zeroes (bsize=4096):

    for bsize in 1024 4096; do
        rm -f ext4.img
        mkfs.ext4 -b $bsize ext4.img 10000
        mkdir -p mnt
        sudo mount -t ext4 -o loop ext4.img mnt
        sudo cp /dev/urandom mnt/fill
        sudo umount mnt
        ext2simg ext4.img ext4.simg
        simg2img ext4.simg ext4.img.restored
        cmp ext4.img ext4.img.restored
    done

Fixes: db6f320912cf ("AOSP: android: add the ext2simg tool")
Reported-by: Clemens Lang <clemens.lang@bmw.de>
Change-Id: I3b64c4fbffa5821b431f29e99b36168617da7563
Signed-off-by: Eric Biggers <ebiggers@google.com>
From AOSP commit: 1e498908c6ac13b4d5ec0117f4ddcd577aac607e

12 months agoAOSP: Mostly restore -Werror for macOS build
Eric Biggers [Wed, 8 Feb 2023 21:42:02 +0000 (21:42 +0000)]
AOSP: Mostly restore -Werror for macOS build

It turns out the "Can't use getmntent or getmntinfo" warning was the
only warning remaining in the macOS build via the Android build system.
So now that it's fixed, -Wno-error can be removed.

That being said, the upstream CI (GitHub Actions) currently uses
-Wno-error=deprecated-declarations for the macOS build, since it's still
needed for some files (which aren't built by the Android build system).

For now, let's just replace -Wno-error with
-Wno-error=deprecated-declarations to match what the upstream CI uses.

Change-Id: I77f6649b99432ef1d73a0c7e30bbb150c3111b27
From AOSP commit: 6ea38ded59fe970704612a31a3aea4ccaf923d6a

12 months agoAOSP: Android: define HAVE_GETMNTINFO on macOS
Eric Biggers [Thu, 2 Feb 2023 22:42:19 +0000 (22:42 +0000)]
AOSP: Android: define HAVE_GETMNTINFO on macOS

macOS supports getmntinfo(), but not getmntent().  To match what the
'configure' script detects, define HAVE_GETMNTINFO to 1.

This prevents the following warning:

    #warning "Can't use getmntent or getmntinfo to check for mounted filesystems!"

Bug: 267448785
Change-Id: I3131563fc317fa9fef7745937ec2c4b09a1d29b0
From AOSP commit: bb6d46cc9770f4f15a5e52122a16f762c1bb567a

12 months agoAOSP: Revert "Android: stop suppressing warnings from macOS build"
Eric Biggers [Thu, 2 Feb 2023 17:56:20 +0000 (17:56 +0000)]
AOSP: Revert "Android: stop suppressing warnings from macOS build"

Unfortunately, the macOS build is not tested either by presubmit or by
local builds.  A macOS build *is* being tested in the upstream GitHub
Actions workflow now; however, that uses the autotools-based build
system, and there can be issues specific to the Android build system.

As a result, removing -Wno-error was not safe yet, and the macOS build
is currently broken in postsubmit.  As there could be multiple issues,
let's restore -Wno-error until I've had a chance to fix the warnings.

Bug: 267448785
Change-Id: I305f73d1f8637477da3d57b6c93037a6e3d9e829
From AOSP commit: 0ed82a3f0a393605b56672704379f4fc1e53d281

12 months agoUse ext2/ext3/ext4 instead of "second extended file system" in man pages
Theodore Ts'o [Sat, 18 May 2024 05:10:01 +0000 (01:10 -0400)]
Use ext2/ext3/ext4 instead of "second extended file system" in man pages

Addresses-Debian-Bug: #1041115
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agolibext2fs: avoid using a C++ reserved identifier in rbtree.h
Theodore Ts'o [Sat, 18 May 2024 05:00:14 +0000 (01:00 -0400)]
libext2fs: avoid using a C++ reserved identifier in rbtree.h

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
12 months agolibext2fs: add ext2_types.h to qcow2.h
Theodore Ts'o [Sat, 18 May 2024 04:56:58 +0000 (00:56 -0400)]
libext2fs: add ext2_types.h to qcow2.h

The qcow2.h header file uses types such __u32 which are defined in
ext2_types.h.  So include it directly to avoid relying on users of the
qcow2.h header file to include right dependencies.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agoe2fsck: fix golden output of several tests
Jan Kara [Mon, 6 May 2024 17:41:19 +0000 (19:41 +0200)]
e2fsck: fix golden output of several tests

Some old tests of EA inodes were not in fact completely fixing the
filesystem (like they were leaving directories with EA_INODE_FL set or
EA inodes referenced from directory hierarchy). New e2fsck checks fix
these so golden output changes. Update it.

Signed-off-by: Jan Kara <jack@suse.cz>
Link: https://lore.kernel.org/r/20240506174132.12883-3-jack@suse.cz
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agoe2fsck: add tests for EA inodes
Jan Kara [Mon, 6 May 2024 17:41:18 +0000 (19:41 +0200)]
e2fsck: add tests for EA inodes

Add tests exercising EA inodes and testing various types of corruption.

Signed-off-by: Jan Kara <jack@suse.cz>
Link: https://lore.kernel.org/r/20240506174132.12883-2-jack@suse.cz
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agoe2fsck: add more checks for ea inode consistency
Jan Kara [Mon, 6 May 2024 17:41:17 +0000 (19:41 +0200)]
e2fsck: add more checks for ea inode consistency

Currently checking of EA inodes was rather weak. Add several more
consistency checks.

1) Check that EA inode is a regular file.
2) Check that EA_INODE feature is set if the filesystem has EA inodes.
3) Make sure that no EA inode is referenced from directory hierarchy.

Signed-off-by: Jan Kara <jack@suse.cz>
Link: https://lore.kernel.org/r/20240506174132.12883-1-jack@suse.cz
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agoe2fsck: fix acl block leak when process orphan list
Ye Bin [Thu, 18 Apr 2024 06:39:46 +0000 (14:39 +0800)]
e2fsck: fix acl block leak when process orphan list

There's a issue:
[]$~/e2fsprogs/e2fsck/e2fsck -f scsi-disk2.img
e2fsck 1.47.0 (5-Feb-2023)
scsi-disk2.img: recovering journal
Clearing orphaned inode 12 (uid=0, gid=0, mode=0140777, size=0)
Pass 1: Checking inodes, blocks, and sizes
Extended attribute block 4247 has reference count 3, should be 2.  Fix<y>? no
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Free blocks count wrong (249189, counted=249188).
Fix<y>? no
Free inodes count wrong (65526, counted=65523).
Fix<y>? no

scsi-disk2.img: ***** FILE SYSTEM WAS MODIFIED *****

scsi-disk2.img: ********** WARNING: Filesystem still has errors **********

scsi-disk2.img: 10/65536 files (0.0% non-contiguous), 12955/262144 blocks

Above issue can reproduce as follows:
step1: socat UNIX-LISTEN:/home/test/mysocket.sock,mode=777,reuseaddr,fork EXEC:/home/test  &
step2: setfacl some xattr for mysocket.sock
step3: cp -a /home/test/mysocket.sock /home/test/sock1
       cp -a /home/test/mysocket.sock /home/test/sock2
step4: sync
step5: Power-off
step6: run e2fsck

As after commit 42475e281d22 add ext2fs_inode_has_valid_blocks() judgement in
release_inode_blocks() which means socket type file skip realse block include
ACL block. The kernel does not restrict the setting of extended attributes for
socket files. So this will lead to ACL block leak.
To solve above issue there's need to release ACL block for other kind of
special file.

Fixes: 42475e281d22 ("super.c (release_inode_blocks): Don't try to release the blocks if the orphaned inode is a device file, symlink, or some other kind of special file that doesn't have a block list.")
Signed-off-by: Ye Bin <yebin10@huawei.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Link: https://lore.kernel.org/r/20240418063946.2802835-1-yebin10@huawei.com
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agoconfigure: remove duplicated/unnecessary test for compiler fuzzing support
Theodore Ts'o [Sun, 5 May 2024 03:56:12 +0000 (23:56 -0400)]
configure: remove duplicated/unnecessary test for compiler fuzzing support

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agolibext2fs: fix potential divide by zero bug caused by a lxcfs bug
Theodore Ts'o [Thu, 2 May 2024 16:23:44 +0000 (12:23 -0400)]
libext2fs: fix potential divide by zero bug caused by a lxcfs bug

If sysconf(_SC_NPROCESSORS_CONF) returns zero, this can cause a divide
by zero.  Make ext2fs_rw_bitmaps() more robust defaulting to 4 threads
if _SC_NPROCESSORS_CONF returns an invalid value.

https://github.com/tytso/e2fsprogs/issues/114
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agoe2image: fix typo which causes a compile failure on i386
Theodore Ts'o [Thu, 2 May 2024 03:02:31 +0000 (23:02 -0400)]
e2image: fix typo which causes a compile failure on i386

Fixes: 80abfebc673b ("e2image: add support for post-2038 dates...")
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agodebian: build-depend on pkgconf instead of pkg-config
Theodore Ts'o [Thu, 2 May 2024 02:10:34 +0000 (22:10 -0400)]
debian: build-depend on pkgconf instead of pkg-config

The pkg-config package has been obsoleted by pkgconf.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agolibsupport: silence gcc -Wall complaints
Theodore Ts'o [Wed, 1 May 2024 21:27:24 +0000 (17:27 -0400)]
libsupport: silence gcc -Wall complaints

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agoe4defrag: use snprintf to assure that there can't be a buffer overflow
Theodore Ts'o [Wed, 1 May 2024 21:22:55 +0000 (17:22 -0400)]
e4defrag: use snprintf to assure that there can't be a buffer overflow

The size of msg_buffer is carefully calculated so it can never
overflow, but it triggers a Coverity warning.  Use snprintf instead of
sprintf to silence the Coverity warning.

Addresses-Coverty-Bug: 1520603
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agolibsupport: use explicit type widths instead of time_t
Theodore Ts'o [Wed, 1 May 2024 20:58:50 +0000 (16:58 -0400)]
libsupport: use explicit type widths instead of time_t

The in-memory data structures used time_t for the grace period (which
is a delta timestamp denominated in seconds), as well as the soft
limit expiration time (which is an actual time_t).  Use an explicit
__u32 for the former, and the __u64 for the latter.

This silences a Coverity warning, but more importantly, using an
explicit __u64 for the expiration time means that running e2fsck on a
platform with a 32-bit time_t, and it needs to read and then modify a
quota structure, we won't lose the high 32-bits of the quota
expiration time.

Addresses-Coverity-Bug: 1531824
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agoe2image: add support for post-2038 dates in the e2image header
Theodore Ts'o [Wed, 1 May 2024 20:54:26 +0000 (16:54 -0400)]
e2image: add support for post-2038 dates in the e2image header

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agoUpdate release notes, etc., for the 1.47.1-rc2 release v1.47.1-rc2
Theodore Ts'o [Wed, 1 May 2024 04:55:57 +0000 (00:55 -0400)]
Update release notes, etc., for the 1.47.1-rc2 release

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agopo: update zh_CN.po (from translationproject.org)
Wenbin Lv [Wed, 1 May 2024 04:34:15 +0000 (00:34 -0400)]
po: update zh_CN.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agopo: update uk.po (from translationproject.org)
Yuri Chornoivan [Wed, 1 May 2024 04:34:15 +0000 (00:34 -0400)]
po: update uk.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agopo: update sv.po (from translationproject.org)
Göran Uddeborg [Wed, 1 May 2024 04:34:15 +0000 (00:34 -0400)]
po: update sv.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agopo: update ro.po (from translationproject.org)
Remus-Gabriel Chelu [Wed, 1 May 2024 04:34:15 +0000 (00:34 -0400)]
po: update ro.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agopo: update pl.po (from translationproject.org)
Jakub Bogusz [Wed, 1 May 2024 04:34:14 +0000 (00:34 -0400)]
po: update pl.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agopo: update fr.po (from translationproject.org)
Samuel Thibault [Wed, 1 May 2024 04:34:14 +0000 (00:34 -0400)]
po: update fr.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agopo: update cs.po (from translationproject.org)
Petr Pisar [Wed, 1 May 2024 04:34:14 +0000 (00:34 -0400)]
po: update cs.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agoe2fsck: check the error return from the forced rewrite write
Theodore Ts'o [Wed, 1 May 2024 04:24:52 +0000 (00:24 -0400)]
e2fsck: check the error return from the forced rewrite write

If read of a block fails, we offer the user the opportunity to force a
rewrite to that sector to force the storage device to remap the LBA to
its spare block pool.  Check that write so if it fails, we can let the
user know.

Addresses-Coverity-bug: 1432422
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agoresize2fs: mark that the error return is deliberately ignored
Theodore Ts'o [Wed, 1 May 2024 04:20:10 +0000 (00:20 -0400)]
resize2fs: mark that the error return is deliberately ignored

When moving the inode table, if writing the (partially overlapping)
inode table fails, we need to write it back in its original location
before bailing out.  If that write unding the initial write fails,
there's nothing we can do, so we ignore it.  Mark this to avoid a
false positive from Coverity.

Fixes-Coverity-bug: 1432422
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agoe2scrub: test for the presence of systemd using test -e /run/systemd/system
Theodore Ts'o [Wed, 1 May 2024 03:54:26 +0000 (23:54 -0400)]
e2scrub: test for the presence of systemd using test -e /run/systemd/system

Debian has a package called "systemctl" which provides a systemctl
executable to "manage services without systemd".  So test for whether
we have a fully functional systemd system by checking for the
existence of /run/systemd/system instead testing for the presence of
the command named systemctl.

Addresses-Debian-Bug: #1070107
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agoRemove explicit #define of _FILE_OFFSET_BITS
Theodore Ts'o [Tue, 30 Apr 2024 23:54:08 +0000 (19:54 -0400)]
Remove explicit #define of _FILE_OFFSET_BITS

The problem with explicitly setting _FILE_OFFSET_BITS is that
it's not necessarily a no-op on a 64-bit platform with a 64-bit off_t.
Apparently glibc's mips64el which end up using a different structure
definition for struct stat, and this causes a compatibility problem
with libarchive.  It's not needed on mips64el, since off_t is 64-bits,
but it actually causes problems.

So remove it, since we now use the autoconf's AC_SYS_LARGEFILE, which
will set _FILE_OFFSET_BITS when it is necessary (such as on a 32-bit
i386 Linux platform), and will skip it when it is unnecessary.

Addresses-Debian-Bug: #1070042
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agodebian: add support for DEB_BUILD_OPTIONS=parallel=N master next
Theodore Ts'o [Tue, 30 Apr 2024 01:08:19 +0000 (21:08 -0400)]
debian: add support for DEB_BUILD_OPTIONS=parallel=N

This speeds up package builds using "make -jN" and "make -jN check".

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agodebian: don't build with libarchive on mips64el
Theodore Ts'o [Mon, 29 Apr 2024 20:31:14 +0000 (16:31 -0400)]
debian: don't build with libarchive on mips64el

The libarchive functionality in "mke2fs -d foo.tar" is breaking the
regression test[1].  Since this is working everywhere _except_
mips64el, as a short-term workaround disable libarchive support on
this platform until it can be fixed.

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1070042

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agodebian: don't try to install e2scrub on Hurd
Theodore Ts'o [Mon, 29 Apr 2024 18:13:47 +0000 (14:13 -0400)]
debian: don't try to install e2scrub on Hurd

The e2scrub scripts rely on systemd, which isn't present on non-Linux
systems, so they aren't built.  So we need to skip trying to run
dh_installsystemd since it will fail on the Hurd build since the
requisite files aren't being built.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agoconfigure: add --without-libarchive option
Theodore Ts'o [Mon, 29 Apr 2024 05:23:29 +0000 (01:23 -0400)]
configure: add --without-libarchive option

Teach configure the --without-libarchive option, which forcibly
disables use of the libarchive library.

The option --with-libarchive=direct will disable the use of dlopen,
and will link mke2fs with -larchive directly.  This doesn't work when
building mke2f.static, since -larchive has a large number of
depedencies, and even "pkgconf --libs --static libarchive" doesn't
provide all of the appropriate library dependencies.  :-(

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agoresize2fs: fix last bg's free clusters calculation on 64-bit file systems
Theodore Ts'o [Mon, 29 Apr 2024 03:26:19 +0000 (23:26 -0400)]
resize2fs: fix last bg's free clusters calculation on 64-bit file systems

Fixes-Coverity-bug: 1596645
Fixes: d43fb24ca0db ("resize2fs: fix r_bigalloc_big_expand test failure")
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agoRemove duplicated word in release notes
Theodore Ts'o [Sun, 28 Apr 2024 16:34:42 +0000 (12:34 -0400)]
Remove duplicated word in release notes

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
13 months agodebian: add a note in debian/changelog regarding features being re-enabled v1.47.1-rc1
Theodore Ts'o [Sun, 28 Apr 2024 03:06:09 +0000 (23:06 -0400)]
debian: add a note in debian/changelog regarding features being re-enabled

The metadata_csum_seed and orphan_file features were disabled before
Debian Bookworm was released, but now that it's released, we are now
re-enabling those features for Debian testing and the next version of
Debian stable (trixie).

Also, remove some spurious whitespace.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>