Whamcloud - gitweb
Theodore Ts'o [Thu, 11 Oct 2012 11:38:01 +0000 (07:38 -0400)]
Merge branch 'maint' into next
Conflicts:
lib/ext2fs/blkmap64_rb.c
Theodore Ts'o [Wed, 10 Oct 2012 02:45:40 +0000 (22:45 -0400)]
e2fsck: only consult inode_dir_map if needed in pass4
In e2fsck_pass4(), we were consulting inode_dir_map using
ext2fs_test_inode_bitmap2() for every single inode in the file system.
However, there were many cases where we never needed the result of the
test --- most notably if the inode is not in use.
I was a bit surprised that GCC 4.7 with CFLAGS set to "-g -O2" wasn't
able to optimize this out for us, but here is the pass 4 timing for an
empty 3T file system before this patch:
Pass 4: Memory used: 672k/772k (422k/251k), time: 3.67/ 3.66/ 0.00
and afterwards, we see a 43% improvement:
Pass 4: Memory used: 672k/772k (422k/251k), time: 2.09/ 2.08/ 0.00
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Sat, 6 Oct 2012 02:01:31 +0000 (22:01 -0400)]
Fix makefiles to compile e2freefrag with profiling
Also fix a bug caused by a stray continuation backslash which caused
the e2fsck/Makefile to fail when profiling is enabled.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Theodore Ts'o [Sat, 6 Oct 2012 01:59:40 +0000 (21:59 -0400)]
libext2fs: further optimize rb_test_bit
Profiling shows that rb_test_bit() is now calling ext2fs_rb_next() a
lot, and this function is now the hot spot when running e2freefrag.
If we cache the results of ext2fs_rb_next(), we can eliminate those
extra calls, which further speeds up both e2freefrag and e2fsck by
reducing the amount of CPU time spent in userspace.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Sat, 6 Oct 2012 00:57:49 +0000 (20:57 -0400)]
libext2fs: remove pointless indirection in rbtree bitmaps
The code was previously allocating a single 4 or 8 byte pointer for
the rcursor and wcursor fields in the ext2fs_rb_private structure;
this added two extra memory allocations (which could fail), and extra
indirections, for no good reason. Removing the extra indirection also
makes the code more readable, so it's all upside and no downside.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Theodore Ts'o [Fri, 5 Oct 2012 03:38:55 +0000 (23:38 -0400)]
libext2fs: optimize rb_test_bit
Optimize testing for a bit in an rbtree-based bitmap for the case
where the calling application is scanning through the bitmap
sequentially. Previously, we did this for a set of bits which were
inside an allocated extent, but we did not optimize the case where
there was a large number of bits after an allocated extents which were
not in use.
1111111111111110000000000000000000
^ optimized ^not optimized
In my tests of a roughly half-filled file system, the run time of
e2freefrag was halved, and the cpu time spent in userspace was during
e2fsck's pass 5 was reduced by a factor of 30%.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Lukas Czerner <lczerner@redhat.com>
Theodore Ts'o [Fri, 5 Oct 2012 03:30:23 +0000 (23:30 -0400)]
e2freefrag: use 64-bit rbtree bitmaps
Enable the use of 64-bit bitmaps, so e2freefrag will work on file
systems with the 64-bit feature enabled. In addition, enable the
rbtree-based bitmaps, which significantly saves the amount of memory
required (from 97 megs to 1.7 megs for an empty 3T file system) at the
cost of additional CPU overhead (but we will claw back some of the
additional CPU overhead in the next commit).
Addresses-Google-Bug: 7269948
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Sun, 23 Sep 2012 02:29:34 +0000 (22:29 -0400)]
Update for e2fsprogs-1.43-WIP-2012-09-22
Theodore Ts'o [Sun, 23 Sep 2012 01:35:46 +0000 (21:35 -0400)]
Merge branch 'maint' into next
Theodore Ts'o [Sun, 23 Sep 2012 01:26:48 +0000 (21:26 -0400)]
mke2fs: fix progress suppression to make regression tests reliable
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Sat, 22 Sep 2012 21:07:46 +0000 (17:07 -0400)]
mke2fs: prohibit file system features not supported by the HURD
There are certain file system features which can not be supported by
the HURD, since they use fields in the inode which have been claimed
by HURD-specific features (i.e., such as the author field). We will
mask out those features so they are not enabled by accident, but if the
user tries to explicitly specify them we will issue an error message.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Sat, 22 Sep 2012 13:02:52 +0000 (09:02 -0400)]
libext2fs: always define i_checksum_lo
Always #define i_checksum_lo, since otherwise e2fsprogs won't compile
on non-Linux platforms.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Fri, 21 Sep 2012 16:55:45 +0000 (12:55 -0400)]
Merge branch 'maint' into next
Conflicts:
debian/changelog
version.h
Theodore Ts'o [Fri, 21 Sep 2012 16:53:14 +0000 (12:53 -0400)]
Update Release Notes, Changelogs, version.h, etc. for 1.42.6 release
Theodore Ts'o [Fri, 21 Sep 2012 16:31:13 +0000 (12:31 -0400)]
Fix wordwrap.pl so it is more portable
Needed so that wordwrap.pl works with perl 5.14.2
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Fri, 21 Sep 2012 16:06:49 +0000 (12:06 -0400)]
mke2fs: throttle allocating groups progress as well
Throttle updates for the "Allocating Groups" progress updates to once
a second as well. We now do this throttling in libext2fs, so we don't
have to do this for each of mke2fs's progress updates, and because the
updates from ext2fs_allocate_tables() come from within libext2fs
anyway.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Fri, 21 Sep 2012 01:59:13 +0000 (21:59 -0400)]
mke2fs.8: use extents vs extent in the man page
We print "extents" for the feature, so we should document "extents" in
the mke2fs's man page, even though we accept both "extent" and
"extents".
Addresses-Sourceforge-Bug: #3559210
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Fri, 21 Sep 2012 00:48:34 +0000 (20:48 -0400)]
debian: adjust build dependencies
Remove dc from the list of dependencies, since it's not used during
the build anymore.
Addresses-Debian-Bug: #677497
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Mon, 17 Sep 2012 02:06:29 +0000 (22:06 -0400)]
Merge branch 'maint' into next
Conflicts:
resize/resize2fs.c
Andreas Dilger [Mon, 10 Sep 2012 09:04:47 +0000 (09:04 +0000)]
tests: kill debugfs on interrupted MMP test
If the f_mmp test is interrupted during its test run, then it can
leave debugfs busy-looping in the background. Since f_mmp is a
relatively long-running test, and is likely to be running during
a parallel test run, this can happen fairly often.
Set a signal trap for the f_mmp test script being killed, so that
the background debugfs command will always be killed by the test.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 17 Sep 2012 00:43:07 +0000 (20:43 -0400)]
mke2fs: disable progress reporting in mke2fs.conf for regression tests
Add a configuration knob so the regression tests can disable progress
reporting. This fixes a potential lack of predictability since the
progress reports are now time based (once a second) which is
problematic for regression tests which are comparing the expected
output of mke2fs.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Fri, 14 Sep 2012 04:11:07 +0000 (00:11 -0400)]
mke2fs: throttle progress updates to once a second
With lazy itable initialization, the progress updates for writing the
inode table happens so quickly that on a serial console, the time to
write the progress updates can be the bottleneck. Fix this by only
updating the progress indicator once a second.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Thu, 13 Sep 2012 22:17:27 +0000 (18:17 -0400)]
resize2fs: grow uninit_bg file systems more efficiently
If the uninit_bg feature is enabled and the kernel supports
lazy_itable_init, skip zeroing the inode table so that the resize
operation can go much more quickly. Also set the itable_unused fields
so that the first e2fsck after the resize will run faster.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Thu, 13 Sep 2012 20:56:36 +0000 (16:56 -0400)]
resize2fs: enforce restrictions if the kernel doesn't do meta_bg resizing
Enhance the online resizing code to be more nuanced about resizing
restrictions. If the kernel supports meta_bg resizing, then we can
skip all of the restrictions. If the kernel does not support meta_bg
resizing, check more carefully to make sure there are enough reserved
gdt blocks, so that the user gets a clearer error message.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Fri, 31 Aug 2012 19:31:50 +0000 (15:31 -0400)]
resize2fs: allow meta_bg/64-bit file systems to be online resized
Resize2fs can't handle resizing flex_bg file systems that do not have
the resize inode, but when the kernel adds support for resizing using
the meta_bg layout, we should allow it be able to resize the file
system.
So move the flex_bg/resize_inode check to the just before we start
doing the off-line resize, instead of doing it earlier where it would
prohibit these file systems for both on-line and off-line resizes.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Mon, 3 Sep 2012 16:40:54 +0000 (12:40 -0400)]
resize2fs: fix overhead calculation for meta_bg file systems
The file system overhead calculation in calculate_minimum_resize_size
was incorrect meta_bg file systems. This caused the minimum size to
underflow for very large file systems, which threw resize2fs into a
loop generally lasted longer than the user's patience.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Fri, 7 Sep 2012 04:05:21 +0000 (00:05 -0400)]
resize2fs: enforce the 16TB limit on 32-bit file systems correctly
The 16TB limit must be enforced regardless of whether the new size is
specified on the command line or implied by the size of the device,
but only if the file system does not support 64-bit block sizes, or
the kernel does not advertise support of meta_bg resizing.
Previously we were unconditionally enforcing it when it was implied by
the device size, but not if the new size was specified on the command
line.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Mon, 10 Sep 2012 02:52:26 +0000 (22:52 -0400)]
Merge branch 'maint' into next
Conflicts:
lib/ext2fs/ext2fs.h
Theodore Ts'o [Mon, 10 Sep 2012 01:43:08 +0000 (21:43 -0400)]
ext2fs.h: move ext2fs_init_csum_seed() outside of EXT2_CUSTOM_MEMORY_ROUTINES
The function ext2fs_init_csum_seed() has nothing to do with the
ext2fs_get_mem()/ext2fs_get_memzero()/ext2fs_get_array()/ext2fs_get_arrayzero()
functions. (This define is there so that on platforms where we need
to use the standard C functions, they can be replaced --- this is
primarily needed when trying to compile libext2fs for strange,
non-quite-standards-compliant platforms, such as Windows.)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Mon, 10 Sep 2012 01:35:39 +0000 (21:35 -0400)]
Allow e2fsprogs to be built using the clang (LLVM) frontend
Since clang uses C99 semantics by default, the main changes required
to allow clang to build e2fsprogs was to add support the C99 inline
semantics, while still allowing us to be built when the legacy (but
still default for gcc) GNU C89 inline semantics are in force.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Akira Fujita [Thu, 30 Aug 2012 11:16:01 +0000 (20:16 +0900)]
mke2fs: recalculate the reserved blocks when the last BG is dropped
mke2fs -m option can set reserved blocks ratio up to 50%. But if the
last block group is not big enough to support the necessary data
structures, it gets dropped, we have to recalculate the number of
reserved blocks so that the reserved blocks matches the requested
percentage.
It also avoids a problem where if the user specifies a reserved blocks
of 50%, and after the last partial block group was dropped, if the
number of reserved blocks is greater than 50%, e2fsck will complain.
Steps to reproduce:
1. Create a FS which has the overhead for the last BG
and specify 50 % for reserved blocks ratio
# mke2fs -m 50 -t ext4 DEV 1025M
mke2fs 1.42.5 (29-Jul-2012)
warning: 256 blocks unused.
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
656640 inodes, 2621440 blocks
1310848 blocks (50.00%) reserved for the super user
~~~~~~~ <-- Reserved blocks exceed 50% of FS blocks count!
2. e2fsck outputs filesystem corruption
# e2fsck DEV
e2fsck 1.42.5 (29-Jul-2012)
Corruption found in superblock. (r_blocks_count = 1310848).
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 32768 <device>
Signed-off-by: Akira Fujita <a-fujita@rs.jp.ne.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 7 Sep 2012 03:01:14 +0000 (23:01 -0400)]
configure: fix --enable-relative-symlink
The configure option --enable-relative-symlinks was incorrectly
specified in configure.in, as --enable-symlink-relative-symlinks. Fix
the configure script so that --enable-relative-symlinks works, as well
as previous incorrect command line option. We will keep the older,
incorrect --enable-symlink-relative-symlinks for at least two years
before removing it.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Mon, 3 Sep 2012 03:32:14 +0000 (23:32 -0400)]
Merge branch 'next'
Theodore Ts'o [Sun, 19 Aug 2012 22:11:46 +0000 (18:11 -0400)]
Merge branch 'maint' into next
Mike Frysinger [Sat, 18 Aug 2012 12:17:41 +0000 (12:17 +0000)]
util: respect HAVE_MALLOC_H
Most places respect this define, but this one doesn't.
Reported-by: Dmitri Bogomolov <4glitch@gmail.com>
Signed-off-by: Mike Frysinger <vapier@gentoo.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Bryce Harrington [Sat, 18 Aug 2012 00:10:46 +0000 (17:10 -0700)]
e2fsprogs: fix spelling of 'filesystme' in debugfs help
Signed-off-by: Bryce Harrington <bryce@canonical.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Tue, 14 Aug 2012 15:33:24 +0000 (11:33 -0400)]
tests: remove unused temporary files for MMP tests
The MMP tests need to be run on a real disk instead of tmpfs, since
the MMP block access is using O_DIRECT. As such, they create their
own test files in the local testing directory instead of using the
temporary file created in /tmp by the test_one script. Delete the
tempfs file before clobbering TMPFILE, otherwise it will leave the
unused file in /tmp after the test is completed.
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Zheng Liu [Tue, 7 Aug 2012 17:56:47 +0000 (13:56 -0400)]
libe2p/libext2fs: add EXT4_INLINE_DATA_FL flag
[ Also teach libe2p's print_flags() function to display this flag so
that lsattr will allow us to see whether a file has inline data or not.
--tytso ]
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Tue, 7 Aug 2012 17:53:22 +0000 (13:53 -0400)]
libext2fs: rename INCOMPAT_INLINEDATA to INCOMPAT_INLINE_DATA
This is what the patches from Zhen Liu uses, so let's make this change
now to keep things easier. INCOMPAT_INLINE_DATA also looks better
IMHO. :-)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Tue, 7 Aug 2012 17:46:13 +0000 (13:46 -0400)]
libext2fs: refactor the quota feature flag in the supported flags mask
Handle EXT4_FEATURE_RO_COMPAT_QUOTA the same way we handle INCOMPAT
features, so we don't have to have two definitions for
EXT2_LIB_FEATURE_RO_COMPAT_SUPP depending on whether or not
CONFIG_QUOTA is enabled or not.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Wed, 15 Aug 2012 20:43:55 +0000 (16:43 -0400)]
Merge branch 'maint' into next
Theodore Ts'o [Wed, 15 Aug 2012 17:00:14 +0000 (13:00 -0400)]
ext4: fix rehashing of the lost+found directory
Commit
07307114dea didn't correctly handle the lost+found directory
when it added support for metadata checksums. First of all,
e2fsck_get_lost_and_found() assumed that the inode_dir_map bitmap was
initialized, and it wasn't when it was called earlier by a change in
that commit. Secondly, it's important that lost+found dirctory is
processed in case its directory checksums are incorrect, but should
preserve any empty dirctory blocks so there space available for e2fsck
to reconnect any orphan inodes.
Fix these problems, to fix test failures: f_holedir2 and f_rehash_dir
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Zheng Liu [Tue, 7 Aug 2012 17:56:47 +0000 (13:56 -0400)]
libe2p/libext2fs: add EXT4_INLINE_DATA_FL flag
[ Also teach libe2p's print_flags() function to display this flag so
that lsattr will allow us to see whether a file has inline data or not.
--tytso ]
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Tue, 7 Aug 2012 17:53:22 +0000 (13:53 -0400)]
libext2fs: rename INCOMPAT_INLINEDATA to INCOMPAT_INLINE_DATA
This is what the patches from Zhen Liu uses, so let's make this change
now to keep things easier. INCOMPAT_INLINE_DATA also looks better
IMHO. :-)
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Tue, 7 Aug 2012 17:46:13 +0000 (13:46 -0400)]
libext2fs: refactor the quota feature flag in the supported flags mask
Handle EXT4_FEATURE_RO_COMPAT_QUOTA the same way we handle INCOMPAT
features, so we don't have to have two definitions for
EXT2_LIB_FEATURE_RO_COMPAT_SUPP depending on whether or not
CONFIG_QUOTA is enabled or not.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Tue, 7 Aug 2012 01:09:49 +0000 (21:09 -0400)]
Merge branch 'maint' into next
Jim Keniston [Mon, 6 Aug 2012 22:46:03 +0000 (18:46 -0400)]
e2fsck: fix potential segv when handling a read error in a superblock
When passed a negative count (indicating a byte count rather than
a block count) e2fsck_handle_read_error() treats the data as a full
block, causing unix_write_blk64() (which can handle negative counts
just fine) to try to write too much. Given a faulty block device,
this resulted in a SEGV when unix_write_blk64() read past the bottom
of the stack copying the data to cache. (check_backup_super_block ->
unix_read_blk64 -> raw_read_blk -> e2fsck_handle_read_error)
Reported-by: Alex Friedman <alexfr@il.ibm.com>
Signed-off-by: Jim Keniston <jkenisto@us.ibm.com>
Signed-off-by: Dan Streetman <ddstreet@us.ibm.com>
Reviewed-by: Mingming Cao <mcao@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sat, 4 Aug 2012 20:56:55 +0000 (16:56 -0400)]
Put ELF_OTHER_LIBS in the right place for the linker
Commit
a7c17431b9 attempted to fix a problem where the system
libraries might get used instead of local libraries for things like
-lcom_err. It tried to accomplish this by moving $(ELF_OTHER_LIBS) to
before $(LDFLAGS).
Unfortunately, this was the wrong fix; $(ELF_OTHER_LIBS) *MUST* be
after the object files, or the linker might not pull in the necessary
library and not include it into the DT_NEEDED section of the shared
library. The proper fix is to add a -L$(LIB) before $(LDFLAGS), and
then remove the -L option from all of the ELF_OTHER_LIBS definitions
in the library Makefiles.
Addresses-Sourceforge-Bug: #3554345
Cc: Olivier Blin <olivier.blin@softathome.com>
Reported-by: Mike Frysinger <vapier@gentoo.org>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Fri, 3 Aug 2012 02:06:16 +0000 (22:06 -0400)]
debian: Apply debian packaging fixes for metadata csum
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Theodore Ts'o [Fri, 3 Aug 2012 00:47:46 +0000 (20:47 -0400)]
libext2fs: when checking the inode's checksum, allow an all-zero inode
When the kernel writes an inode where all of the other inodes in in
the inode table (itable) block are unused, it skips reading the itable
block from disk, and instead uses an all zeros block. This can cause
e2fsck to complain when it iterates over the inodes using
ext2fs_get_next_inode() since the inode apparently has an invalid
checksum. Normally the inode won't be returned at all if it is at the
end of the block group's part of the inode table, thanks to the
bg_itable_unused field. But it's possible for this situation to
happen earlier in the inode table block.
Fix this by changing ext2fs_inode_csum_verify() to allow the inode to
be all zero's; if the checksum fails, and the inode is all zero's,
treat it as a valid checksum.
Reported-by: Tao Ma <boyu.tm@taobao.com>
Reported-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:46 +0000 (20:47 -0400)]
mke2fs: enable metadata_csum on ext4dev filesystems
Enable full-power metadata checksumming by default on 'ext4dev'
filesystems. This should be fairly safe for now, since only
developers should be using this new feature.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:46 +0000 (20:47 -0400)]
e2fsck: refactor crc32_be code
Remove crc32_be in favor of the implementation in libext2fs.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:46 +0000 (20:47 -0400)]
libext2fs: optimize the CRC32c implementation
The crc32c implementation in the kernel has been refactored a bit to
reduce the amount of code that needs to be maintained, and to speed up
tune2fs/e2fsck on PowerPC by 5-10%. Port the crc32c changes over, and
provide a crc32_be so that we can remove the duplicate functionality
from e2fsck. Also drop crc32c_be and crc32_le since neither got used.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:46 +0000 (20:47 -0400)]
libext2fs: enable support for the metadata checksumming feature
Add metadata checksumming to the list of supported features.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:46 +0000 (20:47 -0400)]
e2fsck: verify data block checksums when recovering journal
Check the data block checksums when recovering the journal.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:46 +0000 (20:47 -0400)]
e2fsck: check commit block checksum during recovery
When recovering a journal with checksum v2, verify the commit block
checksum.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:46 +0000 (20:47 -0400)]
e2fsck: check descriptor block checksum when recovering journal
Verify the descriptor block checksum when recovering a journal.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:45 +0000 (20:47 -0400)]
e2fsck: check revoke block checksum during recovery
Verify the revoke block checksum when recovering the journal.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:45 +0000 (20:47 -0400)]
e2fsck: check journal superblock checksum prior to recovery
Ensure that the journal superblock passes checksum before recovering the
filesystem.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:45 +0000 (20:47 -0400)]
libext2fs: dump feature flags for jbd2 v2 checksums
Modify the dump code to print information about jbd2 v2 checksum data.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:45 +0000 (20:47 -0400)]
libext2fs: change on-disk journal layout to support metadata checksumming
Define flags and change journal structure definitions to support v2 journal
checksumming.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:45 +0000 (20:47 -0400)]
tune2fs: force MMP update when changing metadata_csum flag
When changing the metadata_csum flag, always force out a new MMP block.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:45 +0000 (20:47 -0400)]
e2fsck: verify and correct MMP checksum problems
Check and handle MMP checksum problems by resetting the block.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:45 +0000 (20:47 -0400)]
libext2fs: add checksum to MMP block
Calculate and verify a checksum of the MMP block.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:45 +0000 (20:47 -0400)]
mke2fs: warn if not enabling all the features that metadata_csum wants
The metadata_csum feature works best when two features are enabled.
These features are "extents" (because the block map has no space for
checksums) and "64bit" (this enables storage of full 32-bit checksums
in certain fields). Print a warning if the user tries to create a
filesystem without those features.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:45 +0000 (20:47 -0400)]
mke2fs: write new group descriptors with the appropriate checksum
Update mke2fs to use the helper function to determine if group
descriptors should have checksums calculated. Since metadata_csum
supersedes uninit_bg, quietly drop uninit_bg if metadata_csum is set,
so that older kernels don't get confused.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:45 +0000 (20:47 -0400)]
tune2fs: rewrite block group checksums when changing checksumming feature flags
When toggling the metadata_csum and uninit_bg feature flags, we should
rewrite the block groups with the desired checksum.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:45 +0000 (20:47 -0400)]
e2fsck: ensure block group checksum uses
Use the helper function to determine if group descriptors have a
checksum. Ensure that metadata_csum and uninit_bg flags are not set
simultaneously, as part of pass 0.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:45 +0000 (20:47 -0400)]
libext2fs: block group checksum should use metadata_csum algorithm
Change the block group algorithm to use the same algorithm as the rest
of the metadata_csum. This mostly involves providing a helper
function to tell if group descriptors should have checksums set or
verified, and modifying the gdt checksum code to use the correct
algorithm.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:45 +0000 (20:47 -0400)]
mke2fs: record the checksum algorithm in use in the superblock
Record the type of checksum algorithm we're using for metadata in the
superblock when creating a filesystem.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:44 +0000 (20:47 -0400)]
tune2fs: store checksum algorithm type in superblock
Actually records the checksum algorithm type in the superblock when
enabling checksumming.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:44 +0000 (20:47 -0400)]
libext2fs: record the checksum algorithm in use in the superblock
Record the type of checksum algorithm we're using for metadata in the
superblock, in case we ever want/need to change the algorithm.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:44 +0000 (20:47 -0400)]
e2fsck: handle superblock checksum errors gracefully
If e2fsck finds a superblock with an invalid checksum, try the
backups.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:44 +0000 (20:47 -0400)]
libext2fs: calculate and verify superblock checksums
Calculate and verify the superblock checksums. Each copy of the
superblock records the number of the group it's in and the FS UUID, so
we can simply checksum the whole block.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:44 +0000 (20:47 -0400)]
tune2fs: rewrite extended attribute block checksums
When enabling metadata checksums, rewrite separate extended attribute
blocks.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:44 +0000 (20:47 -0400)]
e2fsck: check extended attribute block checksums
Verify the checksums of separate extended attribute blocks and offer
to clear it if there is a mismatch.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:44 +0000 (20:47 -0400)]
libext2fs: verify and calculate extended attribute block checksums
Calculate and verify the checksum for separate (i.e. not in the inode)
extended attribute blocks; the checksum lives in the header.
[ Merged in change from Tao so that we always use the fs checksum seed
for the xattr blocks. ]
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Tao Ma <boyu.mt@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 3 Aug 2012 00:47:35 +0000 (20:47 -0400)]
tune2fs: rebuild and checksum directories when necessary
Since all the metadata checksums depend on the fs UUID, tune2fs must
be able to rewrite the checksums of _all_ metadata. It's not that
hard to add in the bits to resize the directory block structures at
the same time.
[ Merged in fix from Zheng Liu where ctx.errcode wasn't getting
cleared in rewrite_directory(). ]
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Thu, 2 Aug 2012 21:27:43 +0000 (17:27 -0400)]
e2fsck: check directory leaf block checksums
Checks that directory leaf blocks have the necessary fake dir_entry at
the end of the block to hold a checksum and that the checksum is
valid. It will resize the block and/or rebuild the directory if
necessary.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Thu, 2 Aug 2012 21:27:43 +0000 (17:27 -0400)]
libext2fs: add checksums to the end of directory leaf nodes
Introduce small structures for recording directory tree checksums, and
some API changes to support writing out directory blocks with
checksums.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Thu, 2 Aug 2012 21:27:30 +0000 (17:27 -0400)]
e2fsck: verify htree root/node checksums
Check htree internal node checksums. If broken, ask user to clear
the htree index and recreate it later.
[ Move the check for not rehashing the lost+found directory to pass1
so that we don't end up truncating lost+found when the metadata
checksum feature is enabled. -- TYT ]
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 23:22:04 +0000 (19:22 -0400)]
libext2fs: add dx_root/dx_node checksum calculation and verification helpers
Verify and calculate checksums of htree internal node blocks.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 23:20:04 +0000 (19:20 -0400)]
debugfs: print htree internal node checksums
Print htree node checksums when dumping a directory index.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 23:18:04 +0000 (19:18 -0400)]
e2fsck: verify extent tree blocks and clear the bad ones
When we encounter an extent tree block that passes the header check
but fails the checksum, offer to clear just that extent block instead
of failing the whole tree, which results in the entire inode being
wiped out.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 23:16:04 +0000 (19:16 -0400)]
tune2fs: enable extent tree checksums
Add to tune2fs the ability to recalculate extent tree checksums when
altering the metadata checksum feature flag.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 23:14:04 +0000 (19:14 -0400)]
libext2fs: verify and calculate extent tree block checksums
Verify and calculate extent tree block checksums when processing
filesystems.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 23:12:04 +0000 (19:12 -0400)]
tune2fs: rewrite block bitmap checksums
When toggling metadata_csum, mark the block bitmap dirty so that it
gets written with new checksums.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 23:10:04 +0000 (19:10 -0400)]
e2fsck: don't verify bitmap checksums
Since the correct inode and block bitmaps are calculated in pass 5,
don't fail the bitmap read operation in prior passes since (a)
incorrect results won't kill us and (b) if we fail early, we'll never
_get_ to pass 5.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 23:08:04 +0000 (19:08 -0400)]
e2fsck: verify block bitmap checksum
Check block bitmap checksum and write a new checksum if the
verification fails. This is ok because e2fsck has already computed
the correct block bitmap.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 23:06:04 +0000 (19:06 -0400)]
dumpe2fs: display block bitmap checksum
Display the block bitmap checksum when displaying block groups.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 23:04:04 +0000 (19:04 -0400)]
libext2fs: create the block bitmap checksum
Calculate and verify the block bitmap checksum.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 23:02:04 +0000 (19:02 -0400)]
e2fsck: verify inode bitmap checksum
Rewrite the block bitmap when the checksum doesn't match. This is
ok since e2fsck will have already computed the correct inode bitmap.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 23:00:04 +0000 (19:00 -0400)]
dumpe2fs: display inode bitmap checksum
Display the inode bitmap checksum for each block group.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 22:58:04 +0000 (18:58 -0400)]
tune2fs: rewrite inode bitmap checksums
When toggling metadata_csum, mark the inode bitmap dirty so that they
are written out with new checksums.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 22:56:04 +0000 (18:56 -0400)]
libext2fs: create the inode bitmap checksum
Provide a field in the block group descriptor to store inode bitmap
checksum, and some helper functions to calculate and verify it.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 22:54:04 +0000 (18:54 -0400)]
mke2fs: allow metadata checksums to be turned on at mkfs time
Write out reserved inodes with full checksums when writing out a
zeroed table.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 22:52:04 +0000 (18:52 -0400)]
e2fsck: verify and correct inode checksums
Detect mismatches of the inode and checksum, and prompt the user to
fix the situation.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 22:50:04 +0000 (18:50 -0400)]
tune2fs: add inode checksum support
This patch adds to tune2fs the ability to toggle the metadata checksum
rocompat feature flag, which will rewrite the inode table with
checksums. Disallow changing the UUID while the fs is mounted,
because rewriting the metadata objects is racy.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 22:48:04 +0000 (18:48 -0400)]
debugfs: dump inode checksum when appropriate
Dump inode checksum when displaying inode info.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 30 Jul 2012 22:46:04 +0000 (18:46 -0400)]
libext2fs: add inode checksum support
This patch adds the ability for the libext2fs functions to read and
write the inode checksum.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>