Whamcloud - gitweb
Darrick J. Wong [Sun, 14 Dec 2014 02:57:20 +0000 (21:57 -0500)]
tune2fs: warn if extents are not enabled when turning on metadata_csum
Warn the user if we're trying to enable metadata_csum on a FS that
doesn't support extents (since block maps cannot contain checksums).
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 02:55:22 +0000 (21:55 -0500)]
e2fsck: don't complain about root dir csum failures when getting lnf
Don't complain about checksum failures on the root dir when we're
trying to find l+f if the root dir is going to be rehashed anyway.
The test case for this is t_enable_mcsum in the next patch.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 02:55:12 +0000 (21:55 -0500)]
e2fsck: only complain about no-checksum directory blocks once
If a directory block lacks space for a checksum and the user directs
e2fsck to fix the directory block (by rehashing it), don't complain a
second time about the checksum verification failure when we get to the
end of the directory block.
Also, don't complain about broken HTREE directories if we're already
planning to rebuild the HTREE directory.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 02:52:48 +0000 (21:52 -0500)]
dumpe2fs: output cleanup
Don't display unused inodes twice, and make it clear that we're
printing a descriptor checksum.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Cc: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 02:50:19 +0000 (21:50 -0500)]
dumpe2fs: reduce dumpe2fs output to 80 columns or less
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 02:50:13 +0000 (21:50 -0500)]
misc: don't allow mk_hugefiles unless the fs supports extents
The current mk_hugefile code in mke2fs doesn't support creating
non-extent files, so disable the functionality when we're mkfs'ing
without extent support.
The fallocate patches further on will eliminate the need for this.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 02:13:40 +0000 (21:13 -0500)]
libext2fs: add a way to check the theoretical maximum extent tree depth
Add an API so that client programs can discover a reasonable maximum
extent tree depth. This will eventually be used by e2fsck as one of
the criteria to decide if an extent-based file should have its extent
tree rebuilt.
Turn some related magic numbers into constants while we're at it.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 01:14:14 +0000 (20:14 -0500)]
libext2fs: set interior tree block goal more intelligently
When we're splitting an extent node, try to allocate the new interior
tree block just prior to the first extent in the block we're trying to
split. The previous logic only set a goal block if we had to split
both the current node and its parent, which is somewhat infrequent.
When that would happen, the goal would start at zero, leading to poor
locality.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sun, 14 Dec 2014 01:00:36 +0000 (20:00 -0500)]
libext2fs: find inode goal when allocating blocks
Try to be a little smarter about where we go to allocate blocks for a
inode. For a given inode and logical offset, set the goal as if the
file were physically continuous. If it's bmapped, just start looking
at wherever lblk 0 is. If that's not possible (the file has no
lblk>pblk mappings, inline data, etc.) then start looking in the
inode's block group.
[ Fixed memory leak --tytso ]
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sat, 13 Dec 2014 03:12:45 +0000 (22:12 -0500)]
libext2fs: use block_buf in ext2fs_alloc_block2() if it is provided
If the caller supplies a buffer to ext2fs_alloc_block2(), use it
instead of calling ext2fs_zero_blocks2().
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 12 Dec 2014 23:27:12 +0000 (18:27 -0500)]
libext2fs: use a dynamically sized block zeroing buffer
Dynamically grow the block zeroing buffer to a maximum of 4MB, and
allow callers to provide their own zeroed buffer in
ext2fs_zero_blocks2().
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Dmitry Monakhov [Thu, 11 Dec 2014 22:57:12 +0000 (17:57 -0500)]
ext2fs: fix integer overflow in rb_get_bmap_range
bmap_rb_extent is defined as __u64:blk __u64:count. So count can
exceed INT_MAX on populated filesystems.
TESTCASE: xfstest ext4/004
Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Thu, 11 Dec 2014 22:48:03 +0000 (17:48 -0500)]
e2fsck: force-reread of inode from disk when re-checking a checksum error
When we're rechecking an inode checksum failure, we need to force the
inode to be re-read from disk so that the verification routine runs,
so drop the stashed inode.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Wed, 3 Dec 2014 05:01:44 +0000 (00:01 -0500)]
e2fsck: remove 'invalid' wording from error when extent block fails checksum
Don't say the physical block number is invalid if an extent block
fails only the checksum. It passes checks, so it's not invalid.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Wed, 3 Dec 2014 03:57:14 +0000 (22:57 -0500)]
libext2fs: file IO routines should handle uninit blocks
The file IO routines do not handle uninit blocks at all. The read
method should check for the uninit flag and return a buffer of zeroes,
and the write routine should convert unwritten extents.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Wed, 3 Dec 2014 03:55:04 +0000 (22:55 -0500)]
libext2fs: refactor extent head creation
Don't open-code the creation of the extent tree header, since
ext2fs_extent_open2() knows how to take care of this.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Wed, 3 Dec 2014 03:38:19 +0000 (22:38 -0500)]
tests: regression test for resizing inodes on a metadata_csum fs
Regression test for a problem inadvertently fixed by the patchset
"e2fsprogs/tune2fs: fix memory leak in inode_scan_and_fix()" by Xiaoguang Wang.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Xiaoguang Wang [Wed, 3 Dec 2014 03:31:11 +0000 (22:31 -0500)]
tune2fs: fix memory write overflow
If we apply this patch 'e2fsprogs/tune2fs: rewrite metadata checksums
when resizing inode size', we will trigger a segfault, this is because
of the inode cache issues.
Firstly we should notice that in expand_inode_table(), we have change
the super block's s_inode_size to new inode size(for example, 256).
Then we re-compute metadata checksums, see below code flow:
|-->rewrite_metadata_checksums
|----->rewrite_inodes
|-------->ext2fs_write_inode_full
In ext2fs_write_inode_full(), if an inode cache is hit, the below code will be executed:
/* Check to see if the inode cache needs to be updated */
if (fs->icache) {
for (i=0; i < fs->icache->cache_size; i++) {
if (fs->icache->cache[i].ino == ino) {
memcpy(fs->icache->cache[i].inode, inode,
(bufsize > length) ? length : bufsize);
break;
}
}
}
Before executing rewrite_inodes(), actually the inode in inode cache
is allocated by old inode size(for example, 128), but here the memcpy
will obviously write overflow, '(bufsize > length) ? length : bufsize'
here will return 256(new inode size), so this is wrong, we need to fix
this. I think we should call ext2fs_free_inode_cache() in
expand_inode_table(), to drop the inode cache, because inode size has
changed, if necessary, we will re-create this inode cache.
Steps to reproduce this bug (apply 'tune2fs: rewrite metadata checksums
when resizing inode size' first):
dd if=/dev/zero of=file.img bs=1M count=128
device_name=$(/sbin/losetup -f)
/sbin/losetup -f file.img
mkfs.ext4 -I 128 -O ^flex_bg $device_name
tune2fs -I 256 $device_name
Signed-off-by: Xiaoguang Wang <wangxg.fnst@cn.fujitsu.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Xiaoguang Wang [Wed, 3 Dec 2014 03:29:29 +0000 (22:29 -0500)]
tune2fs: rewrite metadata checksums when resizing inode size
When we use tune2fs -I new_ino_size to change inode size, if
everything is OK, the corresponding ext4_group_desc.bg_free_blocks_count
will be decreased, so obviously, we need to re-compute the group
descriptor checksums, and the inode 's size has also changed, we also
need to recompute the checksums of inodes for metadata_csum
filesystem, so here we choose to call a rewrite_metadata_checksums(),
this will fix checksum issues.
Meanwhile, the patch will trigger an existing memory write overflow,
which will casue segfault, please see the next patch.
Signed-off-by: Xiaoguang Wang <wangxg.fnst@cn.fujitsu.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Wed, 3 Dec 2014 03:17:10 +0000 (22:17 -0500)]
libext2fs: don't report garbage inodes with really large inodes
If the inode size is large enough that there are fewer than two inodes
per block, don't report an inode checksum failure as a garbage inode
during the scan because the "more than half are broken" criteria that
we use to decide if a block of inodes is garbage doesn't really apply.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 3 Dec 2014 03:15:25 +0000 (22:15 -0500)]
Merge branch 'maint' into next
Darrick J. Wong [Wed, 3 Dec 2014 03:00:04 +0000 (22:00 -0500)]
misc: fix infinite loop when finding the start of the hugefile start range
When looking for the start of the hugefile range, the 'next' variable
is incorrectly decremented. If we happened to find a single free
block, the effect of this decrement is that blk == next, which means
that we never modify the loop control variable, so get_start_block
never returns.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Xiaoguang Wang [Wed, 3 Dec 2014 02:06:40 +0000 (21:06 -0500)]
tune2fs: fix memory leak in inode_scan_and_fix()
When we use ext2fs_open_inode_scan() to iterate inodes and finish
jobs, we also need a ext2fs_close_inode_scan(scan) operation, but in
inode_scan_and_fix(), we forgot to call it, fix this error.
Signed-off-by: Xiaoguang Wang <wangxg.fnst@cn.fujitsu.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Bernhard M. Wiedemann [Tue, 2 Dec 2014 20:23:55 +0000 (15:23 -0500)]
debugfs: fix typo in message
Signed-off-by: Bernhard M. Wiedemann <bwiedemann@suse.de>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 23 Nov 2014 20:14:19 +0000 (15:14 -0500)]
Update version.h file for experimental release
Darrick J. Wong [Mon, 17 Nov 2014 22:59:42 +0000 (17:59 -0500)]
libext2fs: don't allow alloc_stats on bad inode/block numbers
Don't allow callers to feed bad block/inode numbers to
ext2fs_*_alloc_stats2, because evil callers (<cough>resize2fs<cough>)
can corrupt library state this way, leading to a crash.
(There will be a subsequent patch to resize2fs to fix its bad
behavior.)
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 17 Nov 2014 22:41:07 +0000 (17:41 -0500)]
libext2fs: set BLOCK_UNINIT for non-last blockgroups if all blocks are free
Set BLOCK_UNINIT in any group whose blocks are all unused, so long as
it isn't the last group. This helps us speed up future e2fsck runs
and mounts because we don't need to read or checksum block bitmaps for
these groups.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sat, 8 Nov 2014 02:27:53 +0000 (21:27 -0500)]
libext2fs: fix endian handling error; reduce fragmentation some
If we're going to read the "nr - 1" entry in an indirect block for use
as a "goal" input to the block allocator, we need to byteswap the
entry. While we're at it, if we're allocating blocks for the zeroth
entry in the indirect block, we might as well use the indirect block
as the starting point to try to reduce fragmentation.
(d_fallocate_blkmap will test this...)
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sat, 8 Nov 2014 02:26:14 +0000 (21:26 -0500)]
dumpe2fs: don't crash when the user provides no block device argument
If the user doesn't provide any arguments, the guard fails to run and
the whole thing segfaults on ext2fs_open2(). Don't do that.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sat, 8 Nov 2014 02:23:41 +0000 (21:23 -0500)]
misc: fix compiler warnings and minor build errors
Fix some gcc-4.8 warnings and other problems that broke the build.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 7 Nov 2014 21:50:49 +0000 (13:50 -0800)]
tests: fix test scripts that don't work on non-Linux systems.
Fix various tests that break on non-Linux systems; in particular,
sed -i doesn't work the same on all platforms; and try to keep the
GNU getopt-isms out.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 27 Oct 2014 23:32:31 +0000 (16:32 -0700)]
mke2fs: warn if enabling metadata_csum on a pre-3.18 kernel
The metadata_csum feature (really, the journal checksum disk format)
didn't stabilize until the 3.18 kernel, at which point the companion
journal_csum feature was turned on by default if metadata_csum was
enabled. Therefore, warn the user if they try to create such a
filesystem on a pre-3.18 kernel.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 5 Nov 2014 16:15:06 +0000 (11:15 -0500)]
Merge branch 'maint' into next
Darrick J. Wong [Wed, 5 Nov 2014 16:14:26 +0000 (11:14 -0500)]
e2fsck: fix dangling pointer when dir_info array is resized
e2fsck uses an array to store directory usage information during pass
3; the usage context also contains a pointer to the last directory
looked up. When expanding the dir_info array, this cache pointer
needs to be cleared if the array resize changed the pointer location,
or else we'll later walk off the end of this dead pointer.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Wed, 5 Nov 2014 16:10:31 +0000 (11:10 -0500)]
e2fsck: fix reporting of unknown htree block inode number
Sami Liedes reports that e2fsck fails to report the correct directory
inode number during a pass2 check for unexpected HTREE blocks.
Provide the inode number in the problem report.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Wed, 5 Nov 2014 16:08:32 +0000 (11:08 -0500)]
libext2fs: fix endian checking bits
Commit
3e683eef936e6d ("define bitwise types and annotate conversion
routines") broke the build on various platforms. Turns out that
crossing our fingers wasn't such a good idea, so just define it
separately.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 4 Nov 2014 16:52:26 +0000 (11:52 -0500)]
tune2fs: don't change metadata_csum on a mounted fs
Don't let users change metadata_csum on a mounted filesystem because
there's no way to tell the kernel to turn on the feature; there's no
way to prevent the kernel from rewriting on-disk structures while
tune2fs is also rewriting them; and there's no way to tell the kernel
to reload them after tune2fs is finished.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 4 Nov 2014 16:52:03 +0000 (11:52 -0500)]
tune2fs: speed up rewriting extent tree when enabling metadata_csum
When enabling checksums, tune2fs naively rewrites every extent in the
entire tree! This is unnecessary since we only need to rewrite each
extent tree block; therefore, only rewrite the extent if it's the
first one in an internal extent tree block.
Also, don't bother iterating the extent tree when clearing checksums.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 4 Nov 2014 16:49:39 +0000 (11:49 -0500)]
misc: fix broken libmagic interaction with plausibility check
If we get as far as calling libmagic, return the correct error code so
that mkfs asks for confirmation if libmagic finds something and
doesn't ask if nothing is found.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 4 Nov 2014 16:47:30 +0000 (11:47 -0500)]
libext2fs: zero the EA block buffer before filling it
When writing an extended attribute (EA) block, it's quite possible
that the EA formatting code will not write the entire buffer.
Therefore, we must zero the buffer beforehand to avoid writing random
heap contents to disk.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Tue, 4 Nov 2014 16:46:55 +0000 (11:46 -0500)]
Merge branch 'maint' into next
Conflicts:
lib/ext2fs/dir_iterate.c
Darrick J. Wong [Tue, 4 Nov 2014 16:43:08 +0000 (11:43 -0500)]
libext2fs: don't memcpy identical pointers when writing a cache block
Sami Liedes found a scenario where we could memcpy incorrectly:
If a block read fails during an e2fsck run, the UNIX IO manager will
call the io->read_error routine with a pointer to the internal block
cache. The e2fsck read error handler immediately tries to write the
buffer back out to disk(!), at which point the block write code will
try to copy the buffer contents back into the block cache. Normally
this is fine, but not when the write buffer is the cache itself!
So, plumb in a trivial check for this condition. A more thorough
solution would pass a duplicated buffer to the IO error handlers, but
I don't know if that happens frequently enough to be worth the extra
point of failure.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sat, 25 Oct 2014 20:56:42 +0000 (13:56 -0700)]
libext2fs: directory iteration mustn't walk off the buffer end
When we're iterating a directory, the loop control code reads the
length of the next directory record, failing to account for the fact
that there must be at least 8 bytes (the minimum size of a directory
entry) left in the buffer to read the next directory record. Fix the
loop conditional so that we don't read off the end of the buffer.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 4 Nov 2014 16:36:28 +0000 (11:36 -0500)]
e2fuzz: exercise fuzzed blocks more aggressively
Exercise fuzzed metadata blocks more aggressively by expanding up to
50000 files (instead of just test.1, which might not hit anything).
Fix a typo while we're at it.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Tue, 4 Nov 2014 16:35:56 +0000 (11:35 -0500)]
e2fsck: fix compiler warnings
Fix some gcc-4.8 warnings.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Sandeen [Tue, 4 Nov 2014 16:26:24 +0000 (11:26 -0500)]
quotaio: annotate & fix up for sparse endian checker
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Sandeen [Tue, 4 Nov 2014 16:26:23 +0000 (11:26 -0500)]
libext2: minor sparse endian checker fixup
The sparse checker treats 0 assignments as special, but
doesn't catch a = b = 0; separate them to make it quieter.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Sandeen [Tue, 4 Nov 2014 16:26:21 +0000 (11:26 -0500)]
debugfs: don't swap htree nodes in-place
htree_dump_int_node() was swapping htree nodes in-place,
something not done elsewhere, so it made the sparse checker
unhappy. Just use dedicated variables to hold the 2 needed
(endian-swapped) values.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Sandeen [Tue, 4 Nov 2014 16:24:56 +0000 (11:24 -0500)]
endian-annotate most on-disk structures
This annotates most on-disk structures for endianness;
however it does not annotate some, like the superblock, inodes,
mmp, etc, as these are swapped in-place at this point. This is
a little inconsistent, but should help catch some endian mistakes,
at least.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Sandeen [Tue, 4 Nov 2014 16:21:57 +0000 (11:21 -0500)]
libext2fs: fix endian handling of ext3_extent_header in inline_data
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Eric Sandeen [Tue, 4 Nov 2014 16:21:13 +0000 (11:21 -0500)]
define bitwise types and annotate conversion routines
This lays the groundwork for sparse-checking e2fsprogs for
endianness; defines bitwise types, and fixes up the ext2fs_*
swapping routines to do the proper casts.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Tue, 4 Nov 2014 16:20:09 +0000 (11:20 -0500)]
Merge branch 'maint' into next
Conflicts:
configure
Eric Sandeen [Thu, 23 Oct 2014 21:27:32 +0000 (16:27 -0500)]
libext2fs: fix endian handling of ext3_extent_header
This turned up when trying to resize a filesystem containing
a file with many extents on PPC64.
Fix all locations where ext3_extent_header members aren't
handled in an endian-safe manner.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Theodore Ts'o [Mon, 20 Oct 2014 02:02:48 +0000 (22:02 -0400)]
util: allow subst to build on systems that do not have utimes()
Make subst more portable so it can deal with such oler systems that do
not have utimes(). Note that it is important that subst build
correctly without an autoconf-generated config.h (since that is what
happens on a cross-compile), as well as using whatever features are
available as determined by autoconf when doing a native build. We
currently assume the presence of utime(), but not utimes() or
futimes().
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sat, 18 Oct 2014 16:15:03 +0000 (12:15 -0400)]
contrib: add support for COLLAPSE_RANGE and ZERO_RANGE to falocate program
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sat, 18 Oct 2014 13:13:09 +0000 (09:13 -0400)]
debugfs: fix build on systems that don't have gettext built-in
Debugfs (unlike all of the other programs in e2fsprogs) is not set up
to use translated strings. So when building misc/plausible.c for
debugfs, we need to disable NLS.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Mon, 13 Oct 2014 15:59:30 +0000 (11:59 -0400)]
debugfs: add allocate and set functionality to the bmap command
This also makes it easier test the ext2fs_bmap2() function.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 13 Oct 2014 08:31:17 +0000 (04:31 -0400)]
libext2fs/e2fsck: refactor everyone who writes zero blocks to disk
Convert all call sites that write zero blocks to disk to use
ext2fs_zero_blocks2() since it can use Linux's zero out feature to do
the writes more quickly. Reclaim the zero buffer at freefs time and
make the write-zeroes fallback use a larger buffer.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Whitney [Mon, 13 Oct 2014 08:19:24 +0000 (04:19 -0400)]
mke2fs: fix man page discussion of usage type defaults
The man page description of the file system size thresholds used by
mke2fs to select a usage type when not otherwise specified by the -T
switch does not match the code.
Signed-off-by: Eric Whitney <enwlinux@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sat, 11 Oct 2014 23:26:55 +0000 (19:26 -0400)]
Merge branch 'maint' into next
Theodore Ts'o [Wed, 8 Oct 2014 16:09:35 +0000 (12:09 -0400)]
resize2fs: fix fs->blocksize dereference after fs has been freed
Commit
77255cf36944b introduced a use after free bug.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 8 Oct 2014 15:18:41 +0000 (11:18 -0400)]
e2fsck: fix free pointer dereferences
Commit
47fee2ef6a23a introduces some free pointer dereference bugs by
not clearing ctx->fs after calling ext2fs_close_free().
Reported-by: Matthias Andree <mandree@FreeBSD.org>
Cc: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Artemiy Volkov [Sat, 4 Oct 2014 17:22:52 +0000 (13:22 -0400)]
misc: fix build on absence of magic.h
If magic.h is not present, there will be unresolved references to the magic_t
type in plausible.c. Fix that by moving the protecting #ifdef directive.
Signed-off-by: Artemiy Volkov <artemiyv@acm.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Eric Whitney [Wed, 1 Oct 2014 12:36:36 +0000 (08:36 -0400)]
mke2fs: clarify inline data inode size error message
The existing error message can be made more helpful by more clearly
implying the attempt to make a file system with undersized inodes is
failing and suggesting a corrective action.
Signed-off-by: Eric Whitney <enwlinux@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Wed, 1 Oct 2014 12:36:28 +0000 (08:36 -0400)]
Merge branch 'maint' into next
Eric Sandeen [Wed, 1 Oct 2014 12:33:54 +0000 (08:33 -0400)]
e2fsprogs: add large_file to base mkfs features
large_file (> 2G) support has been around since at least kernel 2.4;
mkfs of any sufficiently large filesystem sets it "accidentally"
when the resize inode exceeds 2G. This leaves very small
filesystems lacking the feature, which potentially changes
their behavior & codepaths the first time a > 2G file gets
written.
There's really no reason to be making fresh filesystems which
strive to keep compatibility with 10 year old kernels; just
enable large_file at mkfs time. This is particularly obvious
for ext4 fielsystems, which set huge_file by default, but not
necessarily large_file.
If old-kernel compatibility is desired, mke2fs.conf can be
modified locally to remove the feature.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 21 Sep 2014 19:36:57 +0000 (15:36 -0400)]
libext2fs: use ~0UL instead of -1UL to avoid static checker warnings
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 21 Sep 2014 19:35:21 +0000 (15:35 -0400)]
MCONFIG: allow developers to override CFLAGS
Move other C compiler flags to ALL_CFLAGS so that CFLAGS only controls
the debugging and optimization flags. This allows a developer to
build with "make CFLAGS=-g" in order to compile w/o optimization.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Sun, 21 Sep 2014 05:25:58 +0000 (01:25 -0400)]
misc: if dlopen() is available, use it to load the libmagic library
This prevents the libmagic library from being a hard dependency; if
it's not there, we can simply do without it.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Cc: Darrick J. Wong <darrick.wong@oracle.com>
Darrick J. Wong [Sat, 13 Sep 2014 22:12:46 +0000 (15:12 -0700)]
misc: use libmagic when libblkid can't identify something
If we're using check_plausibility() to try to identify something that
obviously isn't an ext* filesystem and libblkid doesn't know what it
is, try libmagic instead.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Sat, 13 Sep 2014 22:12:39 +0000 (15:12 -0700)]
misc: add plausibility checks to debugfs/tune2fs/dumpe2fs/e2fsck
If any of these utilities detect a bad superblock magic, call
check_plausibility to see if blkid can identify the passed-in argument
as something else (xfs, partition, etc.) in the hopes of catching a
user error.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 19 Sep 2014 17:10:21 +0000 (13:10 -0400)]
misc: move check_plausibility into a separate file
Move check_plausibility() into a separate file so that various
programs can use it without having to declare useless global variables
that the util.c functions seem to require.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 19 Sep 2014 16:16:44 +0000 (12:16 -0400)]
dumpe2fs: provide a machine-readable group-only mode
Spit out just the group descriptor data in a machine readable format.
This is most useful for testing and scripting purposes.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Andreas Dilger [Fri, 19 Sep 2014 16:15:21 +0000 (12:15 -0400)]
ext2fs: add readahead method to improve scanning
Add a readahead method for prefetching ranges of disk blocks. This is
useful for inode table scanning, and other large contiguous ranges of
blocks, and may also prove useful for random block prefetch, since it
will allow reordering of the IO without waiting synchronously for the
reads to complete.
It is currently using the posix_fadvise(POSIX_FADV_WILLNEED)
interface, as this proved most efficient during our testing.
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 19 Sep 2014 06:01:43 +0000 (02:01 -0400)]
debugfs: manage needs_recover feature when messing with the journal
Set the needs_recover incompat feature when debugfs writes journal
transactions so that we actually replay the journal contents at the
next mount.
Likewise, clear it if we successfully recover the journal.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 19 Sep 2014 05:30:04 +0000 (01:30 -0400)]
e2fsck: use ext2fs_get_mem() instad of ext2fs_get_memalign()
There is no reason to request a aligned buffer in
check_{inode,block}_bitmap, and this will cause failures for dietlibc,
which doesn't have support for posix_memalign() or any other way to
request an aligned memory allocation. Fortunately, this is only
needed in very few places where direct I/O is required.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 19 Sep 2014 05:05:14 +0000 (01:05 -0400)]
Fix build failures due to missing $(SYSLIBS)
Two link lines were missing $(SYSLIBS), which is needed for dietlibc.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 19 Sep 2014 04:48:24 +0000 (00:48 -0400)]
e2fsck: fix build under dietlibc
The asm_types.h file needs to include stdio.h and stdlib.h in order to
get integer types included. So add those includes into jfs_user.h to
avoid a build faliure under dietlibc.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 19 Sep 2014 04:46:41 +0000 (00:46 -0400)]
create_inode: don't indiscriminately #include "nls-enable.h"
The create_inode.h header file is pulled in by debugfs, which is not
internationalized. It had no business pulling in nls-enable.h; that
header file should only be used in specific .c files that support
internationalization.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 19 Sep 2014 04:28:46 +0000 (00:28 -0400)]
Merge branch 'maint' into next
Theodore Ts'o [Fri, 19 Sep 2014 04:26:56 +0000 (00:26 -0400)]
misc: fix build breakage with configure --disable-uuidd --enable-profile
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 19 Sep 2014 04:26:26 +0000 (00:26 -0400)]
configure: disable uuidd by default if libuuid is disabled
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 19 Sep 2014 04:04:24 +0000 (00:04 -0400)]
mke2fs: don't depend on <linux/version.h>
Define the KERNEL_VERSION macro explicitly instead of using
<linux/version.h>, since it's not available when using dietlibc.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 19 Sep 2014 01:47:38 +0000 (21:47 -0400)]
misc: fix Coverity complaints
Fix a few problems that Coverity picked up with error handling.
Fixes-Coverity-Bug: 1239278
Fixes-Coverity-Bug: 1239279
Fixes-Coverity-Bug: 1239280
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 19 Sep 2014 01:46:10 +0000 (21:46 -0400)]
e2fsck: free bh when descriptor block checksum fails
Free the buffer head if the journal descriptor block fails checksum
verification. This has been patched before (see "e2fsck: free bh on
csum verify error in do_one_pass") but apparently the patch was never
committed to jbd2 in the kernel, so when we resync'd the recovery code
with 3.16, the bug came back. Sigh.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Cc: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Darrick J. Wong [Fri, 19 Sep 2014 01:29:21 +0000 (21:29 -0400)]
e2fsck: fix sliding the directory block down on bigalloc
If we find a hole in a directory on a bigalloc filesystem, we need to
obey the cluster alignment rules when collapsing the gap to avoid
later complaints.
Specifically, the calculation of the new logical cluster number was
incorrect, and we need to ensure that the logical cluster alignment
respects the physical cluster alignment, since we've concluded that
the extent's logical block number is wrong.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Fri, 19 Sep 2014 01:29:19 +0000 (21:29 -0400)]
e2fsck: offer to clear overlapping extents
If in the course of iterating extents we find that an otherwise
valid-seeming second extent maps the same logical blocks as a
previously examined first extent, offer to clear the duplicate
mapping.
The test for this is already in f_extents.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 19 Sep 2014 01:28:59 +0000 (21:28 -0400)]
Merge branch 'maint' into next
Darrick J. Wong [Fri, 19 Sep 2014 01:24:26 +0000 (21:24 -0400)]
misc: zero s_jnl_blocks when adding journal online or removing external journal
Erase s_jnl_blocks when removing an external journal, or adding an
internal journal online. We can't add the backup for the internal
journal because we have no good way to get the indirect block or ETB
addresses, so the best we can do is hope that the user runs e2fsck,
which will correct that. We are motivated to erase during external
journal removal to state emphatically that there's no journal.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: thomas_reardon@hotmail.com
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Fri, 19 Sep 2014 01:11:33 +0000 (21:11 -0400)]
fix build with configure --disable-uuid --disable-blkid --enable-profile
We need to make sure PROFILED_LIBUUID and PROFILED_LIBBLKID are
defined when we are using the system uuid and blkid libraries.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Michael Forney [Mon, 15 Sep 2014 18:30:00 +0000 (14:30 -0400)]
Don't clear BUILD_CFLAGS and BUILD_LDFLAGS when cross-compiling
Signed-off-by: Michael Forney <forney@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Theodore Ts'o [Thu, 11 Sep 2014 23:15:22 +0000 (19:15 -0400)]
lib/ext2fs: fix Makefile to avoid a build splat when building without VPATH
When building in the source tree, the order of the includes caused the
compiling of debugfs/journal.c while in the lib/ext2fs directory to
find the version in lib/ext2fs instead of the desired version in
e2fsck/jfs_user.h.
We need to eventually get rid of this whole mess and have only one
jfs_user.h and build the journal-related functions once in an internal
library which is used only by e2fsprogs progams.
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reported-by: "Darrick J. Wong" <darrick.wong@oracle.com>
Darrick J. Wong [Thu, 11 Sep 2014 20:17:44 +0000 (13:17 -0700)]
libext2fs: check ea value offset when loading
When reading extended attributes, check e_value_offs to make sure that
it starts in the value area and not the name area. The attached test
case image will crash the kernel if it is mounted and you append more
than 4096 bytes of data to /a, due to insufficient validation.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Thu, 11 Sep 2014 19:48:04 +0000 (12:48 -0700)]
e2fsck: expand root dir if linking l+f fails
If there isn't space in the root directory to add the lost+found
entry, try expanding the root directory before failing the fsck.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Thu, 11 Sep 2014 19:44:49 +0000 (12:44 -0700)]
e2fsck: ignore badblocks if it says badblocks inode is bad
If the badblocks list says that the badblocks inode is bad, it's quite
likely that badblocks is broken. Worse yet, if the root inode is in
the same block as the badblocks inode (likely since they're adjacent),
the filesystem becomes unfixable because pass3 notices the bad root
inode and exits.
So... if we encounter this case, just kill the badblocks inode.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Thu, 11 Sep 2014 19:43:33 +0000 (12:43 -0700)]
tune2fs: always check disable_uninit_bg() return code
Enhance disable_uninit_bg() to return error codes -- if something goes
wrong, we want to flag the FS as needing a fsck and exit. Mr. Reardon
discovered that tune2fs -O ^metadata_csum on a FS with a corrupt
bitmap would leave the FS in a weird state.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Thu, 11 Sep 2014 19:31:41 +0000 (12:31 -0700)]
tests: test e2fsck recovery of corrupt descriptor blocks
Test e2fsck' ability to deal with (a) corrupt descriptor block
checksum; (b) obviously bad journal block tid; and (c) corrupt journal
blocks. These should exercise the journal recovery infinite loop
bugfix earlier in this patchset.
This test also ensures that (with metadata_csum and journal_csum_v3)
journal replay continues past a corrupt journal block.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 8 Sep 2014 23:14:01 +0000 (16:14 -0700)]
tests: test recovery from an external journal
Add a couple of tests to verify that writing to and recovering from
an external journal work properly.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 8 Sep 2014 23:13:48 +0000 (16:13 -0700)]
tests: test e2fsck recovery with broken commit blocks
Test e2fsck' recovery of commit blocks with (a) only a corrupt
checksum and (b) an obviously incorrect tid.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Darrick J. Wong [Mon, 8 Sep 2014 23:13:41 +0000 (16:13 -0700)]
tests: test e2fsck recovery of corrupt revoke blocks
Test e2fsck' ability to deal with (a) revoke blocks with a bad
checksum and (b) revoke blocks with an obviously bad block number.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>