mke2fs: warn if enabling metadata_csum on a pre-3.18 kernel

The metadata_csum feature (really, the journal checksum disk format)
didn't stabilize until the 3.18 kernel, at which point the companion
journal_csum feature was turned on by default if metadata_csum was
enabled.  Therefore, warn the user if they try to create such a
filesystem on a pre-3.18 kernel.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Merge branch 'maint' into next

e2fsck: fix dangling pointer when dir_info array is resized

e2fsck uses an array to store directory usage information during pass
3; the usage context also contains a pointer to the last directory
looked up.  When expanding the dir_info array, this cache pointer
needs to be cleared if the array resize changed the pointer location,
or else we'll later walk off the end of this dead pointer.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: fix reporting of unknown htree block inode number

Sami Liedes reports that e2fsck fails to report the correct directory
inode number during a pass2 check for unexpected HTREE blocks.
Provide the inode number in the problem report.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libext2fs: fix endian checking bits

Commit 3e683eef936e6d ("define bitwise types and annotate conversion
routines") broke the build on various platforms.  Turns out that
crossing our fingers wasn't such a good idea, so just define it
separately.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tune2fs: don't change metadata_csum on a mounted fs

Don't let users change metadata_csum on a mounted filesystem because
there's no way to tell the kernel to turn on the feature; there's no
way to prevent the kernel from rewriting on-disk structures while
tune2fs is also rewriting them; and there's no way to tell the kernel
to reload them after tune2fs is finished.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tune2fs: speed up rewriting extent tree when enabling metadata_csum

When enabling checksums, tune2fs naively rewrites every extent in the
entire tree!  This is unnecessary since we only need to rewrite each
extent tree block; therefore, only rewrite the extent if it's the
first one in an internal extent tree block.

Also, don't bother iterating the extent tree when clearing checksums.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

misc: fix broken libmagic interaction with plausibility check

If we get as far as calling libmagic, return the correct error code so
that mkfs asks for confirmation if libmagic finds something and
doesn't ask if nothing is found.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libext2fs: zero the EA block buffer before filling it

When writing an extended attribute (EA) block, it's quite possible
that the EA formatting code will not write the entire buffer.
Therefore, we must zero the buffer beforehand to avoid writing random
heap contents to disk.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Merge branch 'maint' into next

Conflicts:
lib/ext2fs/dir_iterate.c

libext2fs: don't memcpy identical pointers when writing a cache block

Sami Liedes found a scenario where we could memcpy incorrectly:

If a block read fails during an e2fsck run, the UNIX IO manager will
call the io->read_error routine with a pointer to the internal block
cache.  The e2fsck read error handler immediately tries to write the
buffer back out to disk(!), at which point the block write code will
try to copy the buffer contents back into the block cache.  Normally
this is fine, but not when the write buffer is the cache itself!

So, plumb in a trivial check for this condition.  A more thorough
solution would pass a duplicated buffer to the IO error handlers, but
I don't know if that happens frequently enough to be worth the extra
point of failure.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libext2fs: directory iteration mustn't walk off the buffer end

When we're iterating a directory, the loop control code reads the
length of the next directory record, failing to account for the fact
that there must be at least 8 bytes (the minimum size of a directory
entry) left in the buffer to read the next directory record.  Fix the
loop conditional so that we don't read off the end of the buffer.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fuzz: exercise fuzzed blocks more aggressively

Exercise fuzzed metadata blocks more aggressively by expanding up to
50000 files (instead of just test.1, which might not hit anything).
Fix a typo while we're at it.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: fix compiler warnings

Fix some gcc-4.8 warnings.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

quotaio: annotate & fix up for sparse endian checker

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libext2: minor sparse endian checker fixup

The sparse checker treats 0 assignments as special, but
doesn't catch a = b = 0; separate them to make it quieter.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

debugfs: don't swap htree nodes in-place

htree_dump_int_node() was swapping htree nodes in-place,
something not done elsewhere, so it made the sparse checker
unhappy.  Just use dedicated variables to hold the 2 needed
(endian-swapped) values.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

endian-annotate most on-disk structures

This annotates most on-disk structures for endianness;
however it does not annotate some, like the superblock, inodes,
mmp, etc, as these are swapped in-place at this point.  This is
a little inconsistent, but should help catch some endian mistakes,
at least.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libext2fs: fix endian handling of ext3_extent_header in inline_data

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>

define bitwise types and annotate conversion routines

This lays the groundwork for sparse-checking e2fsprogs for
endianness; defines bitwise types, and fixes up the ext2fs_*
swapping routines to do the proper casts.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Merge branch 'maint' into next

Conflicts:
configure

libext2fs: fix endian handling of ext3_extent_header

This turned up when trying to resize a filesystem containing
a file with many extents on PPC64.

Fix all locations where ext3_extent_header members aren't
handled in an endian-safe manner.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>

util: allow subst to build on systems that do not have utimes()

Make subst more portable so it can deal with such oler systems that do
not have utimes().  Note that it is important that subst build
correctly without an autoconf-generated config.h (since that is what
happens on a cross-compile), as well as using whatever features are
available as determined by autoconf when doing a native build.  We
currently assume the presence of utime(), but not utimes() or
futimes().

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

contrib: add support for COLLAPSE_RANGE and ZERO_RANGE to falocate program

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

debugfs: fix build on systems that don't have gettext built-in

Debugfs (unlike all of the other programs in e2fsprogs) is not set up
to use translated strings.  So when building misc/plausible.c for
debugfs, we need to disable NLS.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

debugfs: add allocate and set functionality to the bmap command

This also makes it easier test the ext2fs_bmap2() function.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libext2fs/e2fsck: refactor everyone who writes zero blocks to disk

Convert all call sites that write zero blocks to disk to use
ext2fs_zero_blocks2() since it can use Linux's zero out feature to do
the writes more quickly.  Reclaim the zero buffer at freefs time and
make the write-zeroes fallback use a larger buffer.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

mke2fs: fix man page discussion of usage type defaults

The man page description of the file system size thresholds used by
mke2fs to select a usage type when not otherwise specified by the -T
switch does not match the code.

Signed-off-by: Eric Whitney <enwlinux@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Merge branch 'maint' into next

resize2fs: fix fs->blocksize dereference after fs has been freed

Commit 77255cf36944b introduced a use after free bug.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: fix free pointer dereferences

Commit 47fee2ef6a23a introduces some free pointer dereference bugs by
not clearing ctx->fs after calling ext2fs_close_free().

Reported-by: Matthias Andree <mandree@FreeBSD.org>
Cc: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

misc: fix build on absence of magic.h

If magic.h is not present, there will be unresolved references to the magic_t
type in plausible.c. Fix that by moving the protecting #ifdef directive.

Signed-off-by: Artemiy Volkov <artemiyv@acm.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

mke2fs: clarify inline data inode size error message

The existing error message can be made more helpful by more clearly
implying the attempt to make a file system with undersized inodes is
failing and suggesting a corrective action.

Signed-off-by: Eric Whitney <enwlinux@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Merge branch 'maint' into next

e2fsprogs: add large_file to base mkfs features

large_file (> 2G) support has been around since at least kernel 2.4;
mkfs of any sufficiently large filesystem sets it "accidentally"
when the resize inode exceeds 2G.  This leaves very small
filesystems lacking the feature, which potentially changes
their behavior & codepaths the first time a > 2G file gets
written.

There's really no reason to be making fresh filesystems which
strive to keep compatibility with 10 year old kernels; just
enable large_file at mkfs time.  This is particularly obvious
for ext4 fielsystems, which set huge_file by default, but not
necessarily large_file.

If old-kernel compatibility is desired, mke2fs.conf can be
modified locally to remove the feature.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libext2fs: use ~0UL instead of -1UL to avoid static checker warnings

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

MCONFIG: allow developers to override CFLAGS

Move other C compiler flags to ALL_CFLAGS so that CFLAGS only controls
the debugging and optimization flags.  This allows a developer to
build with "make CFLAGS=-g" in order to compile w/o optimization.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

misc: if dlopen() is available, use it to load the libmagic library

This prevents the libmagic library from being a hard dependency; if
it's not there, we can simply do without it.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Cc: Darrick J. Wong <darrick.wong@oracle.com>

misc: use libmagic when libblkid can't identify something

If we're using check_plausibility() to try to identify something that
obviously isn't an ext* filesystem and libblkid doesn't know what it
is, try libmagic instead.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

misc: add plausibility checks to debugfs/tune2fs/dumpe2fs/e2fsck

If any of these utilities detect a bad superblock magic, call
check_plausibility to see if blkid can identify the passed-in argument
as something else (xfs, partition, etc.) in the hopes of catching a
user error.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

misc: move check_plausibility into a separate file

Move check_plausibility() into a separate file so that various
programs can use it without having to declare useless global variables
that the util.c functions seem to require.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

dumpe2fs: provide a machine-readable group-only mode

Spit out just the group descriptor data in a machine readable format.
This is most useful for testing and scripting purposes.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

ext2fs: add readahead method to improve scanning

Add a readahead method for prefetching ranges of disk blocks.  This is
useful for inode table scanning, and other large contiguous ranges of
blocks, and may also prove useful for random block prefetch, since it
will allow reordering of the IO without waiting synchronously for the
reads to complete.

It is currently using the posix_fadvise(POSIX_FADV_WILLNEED)
interface, as this proved most efficient during our testing.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

debugfs: manage needs_recover feature when messing with the journal

Set the needs_recover incompat feature when debugfs writes journal
transactions so that we actually replay the journal contents at the
next mount.

Likewise, clear it if we successfully recover the journal.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: use ext2fs_get_mem() instad of ext2fs_get_memalign()

There is no reason to request a aligned buffer in
check_{inode,block}_bitmap, and this will cause failures for dietlibc,
which doesn't have support for posix_memalign() or any other way to
request an aligned memory allocation.  Fortunately, this is only
needed in very few places where direct I/O is required.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Fix build failures due to missing $(SYSLIBS)

Two link lines were missing $(SYSLIBS), which is needed for dietlibc.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: fix build under dietlibc

The asm_types.h file needs to include stdio.h and stdlib.h in order to
get integer types included.  So add those includes into jfs_user.h to
avoid a build faliure under dietlibc.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

create_inode: don't indiscriminately #include "nls-enable.h"

The create_inode.h header file is pulled in by debugfs, which is not
internationalized.  It had no business pulling in nls-enable.h; that
header file should only be used in specific .c files that support
internationalization.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Merge branch 'maint' into next

misc: fix build breakage with configure --disable-uuidd --enable-profile

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

configure: disable uuidd by default if libuuid is disabled

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

mke2fs: don't depend on <linux/version.h>

Define the KERNEL_VERSION macro explicitly instead of using
<linux/version.h>, since it's not available when using dietlibc.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

misc: fix Coverity complaints

Fix a few problems that Coverity picked up with error handling.

Fixes-Coverity-Bug: 1239278
Fixes-Coverity-Bug: 1239279
Fixes-Coverity-Bug: 1239280
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: free bh when descriptor block checksum fails

Free the buffer head if the journal descriptor block fails checksum
verification.  This has been patched before (see "e2fsck: free bh on
csum verify error in do_one_pass") but apparently the patch was never
committed to jbd2 in the kernel, so when we resync'd the recovery code
with 3.16, the bug came back.  Sigh.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Cc: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>

e2fsck: fix sliding the directory block down on bigalloc

If we find a hole in a directory on a bigalloc filesystem, we need to
obey the cluster alignment rules when collapsing the gap to avoid
later complaints.

Specifically, the calculation of the new logical cluster number was
incorrect, and we need to ensure that the logical cluster alignment
respects the physical cluster alignment, since we've concluded that
the extent's logical block number is wrong.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: offer to clear overlapping extents

If in the course of iterating extents we find that an otherwise
valid-seeming second extent maps the same logical blocks as a
previously examined first extent, offer to clear the duplicate
mapping.

The test for this is already in f_extents.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Merge branch 'maint' into next

misc: zero s_jnl_blocks when adding journal online or removing external journal

Erase s_jnl_blocks when removing an external journal, or adding an
internal journal online.  We can't add the backup for the internal
journal because we have no good way to get the indirect block or ETB
addresses, so the best we can do is hope that the user runs e2fsck,
which will correct that.  We are motivated to erase during external
journal removal to state emphatically that there's no journal.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: thomas_reardon@hotmail.com
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

fix build with configure --disable-uuid --disable-blkid --enable-profile

We need to make sure PROFILED_LIBUUID and PROFILED_LIBBLKID are
defined when we are using the system uuid and blkid libraries.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Don't clear BUILD_CFLAGS and BUILD_LDFLAGS when cross-compiling

Signed-off-by: Michael Forney <forney@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

lib/ext2fs: fix Makefile to avoid a build splat when building without VPATH

When building in the source tree, the order of the includes caused the
compiling of debugfs/journal.c while in the lib/ext2fs directory to
find the version in lib/ext2fs instead of the desired version in
e2fsck/jfs_user.h.

We need to eventually get rid of this whole mess and have only one
jfs_user.h and build the journal-related functions once in an internal
library which is used only by e2fsprogs progams.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reported-by: "Darrick J. Wong" <darrick.wong@oracle.com>

libext2fs: check ea value offset when loading

When reading extended attributes, check e_value_offs to make sure that
it starts in the value area and not the name area.  The attached test
case image will crash the kernel if it is mounted and you append more
than 4096 bytes of data to /a, due to insufficient validation.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: expand root dir if linking l+f fails

If there isn't space in the root directory to add the lost+found
entry, try expanding the root directory before failing the fsck.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: ignore badblocks if it says badblocks inode is bad

If the badblocks list says that the badblocks inode is bad, it's quite
likely that badblocks is broken.  Worse yet, if the root inode is in
the same block as the badblocks inode (likely since they're adjacent),
the filesystem becomes unfixable because pass3 notices the bad root
inode and exits.

So... if we encounter this case, just kill the badblocks inode.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tune2fs: always check disable_uninit_bg() return code

Enhance disable_uninit_bg() to return error codes -- if something goes
wrong, we want to flag the FS as needing a fsck and exit.  Mr. Reardon
discovered that tune2fs -O ^metadata_csum on a FS with a corrupt
bitmap would leave the FS in a weird state.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: test e2fsck recovery of corrupt descriptor blocks

Test e2fsck' ability to deal with (a) corrupt descriptor block
checksum; (b) obviously bad journal block tid; and (c) corrupt journal
blocks.  These should exercise the journal recovery infinite loop
bugfix earlier in this patchset.

This test also ensures that (with metadata_csum and journal_csum_v3)
journal replay continues past a corrupt journal block.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: test recovery from an external journal

Add a couple of tests to verify that writing to and recovering from
an external journal work properly.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: test e2fsck recovery with broken commit blocks

Test e2fsck' recovery of commit blocks with (a) only a corrupt
checksum and (b) an obviously incorrect tid.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: test e2fsck recovery of corrupt revoke blocks

Test e2fsck' ability to deal with (a) revoke blocks with a bad
checksum and (b) revoke blocks with an obviously bad block number.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: test how e2fsck recovers from corrupt journal superblocks

Test e2fsck' ability to deal with corrupt journal superblock checksum
and a bad magic.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: test recovery of 32 and 64-bit journals with the v2 checksum

Add tests to ensure that we know how to recover journals with the
csum_v2 feature set.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: write and replay blocks with the old journal checksum

Test that we can write and replay transactions with the old journal
checksum algorithm.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: test writing and recovering 32bit csum_v3 journals

Simple tests for the 32bit journal transaction creation code when
journal and metadata_csum are enabled.  We test the following:

(a) writing and replaying transactions with multiple
    descriptor blocks
(b) same, but with multiple revoke blocks.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: test writing and recovering 64bit csum_v3 journals

Simple tests for the 64bit journal transaction creation code when
journal and metadata_csum are enabled.   We test writing (bad) block
bitmaps out through the journal and replaying them via fsck, with a
few twists:

(a) All bitmaps are committed (fs errors reported)
(b) All the bitmap blocks are revoked (no errors)
(c) The transaction is never committed (no errors)
(d) Same as (a), but debugfs gets to do the replay.

We also test:

(a) writing and replaying transactions with multiple
    descriptor blocks
(b) same, but with multiple revoke blocks.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: test writing and recovering checksum-free 32/64bit journals

Simple tests for the journal transaction creation code.   We test
writing (bad) block bitmaps out through the journal and replaying them
via fsck, with a few twists:

(a) All bitmaps are committed (fs errors reported)
(b) All the bitmap blocks are revoked (no errors)
(c) The transaction is never committed (no errors)
(d) Same as (a), but debugfs gets to do the replay.

We also test:

(a) writing and replaying transactions with multiple
    descriptor blocks
(b) same, but with multiple revoke blocks.
(c) adding the 64bit flag to a journal

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

debugfs: add the ability to write transactions to the journal

Extend debugfs with the ability to create transactions and replay the
journal.  This will eventually be used to test kernel recovery and
metadata_csum recovery.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: fix minor errors in journal handling

The journal superblock's s_sequence field seems to track the tid of
the tail (oldest) transaction in the log.  Therefore, when we release
the journal, set the s_sequence to the tail_sequence, because setting
it to the transaction_sequence means that we're setting the tid to
that of the head of the log.  Granted, for replay these two are
usually the same (and s_start == 0 anyway) so thus far we've gotten
lucky and nobody noticed.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

debugfs: create journal handling routines

Create a journal.c with routines adapted from e2fsck/journal.c to
handle opening and closing the journal, and setting up the
descriptors, and all that.  Unlike e2fsck's versions which try to
identify and fix problems, the routines here have no way to repair
anything.

[ Modified by tytso to fold debugfs/jfs_user.h into e2fsck/jfs_user.h,
  so we don't have to copy recovery.c and revoke.c into debugfs. --tytso ]

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

misc: zero s_jnl_blocks when removing internal journal

When we're removing the internal journal (broken journal, turning it
off, or adding an external journal), zero s_jnl_blocks so that they
can't be picked up by accident later.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Cc: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

dumpe2fs: display external journal feature flags

Display the feature flags of an external journal.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tune2fs: explicitly disallow tuning of journal devices

Spit out a more specific error if someone tries to modify an
external journal device.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: detect and repair external journal superblock checksum errors

Verify the (ext4) superblock checksum of an external journal device
and prompt to correct the checksum if nothing else is wrong with the
superblock.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Cc: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

mke2fs: allow creation of journal device with superblock checksum

Enable mke2fs to create an external journal device with a superblock
checksum.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Cc: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libext2fs: write_journal_inode should check iterate return value

When creating a journal inode, check the return value from
block_iterate3() because otherwise we fail to capture errors such as
being unable to allocate an extent tree block, which leads to e2fsck
creating broken journals.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

misc: don't return ENOMEM if we run out of disk space

If there aren't enough blocks in the FS to allocate all of the
hugefiles, return ENOSPC, not ENOMEM.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libext2fs: report bad magic over bad sb checksum

We don't want ext2fs_open2() to report bad sb checksum on something
that's not even an ext* superblock.  This apparently happens pretty
easily if we try to open an XFS filesystem.  Thus, make it so that a
bad magic number code always trumps the sb checksum error code.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck/debugfs: fix descriptor block size handling errors with journal_csum

It turns out that there are some serious problems with the on-disk
format of journal checksum v2.  The foremost is that the function to
calculate descriptor tag size returns sizes that are too big.  This
causes alignment issues on some architectures and is compounded by the
fact that some parts of jbd2 use the structure size (incorrectly) to
determine the presence of a 64bit journal instead of checking the
feature flags.  These errors regrettably lead to the journal
corruption reported by Mr. Reardon.

Therefore, introduce journal checksum v3, which enlarges the
descriptor block tag format to allow for full 32-bit checksums of
journal blocks, fix the journal tag function to return the correct
sizes, and fix the jbd2 recovery code to use feature flags to
determine 64bitness.

Add a few function helpers so we don't have to open-code quite so
many pieces.

Switching to a 16-byte block size was found to increase journal size
overhead by a maximum of 0.1%, to convert a 32-bit journal with no
checksumming to a 32-bit journal with checksum v3 enabled.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reported-by: TR Reardon <thomas_reardon@hotmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Merge branch 'maint' into next

Conflicts:
debugfs/debugfs.c
e2fsck/Makefile.in
lib/ext2fs/Makefile.in
tests/test_config

Merge remote-tracking branch 'origin/maint' into maint

e2fsck: notice when the realloc of dir_info fails

If the reallocation of dir_info fails, we will eventually cause e2fsck
to fail with an internal error.  So if the realloc fails, print a
message and bail out with a fatal error early when at the time of the
reallocation failure.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

compile_et: Allow user to override ET_DIR

Signed-off-by: Michael Forney <forney@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Apply LDFLAGS when building tests

Signed-off-by: Michael Forney <forney@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: Add to LD_LIBRARY_PATH instead of overriding

Signed-off-by: Michael Forney <forney@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

debugfs: add better error checking when printing extended attributes

Check to make sure the length of the name and value fields in the
extended attribute don't result in overrun the bounds of the inode.

Addresses-Coverity-Bug: #709517

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Merge tag 'v1.42.12' into next

v1.42.12

Conflicts:
version.h

Update release notes, etc. for final 1.42.12 release

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

po: update vi.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

po: update uk.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

po: update sv.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

po: update pl.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>