e2fsck: move some fixes out of parallel pthreads

We could only use @found_map_block to find free blocks
after we have collectd all used blocks, so something like
handle_fs_bad_blocks(), ext2fs_create_resize_inode(),
e2fsck_pass1_dupblocks() really should be handled after
all threads has been finished.

E2fsprogs-commit: 6b1d459bf6018eea7ff0a0ae6b555276718dcda3

Change-Id: Ibdbaed3d218623daa0baec31837037d773799262
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: serialize fix operations

Allow different threads to fix at the same time could
be dangerous and error-prone now, and most of time
parallel scanning and checking is important.

So this patch adds a mutex to serialize
fix operations during pass1.

And the good benefit of this, we don't need block
allocations and free, superblock updates protection
any more, since only fix operations during pass1
could touch them.

E2fsprogs-commit: 5485120b668c952b6e83f24a1e1a74eaa7bf362e

Change-Id: Iedb64545240cc9325dc0c4ce737a970b27f4f3f6
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: merge quota context after threads finish

Every threads calculate its own quota accounting,
merge them after threads finish.

E2fsprogs-commit: a0af18577fbb960eb20695afbb4af4e23b864909

Change-Id: If9f0e17e560715ed55183f85c443f21ebbce7d40
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: merge context flags properly

e2fsck might restart after pass1, so we should keep
flags if possible, this patch try to fix f_illitable_flexbg failure

E2fsprogs-commit: c36fa1b28bd4171f53ab4f89a90f6b3b8bf80fd2

Change-Id: I6dda1520a2ff6255d244461af75d7609f47ba92b
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: merge dirs_to_hash when threads finish

@dirs_to_hash list need be merged after threads finish,
test covered by t_dangerous.

E2fsprogs-commit: e2db6c19943065909c11af3b03c2904c35e5e4f8

Change-Id: I99b44c40afa22ce8e4530c874b077493cf969e03
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: merge dx_dir_info after threads finish

Merge properly.

E2fsprogs-commit: 45519b8167a00979e86d5a5aa95d0bfb07f72520

Change-Id: Ifaed7a0022b716c42df908327c41aaaaa39aa365
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: merge fs flags when threads finish

merge fs flags properly.

E2fsprogs-commit: d4c0519725d59c9380acc828d24bc070c5bc2c03

Change-Id: I80c773bebc0e321bf2fe3b436c3f0e601b1f1b79
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: merge counts after threads finish

Merge counts properly.

E2fsprogs-commit: 057060ccabf9b5b47afe435d90a1f7f68c03a231

Change-Id: Ieace889f0d1a20a1990424113b0e8fc81f945266
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: add debug codes for multiple threads

These debug codes are added to run the multiple pass1 check
thread one by one in order. If all the codes are correct,
fsck of multiple threads should have exactly the same outcome
with single thread.

E2fsprogs-commit: a8b483a1e7987a6dddc32bc82db06f649d245630

Change-Id: I41b0493a0cd830b76434ff0f86a87c98d8d3ad22
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: merge dblist after thread finishes

Merge dblist properly.

E2fsprogs-commit: f3c2256490297201c63139ba0fea5c6b09bca3cf

Change-Id: Ifd907035ed9762c446c688ccc5af3f3fcaa52d24
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: merge icounts after thread finishes

Merge inode_count and inode_link_info properly after
threads finish.

E2fsprogs-commit: bb78742b5e2e95c3e7e9a76755144fdf8ec12285

Change-Id: If40930a87595010963fd59a54709a4d2b27d223b
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: merge badblocks after thread finishes

Badblocks should be merged properly after threads finish.

E2fsprogs-commit: f39197f53ef5d90cd6952d7e75f2d2e3a02bf7ef

Change-Id: Ieb9a96a5e4438270ba6e939155e2666bb53177e4
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: rbtree bitmap for dir

Only rbtree support merge operation now, use it for bitmaps.

E2fsprogs-commit: 9bab3284e30752bccc36be3e07b9ca4dfd218041

Change-Id: I42aeb009ece78a4baf4369aff3355fc9df51089a
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: merge dir_info after thread finishes

dir_info need be merged after thread finish.

E2fsprogs-commit: fdbf5e226cf931a5713414001b95e4697997a464

Change-Id: I7e33c0035ec8c6bfa7c13d759e3edb468634eae3
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: optimize the inserting of dir_info_db

Binary search is now used when inserting an dir info to the array.
Memmove is now used when moving array. Both of them improves
the performance of inserting.

This patch is also a prepartion for the merging of two dir db
arrays.

E2fsprogs-commit: 13d3c76475d050941d244c85cbb9256ffd980f88

Change-Id: I69041084dbd7e36eefa44744c3fb3737af8e906e
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: do not change global variables

Global variables used in pass1 check are changed to local variables
in this patch. This will avoid conflict between threads.

E2fsprogs-commit: 5a1b22da5ca3e90728cee1dbe6bc1920b245dded

Change-Id: Ifc682874801af15e8e13764832ca569b7da4ac57
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: merge bitmaps after thread completes

A new method merge_bmap has been added to bitmap operations. But
only red-black bitmap has that operation now.

E2fsprogs-commit: ecf85fb474fb7537f14bddd8f85d8372b808d5e9

Change-Id: Id5c387dd813d633adff3afb9208213086c33b6d2
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: print thread log properly

When multi-thread fsck is enabled, logs printed from multiple
threads could overlap with each other. The overlap sometimes
makes the logs unreadable because log_out() is used multiple times
for a single line.

This patch adds leading [Thread XXX] to each logs if multi-thread
is enabed by -m option.

This patch also adds message to show the group ranges and inode
numbers for each thread, which is useful for debuging multi-thread
check.

E2fsprogs-commit: 5b071ed80deca2cdc87585a673076529dd47ca6e

Change-Id: I4bd08bc48813a399ab3a79130ff159b12d22f3a0
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: split groups to different threads

The start/end groups of a thread is calculated according to the
thread number. But still, only one thread is used to check.

E2fsprogs-commit: 562df7468d97042689896cff4b48c6cebc3b8508

Change-Id: I3175b4e6f0423ecd69ed55afea45f1c5919f5c79
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: add start/end group for thread

When multi-threads are used for check, each thread needs to jump
to different group in pass1 check. This patch adds the group
jumping support. But still, only one thread is used to check.

E2fsprogs-commit: f64ff7f53c6fb22c7e4bebe86749b1be31439424

Change-Id: I904bedcbd02bca591665943cff5f2d7a6fa50fbe
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: configure one pfsck thread

This patch creates only one thread to do pass1 check if pthreads are
enabled. The same codes can be used to create multiple threads, but
other functions need to be modified to get ready for that.

E2fsprogs-commit: b2a9a40831dfbb6dacf8bbc819acac2c25ab6980

Change-Id: I3df998a8ecc00d2dc5e959f5a9991a6b65182572
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: create logs for mult-threads

When multi-threads are used, different logs should be created
for different threads. Each thread has log files with suffix
of ".$THREAD_INDEX".

And this patch adds f_multithread_logfile test case.

E2fsprogs-commit: 2643de7d00dc8da9f5be79fab768fca7a27f3745

Change-Id: I86982866b9e8e2d2cc5ac2ec9ac44a3b802c21e0
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: open io-channel when copying fs

This patch also add writethrough flag to the thread io-channel.
When multiple threads write the same disk, we don't want the
data being saved in memory cache. This will be useful in the
future, but even without that flag, the tests can be passed too.

This patch also cleanup the io channel cache of the global
context. Otherwise, after pass1 step, the next steps would use
old data saved in the cache. And the cached data might have
already been overwritten in pass1.

E2fsprogs-commit: f15100003400e5fc01b34c25a7a4b61f5ed76148

Change-Id: I7fcf4038b6cb4eb52808b9a5bafdf909ca2c0c8e
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: copy badblocks when copying fs

This patch copies badblocks when the copying fs.

E2fsprogs-commit: 8095b716e15bb15a47925ae02cb1c06e91fdb651

Change-Id: Ibf5718ad5851a9ccd853a2a1f928bce447aa5de9
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: copy bitmaps when copying context

This patch copies bitmap when the copying context. In the
multi-thread fsck, each thread use different bitmap that copied
from the glboal bitmap. And Bitmaps from multiple threads will
be merged into a global one after the pass1 finishes.

E2fsprogs-commit: 4d47d0e84388bdf2242e09f9ba592ea778c1ba8e

Change-Id: I6a5a2b49b845215a7a735a828709396586520536
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: add assert when copying context

Adding the assert would simplify the copying of context.

E2fsprogs-commit: 464abd60f5b68716d8ac5063d90562f9ecd2b004

Change-Id: I8276c2e90f40b08dea28b157e04df3c0f7099211
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: clear icache when using multi-thread fsck

icache of fs will be rebuilt when needed, so after copying
fs, icache can be inited to NULL.

E2fsprogs-commit: 5d4f09fa148e41b4e0ea01fc364de5bbedf02ecf

Change-Id: I21c58b3f126fd85008d6c732da71b298b2a8b4ff
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: copy fs when using multi-thread fsck

This patch only copy the fs to a new one when -m is enabled.
It doesn't actually start any thread. When pass1 test finishes,
the new fs is copied back to the original context.

This patch handles the fs fields in dblist, inode_map and block_map
properly.

E2fsprogs-commit: d88f9ae76882fc1210158ca3558c0bbad6676a79

Change-Id: Iab763de64e63366bc413d954dd7c8a8af45cb6a2
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: copy context when using multi-thread fsck

This patch only copy the context to a new one when -m is enabled.
It doesn't actually start any thread. When pass1 test finishes,
the new context is copied back to the original context.

Since the signal handler only changes the original context, so
add global_ctx in "struct e2fsck_struct" and use that to check
whether there is any signal of canceling.

This patch handles the long jump properly so that all the existing
tests can be passed even the context has been copied. Otherwise,
test f_expisize_ea_del would fail when aborting.

E2fsprogs-commit: 05a637a9e68d3f7e15323deeab00981d4b7df7e8

Change-Id: I4994e62bdf27c385b02e55de82a9dafcc6a12139
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: add -m option for multithread

-m option is added but no actual functionality is added. This
patch only adds the logic that when -m is specified, one of
-p/-y/-n options should be specified. And when -m is specified,
-C shouldn't be specified and the completion progress report won't
be triggered by sending SIGUSR1/SIGUSR2 signals. This simplifies
the implementation of multi-thread fsck in the future.

Completion progress support with multi-thread fsck will be added
back after multi-thread fsck implementation is finished. Right
now, disable it to simplify the implementation of multi-thread fsck.

E2fsprogs-commit: 842ae5f528f9797e4b2c739daa76884e4349e8f3

Change-Id: I428f4f3b10974b769100bf00169bdd7d8cf86deb
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Saranya Muruganandam <saranyamohan@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

setup-schroot: install the udev and systemd packages separately

On non-Linux Debian ports (e.g., GNU/Hurd and GNU/kFreeBSD) the udev
and systemd packages don't exist.  So try to install them separately,
so they can fail on their own on those platforms.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: support older versions of timeout in r_corrupt_fs

Older versions of the timeout program in coreutils don't support the
-v option.  (This is apparently still in use in the GNU/FreeBSD Debain
port since coreutils hasn't built successfully since Coreutils version
8.28.)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

debian: suppress the no-debian-changes lintian warning

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Update release notes, etc., for the 1.46.5 release

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

mke2fs.conf.5: fix spelling errors in man page

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tst_getsize: use ext2fs_get_device_size2() to support testing large devices

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

po: update zh_CN.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

po: update uk.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

po: update sv.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

po: update sr.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

po: update ms.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

po: update es.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

reisze2fs: sanity check free block group counts when calculating minimum size

If one or more block group descriptor's free blocks count is insane,
it's possible this can lead to a infinite loop in the function
calculate_minimum_resize_size(), which is called by resize2fs -P or
resize2fs -M.

Add some sanity checks to avoid this.  In the case where the file
system is corrupt, this will result in resize2fs -P reporting an
incorrect value, but that's OK, since when we try to do an actual
resize operation, resize2fs requires that the file system be freshly
checked using e2fsck.

https://github.com/tytso/e2fsprogs/issues/94

Fixes: ac94445fc01f ("resize2fs: make minimum size estimates more reliable for mounted fs")
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

setup-schroot: add some additional packages needed to build debian packages

On older Debian systems, "apt-get build-dep e2fsprogs" might not bring
in all of the packages needed to build in the most recent versions of
e2fsprogs.  So explicitly try to install some additional packages
including dh-exec, udev, systemd, and cron.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libuuid: try to use getrandom() or getentropy() if available

If getrandom() or getentropy() is available, use these interfaces in
favor of opening /dev/[u]random.  This avoids a potential TSAN problem
that could potentially cause a fd leak when trying to open
/dev/urandom.  (Which is not a disaster, but these interfaces are more
foolproof and avoids needing to open a file descriptor in a library,
which is a good thing.)

Addresses-Google-Bug: #198050608
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: update the bg_checksum after fixing problems in the bg descriptor

Otherwise, we break the block group descriptor's checksum, and while
this gets fixed by e2fsck, it results unnecessary messages printed or
questions asked of the system administrator.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libext2fs: don't old the CACHE_MTX while doing I/O

A report a deadlock problem caused by I/O errors (caused by e2fsck's
error handler trying to write to a bad block to perform a forced
rewrite) uncovered that we were holding the CACHE_MTX while doing read
operations.  This serialized read operations which destroyed the
performance benefits from doing parallel bitmap loading (or the
parallel e2fsck processing under development).

So restructure the code in unix_read_blk64() so that the read is
always done into the user-provided buffer, and then copied into the
cache afterwards.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: skip sorting extents if there are no valid extents

At the end of a fast commit replay, e2fsck tries merging extents in a
inode. This patch fixes a bug in this logic where we were continuing
this action even if there were no extents to merge resulting in
accessing illegal memory.

Signed-off-by: Harshad Shirwadkar <harshadshirwadkar@gmail.com>

libext2fs: fix missing unlocks in the error path in unix_set_blksize()

https://github.com/tytso/e2fsprogs/pull/83

Signed-off-by: Ryan Cai <ryancaicse@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: Add option to print diff output of failed tests

Add variable $PRINT_FAILED which when set will print the diff output of
a failed test.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

resize2fs: optimize resize2fs_calculate_summary_stats()

Speed up an off-line resize of a 10GB file system to 64TB located on
tmpfs from 90 seconds to 16 seconds by extracting block group bitmaps
using a population count function to count the blocks in use instead
checking each bit in the block bitmap.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

resize2fs: adjust new size of the file system to allow a successful resize

The previous commit in this series (commit 50088b1996cc: "resize2fs:
attempt to keep the # of inodes valid by removing the last bg") allows
a successful off-line resize of a file system with the default 16k
inode ratio to be grown to support a 64TB storage device by dropping
the last block group so the number of inodes is just below the maximum
2**32-1 number of inodes.

However, this is not a complete solution, for two reasons.  First,
this adjustment happens after resize2fs has started potentially making
changes to the file system in the off-line (unmounted) case, which
means resize2fs will do a lot of unnecessary work.  Secondly, in the
on-line resize case, passing the original requested size to the kernel
causes the kernel fail the online resize request.

So teach resize2fs to adjust the new size of the file system much
earlier, which avoids both problems.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Leah Rumancik <leah.rumancik@gmail.com>

resize2fs: attempt to keep the # of inodes valid by removing the last bg

If a the 10GB file system (with the default inode ratio size of 16k)
is resized to 64TB, the number of inodes will become 2**32 --- one
above the maximum allowed number of inodes of 2**32-1.  In
adjust_fs_info(), we already try drop the last block group if there
isn't sufficient space in the last block group to support the metadata
for that block group.  So if dropping the last block group allows the
number of inodes to valid, we should try that as well.  In some cases
this will mean resizing a file system to 64TB will result in it be
resized to a size of 64TB - 128MB, which is close enough for
government work.

Addresses-Google-Bug: 199105099
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Leah Rumancik <leah.rumancik@gmail.com>

tests: update expect files for f_large_dir and f_large_dir_csum

Update expect files for f_large_dir and f_large_dir_csum tests to
include the warning about missing y2038 support with 128-byte inodes.

Fixes: a23b50cd ("mke2fs: warn about missing y2038 support when formatting fresh ext4 fs")
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: update expect files for f_mmp_garbage

Update expect file for f_mmp_garbage test to work correctly with the
new default 256 inode size.

Fixes: d730be5ceeba ("tests: update mke2fs.conf to create 256 byte inodes by default")
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

quota: Drop dead code

Drop unused function from quota support code.

Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

debugfs: Fix headers for quota commands

list_quota and get_quota commands have 'blocks' header while what they
actually show is a used space in bytes. Fix the header to state 'space'
instead.

Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: Expand test checking quota and orphan processing interaction

Expand f_orphquot test to also check handling of quotas for non-root
user and verify that quota limits are properly preserved over orphan
replay.

Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: Do not trash user limits when processing orphan list

When e2fsck was loading quotas to process orphan list, it was loading
only quota usage. However subsequent quota writeout has effectively
overwritten quota limits, loosing them forever. Make sure quota limits
are preserved over orphan replay.

Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tune2fs: Fix conversion of quota files

When tune2fs is enabling quota feature, it looks for old-style quota
files and tries to transfer limits stored in these files into newly
created hidded quota files. However the code doing the transfer setups
the quota scan wrongly and instead of transferring limits we transfer
usage. So not only quota limits are lost (at least they can still be
recovered from the old quota files) but also usage information may be
wrong if the accounting in e2fsprogs does not exactly match the
accounting in quota-tools (which is actually the case). Fix the setup of
the quota scan.

Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

quota: Rename quota_update_limits() to quota_read_all_dquots()

quota_update_limits() is a misnomer because what it actually does is
that it updates 'usage' counters and leaves 'limit' counters intact.
Rename quota_update_limits() to quota_read_all_dquots() and while
changing prototype also add a flags argument so that callers can control
which quota information is actually updated from the disk.

Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

quota: Fold quota_read_all_dquots() into quota_update_limits()

There's just one caller of quota_read_all_dquots(), fold it into its
caller quota_update_limits(). No functional changes.

Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

quota: Add support to version 0 quota format

Version 0 quota format differs from version 1 by having only 32-bit
counters for inodes and block limits. For many installations this is not
limiting and thus the format is widely used. Also quota tools still
create quota files with this format by default. Add support for this
quota format to e2fsprogs so that we can seamlessly convert quota files
in this format into our internal quota files.

Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: update expect file for u_direct_io

The u_direct_io test is normally not run (since it requires root
privileges); as a result, when the mke2fs.conf defaults were changed,
I didn't notice that the expected output for u_direct_io test needed
to be updated.

Fixes: d730be5ceeba ("tests: update mke2fs.conf to create 256 byte inodes by default"
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libss: add newer libreadline.so.8 to dlopen path

OpenSUSE Tumbleweed now has libreadline.so.8. Add it to the list of libs
to look for.

Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

debian/changelog: add missing section for 1.46.2-2

E2fsprogs 1.46.2-2 was uploaded during the release freeze, so the
changelog was assembled outside of the normal maint branch.  Add it
now.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

fix unused-function -Wall warnings

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

debugfs: fix shadow and sign-compare -Wall warnings

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Update release notes, etc., for the 1.46.4 release

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libcom_err: fix suggest-attribute=format -Wall warnings

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libss: fix fallthrough -Wall warnings

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: fix shadow -Wall warnings in the crcsum test program

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: fix "format not a string literal" -Wall warning in test_icount

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libe2p: fix missing-prototypes and sign-compare -Wall warnings

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

debugfs: fix missing byte swap when dumping a revoke block

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

debian: switch to using build dependency on debhelper-compat

This is preferred in favor of using the debhelper/compat file, and we
no longer worry about supporting Debian Jessie or Debian Stretch
(at least without Stretch Backports).

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Update release notes, etc., for the 1.46.4-rc1 release

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

debian: set Rules-Requires-Root to "no".

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Fix miscellaneous spelling errors in man pages, and release notes

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Change "filesystem" to "file system" in the man pages

To improve consistency, use "file system" in all of the man pages in
preference over "filesystem".

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libsupport: fix sort_r.h to work on FreeBSD

FreeBSD defines __GNU_SOURCE so this is not reliable marker that the
OS is using a glibc-style qsort_r(3).

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

mke2fs: allow the default creator os to be specified in /etc/mke2fs.conf

This is useful for keeping the regression test results consistent
when run on non-Linux systems, especially on GNU Hurd.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

config: update config.{guess,sub}

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

resize2fs.8.in: clarify the description of the progress bar option

Addresses-Debian-Bug: #979411

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2scrub: use WantedBy=multi-user.target in e2scrub_reap.service

Addresses-Debian-Bug: #991349
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

po: update sv.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

po: update pt.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

po: update pl.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

po: update nl.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

po: update fr.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

po: update cs.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: update mke2fs.conf to create 256 byte inodes by default

The regression tests have their own private copy of mke2fs which is
used when tests create file systems.  Since we are now using 256 byte
inodes by default, the tests should reflect this.

While we're at it, modify the r_move_itable test so it actually tests
moving the inode table.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

mke2fs: warn that bigalloc is experimental only for large cluster sizes

Since we have done a lot of testing with a cluster size equal to 64k
(or 16 times the default 4k block size), mke2fs will only warn for
bigalloc file systems where the cluster size is greater than 16 times
the block size.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

mke2fs: warn about missing y2038 support when formatting fresh ext4 fs

Filesystems with 128-byte inodes do not support timestamps beyond the
year 2038.  Since we're now less than 16.5 years away from that point,
it's time to start warning users about this lack of support when they
format an ext4 filesystem with small inodes.

(Note that even for ext2 and ext3, we changed the default for
non-small file systems in 2008 in commit commit b1631cce648e ("Create
new filesystems with 256-byte inodes by default").)

So change the mke2fs.conf file to specify 256-byte inodes even for
small filesystems, and then add a warning to mke2fs itself if someone
is trying to make us format a file system with 128-byte inodes.  This
can be suppressed by setting the boolean option warn_y2038_dates in
the mke2fs.conf file to false, which we do in the case of GNU Hurd,
since it only supports 128 byte inodes as of this writing.

[ Patch reworked by tytso to only warn in the case of GNU Hurd, since
  the default for ext2/ext3 was changed for all but small file systems
  in 2008. ]

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: make sure quota files are not referenced from dirs

Quota files must not be referenced from directory entries. Otherwise
they can get corrupted under the hands of the kernel.

Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tests: check quota file space usage does not get accounted

Check that space used by quota files themselves does not get accounted
into the space tracked by quota subsystem.

Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

tst_bitops: don't stop testing on low memory

Some build systems have 512 MiB memory or less, e.g. Ubiquity Edgeroute
Lite which can be used to build for the mips and mips64 architectures.

On these allocating 512 MIB will always fail. So if allocating 512 MiB
fails treat this as expected behavior and return 0.

This will allow testing to continue with the remaining tests.

Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: add maximum string length specifiers to fscanf format strings

When parsing strings from /proc/apm and /proc/acpi/ac_adapter, add
string length limits to prevent possible buffer overruns.

Addresses-Coverty-Bug: 1297496
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

libext2fs: fix coverity nits in tdb.c

Address unchecked returned value and a string not null terminated warnings.

Addresses-Coverity-Bug: 709473
Addresses-Coverity-Bug: 709474
Addresses-Coverity-Bug: 1464578
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

e2fsck: clean up two gcc -Wall warnings in recovery.c

Fix a signed vs unsigned and a void * pointer arithmetic warning.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>