Whamcloud - gitweb
tools/e2fsprogs.git
7 years agotests: report if a test is taking a long time
Andreas Dilger [Wed, 30 Aug 2017 06:05:25 +0000 (02:05 -0400)]
tests: report if a test is taking a long time

Print out a message if a test takes longer than 60s, with a
reminder to potentially add is_slow_test to that test, so
that it is easier to find which tests are taking a long time,
especially if running with "make -j check" or similar.

Add an exclusion for r_expand_full on MacOS since HFS+ does
not support sparse devices and doesn't like large test images.

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agotune2fs: quiet llvm build warning
Andreas Dilger [Wed, 30 Aug 2017 05:56:08 +0000 (01:56 -0400)]
tune2fs: quiet llvm build warning

Quiet a relatively harmless build warning:

tune2fs.c:928:18: warning: '&&' within '||' [-Wlogical-op-parentheses]
if (pass == 1 && (inode->i_flags & EXT4_EA_INODE_FL) ||
    ~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~
tune2fs.c:928:18: place parentheses around '&&' to silence warning

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoMerge branch 'maint' into next
Theodore Ts'o [Tue, 29 Aug 2017 12:03:31 +0000 (08:03 -0400)]
Merge branch 'maint' into next

7 years agoUpdate release notes, etc., for the 1.43.6 release
Theodore Ts'o [Wed, 23 Aug 2017 23:46:04 +0000 (19:46 -0400)]
Update release notes, etc., for the 1.43.6 release

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoSearch for GNU-compatible dd for self-tests.
Matthias Andree [Tue, 29 Aug 2017 05:02:36 +0000 (01:02 -0400)]
Search for GNU-compatible dd for self-tests.

This checks for a dd that supports iflag=fullblock oflag=append,
and looks at gdd and dd for now, and warns of failing self-tests
if neither supports these two flags.

Signed-off-by: Matthias Andree <matthias.andree@gmx.de>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoFix FreeBSD portability problem caused by it using character mode disk devices
Theodore Ts'o [Mon, 28 Aug 2017 22:04:11 +0000 (18:04 -0400)]
Fix FreeBSD portability problem caused by it using character mode disk devices

We were using S_ISBLK() to test if a device could be used as a disk
device.  This doesn't work for FreeBSD.  We need to test for S_ISBLK()
|| S_ISCHR().

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agopo: update pl.po (from translationproject.org)
Jakub Bogusz [Mon, 28 Aug 2017 20:27:29 +0000 (16:27 -0400)]
po: update pl.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoSilence valgrind warnings
Theodore Ts'o [Sat, 26 Aug 2017 17:42:30 +0000 (13:42 -0400)]
Silence valgrind warnings

Valgrind doesn't understand that the kernel will be initializing the
struct termios and struct loop_info64 structures.  Since they occur in
functions which are not in the hot path, preinitialize to zero to
prevent valgrind from producing a huge number of false positives.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: add optimization for heavily hard-linked file systems
Jaco Kroon [Wed, 23 Aug 2017 18:21:43 +0000 (14:21 -0400)]
e2fsck: add optimization for heavily hard-linked file systems

In the case of file system with large number of hard links, e2fsck can
take a large amount of time in pass 2 due to binary search lookup of
inode numbers.  This implements a memory trade-off (storing 2 bytes
in-memory for each inode to store inode counts).

For a 40TB filesystem with 2.8bn inodes this map alone requires 5.7GB
of RAM.  For this reason, we don't enable this optimization by
default.  It can be enabled using either an extended option to e2fsck
or via a seting in e2fsck.conf.

Even when the fullmap optimization is enabled, we don't use this for
the icount structure in pass 1.  This is because the gain CPU gain is
nearly nil for that pass and the sacrificed memory does not justify
the increase in RAM.

(It could be that during pass 1, if more than 17% if possible inodes
has link_count>1 (466m inodes in the 40TB with 2.8bn possible inodes
case) then it becomes more memory efficient to use the full map
implementation in terms of memory.  However, this is extremely
unlikely given that most file systems are heavily over-provisioned in
terms of the number of inodes in the system.)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: optimize out the use region_t in scan_extent_node()
Jaco Kroon [Wed, 23 Aug 2017 17:54:25 +0000 (13:54 -0400)]
e2fsck: optimize out the use region_t in scan_extent_node()

Since extents have a guarantee of being monotonically increasing we
merely need to check that block n+1 starts after block n.  This is a
simple enough check and we can perform this by calculating the next
expected logical block instead of using the region usage tracking data
abstraction.

Signed-off-by: Jaco Kroon <jaco@uls.co.za>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agotune2fs, mke2fs: clarify proceed delay question
Theodore Ts'o [Wed, 23 Aug 2017 15:03:37 +0000 (11:03 -0400)]
tune2fs, mke2fs: clarify proceed delay question

The explanation of the what happens after delaying N seconds is
ambiguous.  Clarify what happens.

Addresses-Debian-Bug: #857336

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agotune2fs: explain why an fsck is needed
Theodore Ts'o [Wed, 23 Aug 2017 14:57:25 +0000 (10:57 -0400)]
tune2fs: explain why an fsck is needed

Currently tune2fs just says without any explanation, "run fsck -f".
Add a short explanation that a freshly checked file system is required
to reduce user confusion.  (We could add even more details, but
hopefully this is enough.)

Addresses-Debian-Bug: #857336

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agotune2fs: do not enable project feature or quota if inode size is 128 bytes
Theodore Ts'o [Wed, 23 Aug 2017 14:44:44 +0000 (10:44 -0400)]
tune2fs: do not enable project feature or quota if inode size is 128 bytes

Don't allow the user to enable the project feature (or project quota)
if the inode size is 128 bytes.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agomke2fs: automatically use 256 byte inodes if project feature enabled
Theodore Ts'o [Wed, 23 Aug 2017 14:30:09 +0000 (10:30 -0400)]
mke2fs: automatically use 256 byte inodes if project feature enabled

If the inode size is not implicitly requested on the command line, and
it is too small to support the project feature, automatically promote
the inode size to be 256 bytes so that the project feature will work.

Note the previous test to check for a too-small inode size didn't work
because it checked before inode size was set in fs_param.  Hence, it
was possible to create file systems with a 128 byte inode and the
project feature enabled.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agomke2fs.conf: remove legacy entry for ext4dev
Theodore Ts'o [Wed, 23 Aug 2017 14:04:54 +0000 (10:04 -0400)]
mke2fs.conf: remove legacy entry for ext4dev

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agolibext2fs: fix Large File Support bug in ext2fs_image_inode_write()
Theodore Ts'o [Tue, 22 Aug 2017 21:09:31 +0000 (17:09 -0400)]
libext2fs: fix Large File Support bug in ext2fs_image_inode_write()

We were using lseek() instead of ext2fs_llseek().  This caused e2image
to fail when writing an old-style e2image file if it was larger than
2 GiB.

Addresses-Debian-Bug: #855246

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: fix the message printed for the project quota type
Theodore Ts'o [Tue, 22 Aug 2017 19:40:23 +0000 (15:40 -0400)]
e2fsck: fix the message printed for the project quota type

Fix a cut and paste error so print_e2fsck_message() correctly prints
the quota type for project quotas.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reported-by: Jakub Bogusz <qboosh@pld-linux.org>
7 years agoMerge branch 'maint' into next
Theodore Ts'o [Tue, 22 Aug 2017 16:19:30 +0000 (12:19 -0400)]
Merge branch 'maint' into next

7 years agodebian: remove support for pre-multiarch versions of Debian
Theodore Ts'o [Tue, 22 Aug 2017 16:15:26 +0000 (12:15 -0400)]
debian: remove support for pre-multiarch versions of Debian

All versions of Debian after Wheezy support Multiarch, so we can
simply the Debian control.in and rules file by removing support for
older versions of Debian without Multiarch support.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agolibext2fs: avoid potential out-of-bounds write if pread/pread64 fails
Theodore Ts'o [Tue, 22 Aug 2017 15:23:21 +0000 (11:23 -0400)]
libext2fs: avoid potential out-of-bounds write if pread/pread64 fails

In unix_io.c's raw_read_block(), if the initial attempt to call
pread/pread64 fails because the offset is insane, the variable
"actual" is left at -1, and then when lseek fails, the cleanup
function will try to clear (as an out-of-bounds write) a single byte
before the buffer.  Fix this.

Addresses-Debian-Bug: #871539

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reported-by: Jakub Wilk <jwilk@jwilk.net>
7 years agodebian: remove support libuuid/libblkid packages
Theodore Ts'o [Tue, 22 Aug 2017 14:37:10 +0000 (10:37 -0400)]
debian: remove support libuuid/libblkid packages

Remove support for util-linux prior to 2.16, when e2fsprogs provided
its own copy of libuuid and libblkid.  This is only needed for Debian
distributions prior to Wheezy, which is no longer supported.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agolibsupport: don't try accessing the project quota for 128 byte inodes
Theodore Ts'o [Tue, 22 Aug 2017 04:54:15 +0000 (00:54 -0400)]
libsupport: don't try accessing the project quota for 128 byte inodes

If the file system has 128 byte inode, it's not possible to access its
project quota id, since the inode is too small.  So prevent a
potential out-of-bounds read in get_qid().

The problem was found by valgrind and American Fuzzy Lop.

Address-Debian-Bug: #871540

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agodebian: break out locale files into a separate package
Theodore Ts'o [Tue, 22 Aug 2017 03:41:35 +0000 (23:41 -0400)]
debian: break out locale files into a separate package

Since e2fsprogs is a required/essential package, this allows us to
significantly shrink the size of the a minbase setup.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agopo: update vi.po (from translationproject.org)
Trần Ngọc Quân [Tue, 22 Aug 2017 01:45:28 +0000 (21:45 -0400)]
po: update vi.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agopo: update sv.po (from translationproject.org)
Göran Uddeborg [Tue, 22 Aug 2017 01:45:28 +0000 (21:45 -0400)]
po: update sv.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agopo: update fr.po (from translationproject.org)
Samuel Thibault [Tue, 22 Aug 2017 01:45:27 +0000 (21:45 -0400)]
po: update fr.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agopo: update cs.po (from translationproject.org)
Petr Pisar [Tue, 22 Aug 2017 01:45:27 +0000 (21:45 -0400)]
po: update cs.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoRevert "mke2fs: fix UI problem caused by fuzzy translations"
Theodore Ts'o [Tue, 22 Aug 2017 01:32:26 +0000 (21:32 -0400)]
Revert "mke2fs: fix UI problem caused by fuzzy translations"

This reverts commit f25b334c5093a1a5e2985c7ed58213a52ba06329.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: in ask_yn() fall back to English yes/no characters
Theodore Ts'o [Tue, 22 Aug 2017 01:20:38 +0000 (21:20 -0400)]
e2fsck: in ask_yn() fall back to English yes/no characters

In the case of missing translations, if the translation for y/n is
missing due to fuzzy transactions, such that user is told to use
<y/n>, those characters will work correctly.

Addresses-Debian-Bug: #856586

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: add optimization for large, fragmented sparse files
Theodore Ts'o [Mon, 14 Aug 2017 23:52:39 +0000 (19:52 -0400)]
e2fsck: add optimization for large, fragmented sparse files

The code which checks for overlapping logical blocks in an extent tree
is O(h*e) in time, where h is the number of holes in the file, and e
is the number of extents in the file.  So a file with a large number
of holes can take e2fsck a long time process.  Optimize this taking
advantage of the fact the vast majority of the time, region_allocate()
is called with increasing logical block numbers, so we are almost
always append onto the end of the region list.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoMerge branch 'maint' into next
Theodore Ts'o [Mon, 14 Aug 2017 02:40:15 +0000 (22:40 -0400)]
Merge branch 'maint' into next

7 years agomke2fs: fix UI problem caused by fuzzy translations
Theodore Ts'o [Mon, 14 Aug 2017 01:07:21 +0000 (21:07 -0400)]
mke2fs: fix UI problem caused by fuzzy translations

When the original message was changed from "(y, n)" to "(y, N)", this
caused the translations to be marked as "fuzzy".  For those
translations that use a different characters for yes and no --- for
example, German, which uses j and n for "ja" and "nein" --- not having
the translation can cause user confusion since the user will type 'y',
and it will be interpreted as "No", since mke2fs is expecting that the
user will type some other character, such as 'j' or 'J' for "Ja" in
the German locale.

Addresses-Debian-Bug: #856586

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoAdd options to gzip and tar to create reproducible artificats
Theodore Ts'o [Sun, 13 Aug 2017 23:54:30 +0000 (19:54 -0400)]
Add options to gzip and tar to create reproducible artificats

Also remove some long-unused rules from resize2fs's Makefile.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agotests: disable use of the BLKID cache while running regression tests
Theodore Ts'o [Sun, 13 Aug 2017 21:52:20 +0000 (17:52 -0400)]
tests: disable use of the BLKID cache while running regression tests

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agolibsupport: fix 32-bit quota test failures
Theodore Ts'o [Sun, 13 Aug 2017 18:45:27 +0000 (14:45 -0400)]
libsupport: fix 32-bit quota test failures

On 32-bit platform some of the util_dqblk structures have a type of
long long.  So we need to use %lld and casts to make sure the right
thing happens on both 32-bit and 64-bit platforms.

Fixes: 968c1c2fe472ddc4a467aaff71305e83cdb0308f
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoMerge branch 'maint' into next
Theodore Ts'o [Fri, 4 Aug 2017 16:30:01 +0000 (12:30 -0400)]
Merge branch 'maint' into next

7 years agoUpdate release notes, etc., for the 1.43.5 release v1.43.5
Theodore Ts'o [Fri, 4 Aug 2017 16:20:53 +0000 (12:20 -0400)]
Update release notes, etc., for the 1.43.5 release

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoAdd new symbol to debian/e2fslibs.symbols
Theodore Ts'o [Fri, 4 Aug 2017 16:19:47 +0000 (12:19 -0400)]
Add new symbol to debian/e2fslibs.symbols

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agodebian: update policy conformance to standards version 4.0
Theodore Ts'o [Fri, 4 Aug 2017 15:27:15 +0000 (11:27 -0400)]
debian: update policy conformance to standards version 4.0

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agodebian/rules: update config.{guess,sub} before running configure, not after
Theodore Ts'o [Fri, 4 Aug 2017 15:26:39 +0000 (11:26 -0400)]
debian/rules: update config.{guess,sub} before running configure, not after

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoRemove special mips libraries from Debian build
Theodore Ts'o [Fri, 4 Aug 2017 06:01:43 +0000 (02:01 -0400)]
Remove special mips libraries from Debian build

These libraries were needed to support arcboot, which is obsolete and
no longer part of Debian.  So drop these non-standard, legacy special
libraries that were only built on the mips platform.

Addresses-Debian-Bug: #864144

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoRemove obsolete Conflicts/Replaces headers in Debian control file
Theodore Ts'o [Fri, 4 Aug 2017 05:54:55 +0000 (01:54 -0400)]
Remove obsolete Conflicts/Replaces headers in Debian control file

Addresses-Debian-Bug: #866623

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoRemove spaces at the end of man pages
Theodore Ts'o [Fri, 4 Aug 2017 05:51:50 +0000 (01:51 -0400)]
Remove spaces at the end of man pages

Addresses-Debian-Bug: #865584

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoClarify description of the the commit mount option in the ext4 man page
Theodore Ts'o [Fri, 4 Aug 2017 05:47:29 +0000 (01:47 -0400)]
Clarify description of the the commit mount option in the ext4 man page

Addresses-Debian-Bug: #867895

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoUpdate translation template for 1.43.5 release
Theodore Ts'o [Fri, 4 Aug 2017 05:30:54 +0000 (01:30 -0400)]
Update translation template for 1.43.5 release

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoUpdate lib/e2p/Makefile.in using "make depend"
Theodore Ts'o [Fri, 4 Aug 2017 05:27:48 +0000 (01:27 -0400)]
Update lib/e2p/Makefile.in using "make depend"

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agopo: update vi.po (from translationproject.org)
Trần Ngọc Quân [Fri, 4 Aug 2017 05:08:46 +0000 (01:08 -0400)]
po: update vi.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agopo: update sr.po (from translationproject.org)
Мирослав Николић [Fri, 4 Aug 2017 05:08:46 +0000 (01:08 -0400)]
po: update sr.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agopo: update hu.po (from translationproject.org)
Balázs Úr [Fri, 4 Aug 2017 05:08:46 +0000 (01:08 -0400)]
po: update hu.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agopo: update da.po (from translationproject.org)
Joe Hansen [Fri, 4 Aug 2017 05:08:46 +0000 (01:08 -0400)]
po: update da.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agopo: update cs.po (from translationproject.org)
Petr Pisar [Fri, 4 Aug 2017 05:08:46 +0000 (01:08 -0400)]
po: update cs.po (from translationproject.org)

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: fix e2fsck -D for encrypted directories
Theodore Ts'o [Tue, 1 Aug 2017 14:26:11 +0000 (10:26 -0400)]
e2fsck: fix e2fsck -D for encrypted directories

If the directory entry is encrypted there may be embedded NUL
characters; hence, we should use memcmp instead of strncmp when
comparing strings.  Otherwise, e2fsck can erroneously report that a
directory have duplicant entries when doing an e2fsck -D check.

Reported-by: Sahitya Tummala <stummala@codeaurora.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoMerge branch 'maint' into next
Theodore Ts'o [Fri, 28 Jul 2017 00:07:22 +0000 (20:07 -0400)]
Merge branch 'maint' into next

7 years agoFix build when configured with --disable-threads
Theodore Ts'o [Thu, 27 Jul 2017 23:43:00 +0000 (19:43 -0400)]
Fix build when configured with --disable-threads

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoFix build when configured with --enable-profile
Theodore Ts'o [Thu, 27 Jul 2017 23:42:23 +0000 (19:42 -0400)]
Fix build when configured with --enable-profile

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agolibext2fs: fix error handling for ext2fs_sync_device()
Theodore Ts'o [Thu, 27 Jul 2017 23:41:22 +0000 (19:41 -0400)]
libext2fs: fix error handling for ext2fs_sync_device()

Return the proper error code instead of -1.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoFurther cleanups of the dir_link documentation in the ext4 man page
Theodore Ts'o [Thu, 27 Jul 2017 22:31:04 +0000 (18:31 -0400)]
Further cleanups of the dir_link documentation in the ext4 man page

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agolibsupport: fix error handling in quota_write_inode
zhangyi (F) [Mon, 24 Jul 2017 07:01:26 +0000 (15:01 +0800)]
libsupport: fix error handling in quota_write_inode

The error return value of quota_file_create() is no longer < 0,
and the error handling in quota_write_inode() is incorrect,
fix these. This also fix a tune2fs segfault that currently
occurs when we add project and quota features to an inode
exhaustion ext4 filesystem.

Fixes: a701823a3150("libsupport: fix gcc -Wall nits")

Signed-off-by: zhangyi (F) <yi.zhang@huawei.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agodebugfs: fix "ls -p" to avoid printing garbage after the file name
Theodore Ts'o [Mon, 24 Jul 2017 17:11:16 +0000 (13:11 -0400)]
debugfs: fix "ls -p" to avoid printing garbage after the file name

In commit 68a1de3df3 (debugfs: pretty print encrypted filenames in the
ls command), a change was introduced in debugfs/ls.c which instead of
copying dirent->name and 0-terminating it, dirent->name is used
directly in printf.

However, instead of using the precision to limit the number of
characters output, the code uses the field width. As a result,
characters are output until a 0 is read, which results in garbage
after the file name.

Also fix two other instances of this in debugging messages that aren't
used, but fixing them will avoid potential future copypasta bugs.

Reported-by: Christian Gabriel <ch_gabriel@web.de>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: eliminate bad update link count attempt
Tahsin Erdogan [Mon, 24 Jul 2017 04:11:30 +0000 (00:11 -0400)]
e2fsck: eliminate bad update link count attempt

Extended attribute inodes have a link count of 1 but they are not
attached to any directories. When an xattr inode with zero ea
references is found, the remedy is to reconnect it to lost+found dir.
Since reconnect operation increments the link count, it would normally
become 2 but to avoid that, check_ea_inode() sets the link count to
zero in anticipation of reconnect operation. And it does it even when
e2fsck is invoked with -n option which causes a fatal e2fsck failure
as can be demonstrated with the following test script:

  mke2fs -b 4096 -I 256 -F -O extent,ea_inode /dev/sdb 1G
  mount /dev/sdb /mnt/sdb
  touch /mnt/sdb/x
  setfattr -n user.ie1 -v $(python -c "print 'v'*5000") /mnt/sdb/x
  umount /dev/sdb
  debugfs -w -R 'rm x' /dev/sdb
  e2fsck -f -n /dev/sdb

In this case, e2fsck fails with the following error:

  e2fsck 1.43.5-WIP (17-Feb-2017)
  Pass 1: Checking inodes, blocks, and sizes
  Pass 2: Checking directory structure
  Pass 3: Checking directory connectivity
  Pass 4: Checking reference counts
  ext2fs_write_inode: Attempt to write to filesystem opened read-only
  while writing inode 13 in check_ea_inode
  e2fsck: aborted

Setting link count to zero is not really required because e2fsck will
detect the discrepancy in link count and offer to fix it.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoresize2fs: add support for resizing filesystems with ea_inode feature
Tahsin Erdogan [Mon, 24 Jul 2017 03:42:43 +0000 (23:42 -0400)]
resize2fs: add support for resizing filesystems with ea_inode feature

Resizing filesystems with ea_inode feature was disallowed so far
because the code for updating the ea entries was missing. This patch
adds that support.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agolibext2fs: add ea_inode support to set xattr
Tahsin Erdogan [Mon, 24 Jul 2017 02:54:58 +0000 (22:54 -0400)]
libext2fs: add ea_inode support to set xattr

This patch is a major update to how we decide where to put extended
attributes.  The main motivation is to enable creating values in
extended attribute inodes.  While doing this, we want to implement a
behavior that is as close to kernel as possible.

Existing set ea code deviates from kernel behavior which makes it harder
to implement ea_inode feature:

 - kernel only sorts ea entries in xattr block, e2fsprogs implementation
   sorts all entries on every update.

 - e2fsprogs implementation shuffled things on every update so the order
   of updates does not matter. Kernel does not reshuffle things.

 - e2fsprogs could evacuate entries from inode body to xattr block and
   vice versa. This behavior does not exist in kernel.

Such differences could lead to inconsistent behavior between fuse2fs and
a kernel mount.

With ea_inode feature, we also need to decide whether to put a value
in an inode or keep it 'inline'.  In kernel implementation this
depends on current placement of entries.

To close the behavioral gap, ext2fs_xattr_set() now takes over the
decision about where to place ea values.  This also allows it to raise
errors early instead of delaying them to a separate
ext2fs_xattrs_write() call later.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agolibext2fs: eliminate empty element holes in ext2_xattr_handle->attrs
Tahsin Erdogan [Mon, 24 Jul 2017 02:47:05 +0000 (22:47 -0400)]
libext2fs: eliminate empty element holes in ext2_xattr_handle->attrs

When an extended attribute is removed, its array element is emptied.
This creates holes in the array so various places that want to walk
filled elements have to do an empty element check.

Have remove operation shift remaining filled elements to the left.
This allows a simple iteration up to ext2_xattr_handle->count to walk
all filled entries, and so empty element checks become unnecessary.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agolibext2fs: rename ext2_xattr_handle->length to capacity
Tahsin Erdogan [Mon, 24 Jul 2017 01:25:32 +0000 (21:25 -0400)]
libext2fs: rename ext2_xattr_handle->length to capacity

ext2_xattr_handle has two fields 'count' and 'length' which
represent number of filled elements vs total element count.
They have close meanings so are easy to confuse, thus make code less
readable. Rename length to capacity.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: update quota inode accounting for ea_inode feature
Tahsin Erdogan [Mon, 24 Jul 2017 01:20:41 +0000 (21:20 -0400)]
e2fsck: update quota inode accounting for ea_inode feature

Extended attribute inodes are charged to all referencing inodes.
Update e2fsck so that it can correctly track inode quota charges.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoClarify how the description of the dir_nlink feature in the ext4 man page
Theodore Ts'o [Sun, 23 Jul 2017 22:51:22 +0000 (18:51 -0400)]
Clarify how the description of the dir_nlink feature in the ext4 man page

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agotune2fs: edit dire warning about check intervals
Eric Sandeen [Sun, 23 Jul 2017 22:34:57 +0000 (18:34 -0400)]
tune2fs: edit dire warning about check intervals

Time & mount-count based checks have been off by default for quite some
time now, but the dire warning about disabling them remains in the
tune2fs manpage, which is confusing.  We did "strongly consider
the consequences" and disabled it by default, no need to scare the
user about it now.  Inform the user of the consequences in a more
measured tone.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoresize2fs: sanity check the free blocks and inode counts
Theodore Ts'o [Sun, 23 Jul 2017 04:46:36 +0000 (00:46 -0400)]
resize2fs: sanity check the free blocks and inode counts

If the free block or free inodes count are larger than the number of
blocks or inodes in the system, request that the file system be
checked.  Otherwise it's possible for calcuate_minimum_resize_size()
to hang in an infinite loop.

This problem was found using American Fuzzy Lop.

Reported-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agodebugfs: add sanity checking to the string_to_inode() utility function
Theodore Ts'o [Sun, 23 Jul 2017 04:45:05 +0000 (00:45 -0400)]
debugfs: add sanity checking to the string_to_inode() utility function

Otherwise it's possible for a corrupt file system or bad user input to
cause debugfs to crash if the resulting inode number is insanely
large.

This problem was found using American Fuzzy Lop.

Reported-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agolibext2fs: add stricter checks on the inode size in ext2fs_open2()
Theodore Ts'o [Sun, 23 Jul 2017 04:26:44 +0000 (00:26 -0400)]
libext2fs: add stricter checks on the inode size in ext2fs_open2()

An inode size larger than the block size can cause userspace programs
to crash.

This problem was found using American Fuzzy Lop.

Reported-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: don't restart after a critical metadata collision in e2fsck -n mode
Theodore Ts'o [Sun, 23 Jul 2017 04:08:18 +0000 (00:08 -0400)]
e2fsck: don't restart after a critical metadata collision in e2fsck -n mode

If the file system isn't going to be changed, there's no point
restarting; it will just cause e2fsck to loop forever.

This problem was found using American Fuzzy Lop.

Reported-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: quiet some harmless bitmap warnings
Theodore Ts'o [Sun, 23 Jul 2017 02:50:22 +0000 (22:50 -0400)]
e2fsck: quiet some harmless bitmap warnings

E2fsck checks block numbers against the block_metadata_map before it
checks to see whether or not the block numbers are valid.  So suppress
these harmless warnings.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agotests: dumpe2fs against an image with bad os_type
Adam Buchbinder [Sat, 22 Jul 2017 21:52:53 +0000 (17:52 -0400)]
tests: dumpe2fs against an image with bad os_type

The os_type here is large enough to be negative when interpreted as a
signed integer.

This test case was generated by american fuzzy lop, starting from a
base filesystem image from files.fuzzing-project.org.

Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2p: fix segfault in e2p_os2string
Adam Buchbinder [Sat, 22 Jul 2017 21:52:24 +0000 (17:52 -0400)]
e2p: fix segfault in e2p_os2string

Passing in a negative integer would lead to a segfault, and
a crafted filesystem image could trigger that.

Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: check for invalid quota inode numbers
Theodore Ts'o [Sat, 22 Jul 2017 20:08:25 +0000 (16:08 -0400)]
e2fsck: check for invalid quota inode numbers

If the superblock has invalid inode numbers for the user, group, or
project quota inodes, e2fsck should notice and offer to fix things by
zeroing out the invalid superblock field.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: add support for printing the quota type in problem reports using %U
Theodore Ts'o [Sat, 22 Jul 2017 18:51:28 +0000 (14:51 -0400)]
e2fsck: add support for printing the quota type in problem reports using %U

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agolibext2fs: fix the s_log_block_size check in ext2fs_open()
Theodore Ts'o [Mon, 17 Jul 2017 23:55:39 +0000 (19:55 -0400)]
libext2fs: fix the s_log_block_size check in ext2fs_open()

The s_log_block_check can fail to detect an invalid value if it is
between UINT_MAX-9 and UINT_MAX, which can lead to ext2fs_open()
crashing with a division by zero error.

This bug was found using American Fuzzy Lop: http://lcamtuf.coredump.cx/afl/

Addresses-Debian-Bug: #868489

Reported-by: jwilk@jwilk.net
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoMerge branch 'maint' into next
Theodore Ts'o [Tue, 11 Jul 2017 02:15:05 +0000 (22:15 -0400)]
Merge branch 'maint' into next

7 years agoAOSP: blkid: Resolve to the exFAT uuid change on reboot.
liminghao [Mon, 10 Apr 2017 01:48:51 +0000 (09:48 +0800)]
AOSP: blkid: Resolve to the exFAT uuid change on reboot.

The volume_serial into exFAT super block is uuid but
not standard uuid, it's just volume serial number.

Change-Id: I376ed9fe1ba1b7f3d367d78cc5e2bb8ea9cc2d13
Signed-off-by: liminghao <liminghao@xiaomi.com>
From AOSP commit: 91e756e829362c66265334e2e20fb3604586ce8b

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoAOSP: blkid: add support to recognize exfat to blkid.
liminghao [Wed, 1 Mar 2017 09:54:42 +0000 (17:54 +0800)]
AOSP: blkid: add support to recognize exfat to blkid.

we can now identify exfat filesystem.

Change-Id: I870e59a14b6bcd8b45562cdd02c2502d60a9eeff
Signed-off-by: liminghao <liminghao@xiaomi.com>
From AOSP commit: 1206f6d8c5ed47ba19cfc30a19dba51fcd2cd5cb

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: remove #ifdef EXT2_NO_64_TYPE blocks
Tahsin Erdogan [Wed, 5 Jul 2017 04:08:07 +0000 (00:08 -0400)]
e2fsck: remove #ifdef EXT2_NO_64_TYPE blocks

EXT2_NO_64_TYPE is not defined anywhere. It appears to be a
historical artifact.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoUse i_size to determine whether a symlink is a fast symlink
Tahsin Erdogan [Fri, 30 Jun 2017 01:31:59 +0000 (18:31 -0700)]
Use i_size to determine whether a symlink is a fast symlink

Current way of determining whether a symlink is in fast symlink
format is to call ext2fs_inode_data_blocks2(). If number of data
blocks is zero and EXT4_INLINE_DATA_FL flag is not set, then symlink
data must be in inode->i_block.

This heuristic is becoming increasingly hard to maintain because
inode->i_blocks count can also be incremented for blocks used by
extended attributes. Before ea_inode feature, extra block could come
from xattr block, now more blocks can be added because of xattr
inodes.

To address the issue, add a ext2fs_is_fast_symlink() function that
gives a direct answer based on inode->i_size field. This is
equivalent to kernel's ext4_inode_is_fast_symlink() function.

This patch also fixes a few issues related to fast symlink handling:

  - Both rdump_symlink() and follow_link() interpreted symlinks with
    0 data blocks to always mean fast symlinks. This is incorrect
    because symlinks that are stored as inline data also have
    0 data blocks. Thus, they try to read everything from
    inode->i_block and miss the symlink suffix in inode extra area.

  - e2fsck_pass1_check_symlink() had code to handle inode with
    EXT4_INLINE_DATA_FL flag twice. The first if block always returns
    from the function so the second one is unreachable code.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoresize2fs: moving xattr inodes is not supported
Tahsin Erdogan [Wed, 5 Jul 2017 03:53:59 +0000 (23:53 -0400)]
resize2fs: moving xattr inodes is not supported

In some cases, resize2fs needs to move inodes because their inode
number is greater than the maximum allowed. Moving extended attribute
inodes would require updating all the references to them. This
is currently not supported.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agomke2fs: ea_inode is not supported for hurd
Tahsin Erdogan [Wed, 5 Jul 2017 03:53:59 +0000 (23:53 -0400)]
mke2fs: ea_inode is not supported for hurd

Extended attribute inodes store their refcount in inode.l_i_version
field which is not available for Hurd. Fail mke2fs for this
combination.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agofuse2fs: refuse to mount fs with ea_inode feature
Tahsin Erdogan [Wed, 5 Jul 2017 03:53:59 +0000 (23:53 -0400)]
fuse2fs: refuse to mount fs with ea_inode feature

ext2fs_xattr_set() currently does not support creating xattr inodes,
so allowing fuse2fs to mount a filesystem with ea_inode feature could
lead to corruption. Refuse to mount if the ea_inode feature is set.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agotune2fs: update ea_inode hashes when fs uuid changes
Tahsin Erdogan [Wed, 5 Jul 2017 03:53:59 +0000 (23:53 -0400)]
tune2fs: update ea_inode hashes when fs uuid changes

Extended attribute inodes maintain a crc32c hash that is used for
deduplication. The crc seed derives from uuid so ea_inode hashes
must be updated when uuid changes.

The ea_inode hash is also incorporated into the xattr entry e_hash
so the entries that reference the inode also must be updated.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: add test for ea_inode feature
Tahsin Erdogan [Wed, 5 Jul 2017 03:53:59 +0000 (23:53 -0400)]
e2fsck: add test for ea_inode feature

f_ea_inode test covers the following scenarios:

- a file that contains old Lustre-style valid ea_inode references
  in inode body and xattr block

- a file that contains new style valid ea_inode references in inode
  body and xattr block

- a file with an extended attribute that references an invalid inode
  number (e_value_inum > s_inodes_count)

- an ea entry with bad e_hash and points to an inode that does not
  have EA_INODE flag set

- an ea entry with bad e_hash but points to a valid ea_inode

- an ea entry with valid e_hash that points to an inode that is
  missing EA_INODE flag

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: track ea_inode references
Tahsin Erdogan [Wed, 5 Jul 2017 03:53:59 +0000 (23:53 -0400)]
e2fsck: track ea_inode references

An extended attribute inode has a ref count to track how many entries
point to it. Update e2fsck to verify that the stored ref count is
correct.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: update i_blocks accounting for ea_inode feature
Tahsin Erdogan [Wed, 5 Jul 2017 03:53:59 +0000 (23:53 -0400)]
e2fsck: update i_blocks accounting for ea_inode feature

With ea_inode feature, i_blocks include the disk space used by
referenced xattr inodes. Make e2fsck aware of that.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: generalize ea_refcount
Tahsin Erdogan [Wed, 5 Jul 2017 03:53:59 +0000 (23:53 -0400)]
e2fsck: generalize ea_refcount

Currently ea_refcount is only used to track ea block refcounts. By
generalizing it, we could use it for ea quota tracking and also
ea_inode refcounts.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: do not early terminate extra space check
Tahsin Erdogan [Wed, 5 Jul 2017 03:53:59 +0000 (23:53 -0400)]
e2fsck: do not early terminate extra space check

When check_inode_extra_space() detects a problem with the value of
i_extra_isize, it adjusts it and then returns without further validation
of contents in the inode body. Change this so that it will proceed to
check inline extended attributes.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: ea_inode hash validation
Tahsin Erdogan [Wed, 5 Jul 2017 03:53:59 +0000 (23:53 -0400)]
e2fsck: ea_inode hash validation

In original implementation of ea_inode feature, each xattr inode had
a single parent. Child inode tracked the parent by storing its inode
number in i_mtime field. Also, child's i_generation matched parent's
i_generation.

With deduplication support, xattr inodes can now be shared so a single
backpointer is not sufficient to achieve strong binding. This is now
replaced by hash validation.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agotune2fs: do not allow disabling ea_inode feature
Tahsin Erdogan [Wed, 5 Jul 2017 03:53:59 +0000 (23:53 -0400)]
tune2fs: do not allow disabling ea_inode feature

Disabling ea_inode feature would require inlining all the existing
xattr values that are currently stored in external inodes. This is
not always possible. Just disallow it.

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe2fsck: add support for large xattrs in external inodes
Andreas Dilger [Wed, 5 Jul 2017 03:53:59 +0000 (23:53 -0400)]
e2fsck: add support for large xattrs in external inodes

Add support for the INCOMPAT_EA_INODE feature, which stores large
extended attributes into an external inode instead of data blocks.
The inode is referenced by the e_value_inum field (formerly the
unused e_value_block field) from the extent header, and stores the
xattr data starting at byte offset 0 in the inode data block.

The xattr inode stores the referring inode number in its i_mtime,
and the parent i_generation in its own i_generation, so that there
is a solid linkage between the two that e2fsck can verify.  The
xattr inode is itself marked with EXT4_EA_INODE_FL as well.

Signed-off-by: Kalpak Shah <kalpak.shah@sun.com>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoMerge branch 'maint' into next
Theodore Ts'o [Wed, 5 Jul 2017 03:53:11 +0000 (23:53 -0400)]
Merge branch 'maint' into next

7 years agolibext2fs: fix compile errors/warnings
Tahsin Erdogan [Fri, 30 Jun 2017 04:19:32 +0000 (21:19 -0700)]
libext2fs: fix compile errors/warnings

Defining DEBUG_QUOTA reveals a few compile errors. Fix these.
Also fix format string type mismatches in a couple of log_debug()
calls.

Fixes: 2d2d799c7261 ("Clean up codes for adding new quota type")

Signed-off-by: Tahsin Erdogan <tahsin@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoFix warnings found using UBSAN
Theodore Ts'o [Tue, 4 Jul 2017 22:00:46 +0000 (18:00 -0400)]
Fix warnings found using UBSAN

Compiling with -fsanitize=undefined -fsanitize=address causes some
warnings of C code that has undefined behavior according to the C
standard bugs.  None of the warnings should cause e2fsprogs
malfunction given a sane compiler running on architectures that Linux
can support.  Still, it's better to clean up to code than not.

To fix up a complaint of a negative shift in hash function, update the
very dated hash we had been using for the revoke table with the
current generic hash used by the kernel.

Other fixes are fairly self-explanatory.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agotests: add j_recover_csum3_64bit
Theodore Ts'o [Tue, 4 Jul 2017 21:51:48 +0000 (17:51 -0400)]
tests: add j_recover_csum3_64bit

Add a test to make sure we are correctly recovering 64-bit journals
using the v3 checksum format.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agolibext2fs: fix build issue for on Windows/Cygwin systems
Theodore Ts'o [Tue, 4 Jul 2017 01:28:53 +0000 (21:28 -0400)]
libext2fs: fix build issue for on Windows/Cygwin systems

Provide ext2fs_get_device_size2() for Windows/Cygwin systems, so
builds can succeed.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
7 years agoe4defrag: fix GCC7.x compiler warning
Marc Thomas [Mon, 26 Jun 2017 15:40:39 +0000 (16:40 +0100)]
e4defrag: fix GCC7.x compiler warning

../../misc/e4defrag.c:1821:8: warning: statement will never be executed
[-Wswitch-unreachable]
    int mount_dir_len = 0;

Also fix a typo in a few comments.

Signed-off-by: Marc Thomas <marc@dragonfly.plus.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>