LU-12561 ldiskfs: Remove 2.6 kernel series Lustre no longer supports 2.6 kernels, so remove the ldiskfs patch series for these kernels. Test-Parameters: trivial Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Change-Id: I2c7941582ee90416149cecd47fd2914bc5265a7f Reviewed-on: https://review.whamcloud.com/35549 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-11790 ldiskfs: add terminating u32 when expanding inodes In ext4_expand_extra_isize_ea(), we calculate the total size of the xattr header, plus the xattr entries so we know how much of the beginning part of the xattrs to move when expanding the inode extra size. We need to include the terminating u32 at the end of the xattr entries, or else if there is uninitialized, non-zero bytes after the xattr entries and before the xattr values, the list of xattr entries won't be properly terminated. Signed-off-by: Li Dongyang <dongyangli@ddn.com> Change-Id: I247b935b3cf315481dc4658133a7eee02b6350e9 Reviewed-on: https://review.whamcloud.com/33893 Tested-by: Jenkins Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Yang Sheng <ys@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
LU-11187 ldiskfs: don't mark mmp buffer head dirty Marking mmp bh dirty before writing it will make writeback pick up mmp block later and submit a write, we don't want the duplicate write as kmmpd thread should have full control of reading and writing the mmp block. Another reason is we will also have random I/O error on the writeback request when blk integrity is enabled, because kmmpd could modify the content of the mmp block(e.g. setting new seq and time) while the mmp block is under I/O requested by writeback. Linux-commit: fe18d649891d813964d3aaeebad873f281627fbc Test-Parameters: testgroup=review-ldiskfs testlist=mmp Signed-off-by: Li Dongyang <dongyangli@ddn.com> Change-Id: I5aa9fd384a4ea25ee52f1198528fae4ecc9c28c7 Reviewed-on: https://review.whamcloud.com/33038 Tested-by: Jenkins Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Wang Shilong <wshilong@ddn.com> Tested-by: Maloo <hpdd-maloo@intel.com>
LU-10048 osd: async truncate osd-ldiskfs should execute truncate outside of main transaction handle. This avoids restarting truncate transaction handles in main transaction, and allows "transaction first, locking second" model on OST. Change-Id: Iffe45c42834c26ca72b65e068ad25ac61d0607c8 Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com> Reviewed-on: https://review.whamcloud.com/27488 Tested-by: Jenkins Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Fan Yong <fan.yong@intel.com>
LU-9333 osd: few cleanups to support gcc6 few cleanups to support gcc6 Change-Id: Ibe664e87ff67eb3d34ce09c9a1892e55a63ab2cb Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com> Reviewed-on: https://review.whamcloud.com/26584 Tested-by: Jenkins Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: James Simmons <uja.ornl@yahoo.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-8410 ldiskfs: new FIEMAP API With RH 6.5 old API was deprecated and was removed. Backport a new API from ext4 upstream in opposite to copy-paste older buggy code as FIEMAP now uses in current code. Kernel upstream commit is 91dd8c114499e9818f2d5919ef0b9eee61810220 ext4: prevent race while walking extent tree for fiemap. Signed-off-by: Alexey Lyashkov <alexey.lyashkov@seagate.com> Change-Id: I7790c9e1a9cbfbd2cc429292aa764250e0525e21 Reviewed-on: http://review.whamcloud.com/21603 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Faccini Bruno <bruno.faccini@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-8216 ldiskfs: fix journal quota files When we hit Lustre crash, e2fsck will always complains Quota accounting mismatch, and sometimes differences are huge. Problems are we don't journal quota updates properly, and with current codes quota files will be only journaled when syncing filesystem, that only happen by sync call or unmounting. Signed-off-by: Wang Shilong <wshilong@ddn.com> Change-Id: I94c148a6b48f2464e96320bbadc88a3c234eacb0 Reviewed-on: http://review.whamcloud.com/20503 Tested-by: Jenkins Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Yang Sheng <yang.sheng@intel.com>
LU-7980 ldiskfs: always pre-allocate max depth for path Upon very specific meta-data and I/O pattern a situation can be encountered where ext_depth() will concurrently (due to i_data_sem release) grow outside of currently used ext4_ext_path[] array boundary. This can lead to potential Slab overrun if current ext_depth() is used to index previously sized+allocated ext4_ext_path[] array. This patch fixes this by already pre-allocating _ext_path[] array of the max possible depth. This is an alternate way to avoid racy cases, that can lead to slab overrun, with current and unsafe implementation that causes ext4_ext_path[] array re-[sizing,allocation] based on current depth. Now unnecessay free an realloc upon depth change detection prior to reuse already allocated path space have also been removed. Signed-off-by: Bruno Faccini <bruno.faccini@intel.com> Change-Id: I0ae955c5eccf032cb02ef55c6381964a5eaf82c2 Reviewed-on: http://review.whamcloud.com/19349 Tested-by: Jenkins Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
LU-6824 ldiskfs: give warning with dir htree growing Currently without large dir feature, ldiskfs directory hash tree will be limited 2 height, this means directory size is limited about 1GB, and in fact users are likely to hit ENOSPC when reaching half of limit because of bad hash. tested by following scripts. i=0 filename="aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbb" while [ 1 ] do touch $filename"$i" if [ $? -ne 0 ];then break fi ((i++)) done When directoy size grow about 590M, we hit ENOSPC. Better way is to add support to e2fsprogs so that we can use large dir feature. As a walkaround way, this patch try to give warning messages to console when 10/16 and 11/16 of limit reach. So this patch will try to give following messages when warning limit or hash index tree limit reach: Directory (inode: 8388610 FID: [0x200000401:0x1:0x0]) has approached maximum limit. Directory (inode: 8388610 FID: [0x200000401:0x1:0x0]) is approaching maximum limit. FID output here is useful for administrators to locate lustre file path Signed-off-by: Wang Shilong <wshilong@ddn.com> Change-Id: I7f78c421bbb89f76298e0174cc46d774ea82eb06 Reviewed-on: http://review.whamcloud.com/15548 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Yang Sheng <yang.sheng@intel.com> Reviewed-by: James Simmons <uja.ornl@yahoo.com>
LU-1026 ldiskfs: make bitmaps corruption not fatal We still hit bitmaps problems for rhel6 series kernel, corruptions happen because ext4_mb_check_ondisk_bitmap() check failed and FS become RO again: ldiskfs_mb_check_ondisk_bitmap: on-disk bitmap for group 294corrupted: 20180 blocks free in bitmap, 20181 - in gd Aborting journal on device dm-6-8. LDISKFS-fs (dm-6): Remounting filesystem read-only ldiskfs_mb_new_blocks: Updating bitmap error: [err -30] [pa ffff880d9d6e4d68] [phy 14974976] [logic 8192] [len 3072] [free 3072] [error 1] [inode 278678] ldiskfs_ext_new_extent_cb: Journal has aborted this might be caused by some ext4 internal bugs, this patch did the following things: 1.Inside ext4_read_block_bitmap() have gaven reasons why it failed, so caller don't need call ext4_error() again. 2. mark block group corrupt and use ext4_warning() instead of ext4_error(). There are still some bitmaps corruption places not handling, let's keep it for now, and if it really hurt, let's add the same handling codes logic later. Tested by following scripts: TEST_DEV="/dev/sdb" TEST_MNT="/mnt/ext4" mkdir -p $TEST_MNT mkfs.ext4 -F $TEST_DEV >&/dev/null mount -t ldiskfs $TEST_DEV $TEST_MNT dd if=/dev/zero of=$TEST_MNT/largefile oflag=direct bs=10485760 count=200 umount $TEST_MNT dd if=/dev/zero of=$TEST_DEV bs=4096 seek=641 count=10 oflag=direct mount -t ldiskfs $TEST_DEV $TEST_MNT rm -f $TEST_MNT/largefile dd if=/dev/zero of=$TEST_MNT/largefile oflag=direct bs=10485760 count=200 && echo "FILESYSTEM still usable after bitmaps corrupts happen" dmesg | tail umount $TEST_MNT e2fsck $TEST_DEV -y Signed-off-by: Wang Shilong <wshilong@ddn.com> Change-Id: Iabb6ebf719d80d9ba4f41bee0b237e304212832b Reviewed-on: http://review.whamcloud.com/16679 Tested-by: Jenkins Reviewed-by: Bob Glossman <bob.glossman@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Yang Sheng <yang.sheng@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-7132 osd-ldiskfs: speedup rewrites This patch slightly speeds up rewrites on OST by replacing bmap calls with fiemap calls. This patch also includes a fiemap deadlock fix created by Alexey Lyashkov. Change-Id: I8af6350a0049a14a3e29304087064ecdffc1be89 Signed-off-by: Andrew Perepechko <andrew.perepechko@seagate.com> Signed-off-by: Alexey Lyashkov <alexey.lyashkov@seagate.com> Reviewed-by: Artem Blagodarenko <artem.blagodarenko@seagate.com> Reviewed-by: Alexander Zarochentsev <alexander.zarochentsev@seagate.com> Xyratex-bug-id: MRP-2559 Xyratex-bug-id: MRP-2688 Reviewed-on: http://review.whamcloud.com/16360 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: James Simmons <uja.ornl@yahoo.com>
LU-6030 ldiskfs: split pdirop patch Split pdirop patch as two parts. One for Nlevel-htree and Largedir, other one for pdirop and htree-lock. Also doing some cleanup work to reduce the patch size. Signed-off-by: Yang Sheng <yang.sheng@intel.com> Change-Id: I08b65d9098be95994f44748dbf14afa9f6d5b372 Reviewed-on: http://review.whamcloud.com/14264 Tested-by: Jenkins Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Bob Glossman <bob.glossman@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-6030 ldiskfs: Remove alloc-policy patch Remove alloc_policy & wantedi two patches since they are no longer used. Signed-off-by: Yang Sheng <yang.sheng@intel.com> Change-Id: Ia283e16afe9d35d667c48e38db83d2a9cacb152c Reviewed-on: http://review.whamcloud.com/14299 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: James Simmons <uja.ornl@yahoo.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Bob Glossman <bob.glossman@intel.com>
LU-6030 osd-ldiskfs: improve mount option handling --handle force-over-128tb option to osd layer --handle bigendian-check option to osd layer --strip out extents option & remove extents-mount-options patch --strip out iopen & mballoc mount options --back LDISKFS_SUPER_MAGIC to EXT4_SUPER_MAGIC Signed-off-by: Yang Sheng <yang.sheng@intel.com> Change-Id: Ic9bf431d0826d6279fc76f7fd1d7e356e421f292 Reviewed-on: http://review.whamcloud.com/13572 Reviewed-by: Bob Glossman <bob.glossman@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Tested-by: Jenkins Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com>
LU-6030 ldiskfs: clean up ext4-fiemap patch Move ext4-fiemap patch to osd-ldiskfs. So we can remove this patch entirely. Signed-off-by: Yang Sheng <yang.sheng@intel.com> Change-Id: I639733f6f106398bbc3d5e2ffc6fa8a06ffe867f Reviewed-on: http://review.whamcloud.com/13571 Tested-by: Jenkins Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Bob Glossman <bob.glossman@intel.com> Reviewed-by: James Simmons <uja.ornl@gmail.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-6137 ldiskfs: simplify nocmtime patch Simplify the nocmtime patch by patching only ext4_current_time(), this fixed the defect that original patch doesn't handle setacl code path, it can also avoid the risk of future changes adding new places that needs to be fixed. Remove the obsolete xattr-no-update-ctime patch. Signed-off-by: Anreas Dilger <andreas.dilger@intel.com> Signed-off-by: Niu Yawei <yawei.niu@intel.com> Change-Id: I02928c4f867e9476f0bc1815dd3256e3d79dadf7 Reviewed-on: http://review.whamcloud.com/13705 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Bobi Jam <bobijam@hotmail.com> Reviewed-by: Bob Glossman <bob.glossman@intel.com> Reviewed-by: Fan Yong <fan.yong@intel.com>
LU-6199 ldiskfs: delete bad WARN_ON_ONCE from ldiskfs lustre needs to call certain ext4/ldiskfs entry points without locking i_mutex in order to avoid deadlocks. This triggers a warning check in ext4 code new in el6.6, not present in el6.5. Already fixed in ldiskfs patches for future kernel versions, but wasn't fixed for el6.6 This mod adds an ldiskfs patch to eliminate the warning. Signed-off-by: Bob Glossman <bob.glossman@intel.com> Change-Id: Ia375a6d851a5262c578d722e2f8f4db2ea5249b7 Reviewed-on: http://review.whamcloud.com/13604 Tested-by: Jenkins Reviewed-by: Yang Sheng <yang.sheng@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4648 ldiskfs: delete external EA in another transaction The transaction credit of ldiskfs_delete_inode() does not count in the external EA inodes deletion, which could lead to transaction credit deficiency. This patch collect the inode's xattr inodes and put them to orphan list, and later delete them in another transaction after deleting the main inode. Signed-off-by: Bobi Jam <bobijam.xu@intel.com> Change-Id: Id9a64e4232e450b2628de9be323aeb65389209ce Reviewed-on: http://review.whamcloud.com/8881 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com> Reviewed-by: James Simmons <uja.ornl@gmail.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-6030 ldiskfs: cleanup ldiskfs_journal_dirty_metadata ldiskfs_journal_dirty_meatdata only exist in ext3 code. In the ext4 ldiskfs_handle_dirty_metadata instead of it. As we won't support ext3 any more, Cleanup its usage from ldiskfs & osd. Signed-off-by: Yang Sheng <yang.sheng@intel.com> Change-Id: I7a1f65bd41fb552cb099831fc0254926ae5c2df3 Reviewed-on: http://review.whamcloud.com/13067 Tested-by: Jenkins Reviewed-by: Bob Glossman <bob.glossman@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Bobi Jam <bobijam@hotmail.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5771 ldiskfs: cleanup orphan inode in error path Commit from upstream: 4538821993f4486c76090dfb377c60c0a0e71ba3 There were some error paths in ext4_delete_inode() which was not dropping the inode from the orphan list. This could lead to a BUG_ON on umount when the orphan list is discovered to be non-empty. Signed-off-by: Wang Shilong <wshilong@ddn.com> Change-Id: Ica2c87ae378cc23eed70c1bfeb96db3852d218dc Reviewed-on: http://review.whamcloud.com/12349 Tested-by: Jenkins Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: James Simmons <uja.ornl@gmail.com> Tested-by: Maloo <hpdd-maloo@intel.com>