Whamcloud - gitweb
wang di [Tue, 11 Mar 2014 15:38:24 +0000 (08:38 -0700)]
LU-4585 out: remove OBJECT_UPDATE_PER_RPC_MAX limit.
Extend tx_args if the update inside 1 RPC is more than
the count of tx_args count in tgt_thread_info.
And also for create remote object, the remote RPC might
happen in trans stop, so we need to add return value
for mdd_trans_stop to track the error.
Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ibb87181d87dfb6c3ed7e6757592bccd4a836a98a
Reviewed-on: http://review.whamcloud.com/9590
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Nathaniel Clark [Wed, 19 Mar 2014 14:33:32 +0000 (10:33 -0400)]
LU-4108 tests: decrease load for zfs in performance-sanity/4
Given the poor performance of metadata on ZFS (cf LU-2600), this patch
decreases the number of files created (similar to what happens in
perforamnce-scale.sh).
Test-Parameters: mdtfilesystemtype=zfs ostfilesystemtype=zfs mdsfilesystemtype=zfs testlist=performance-sanity
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I998c8525bf218dacad07d98ac842f4c12102cd9a
Reviewed-on: http://review.whamcloud.com/9725
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Fan Yong [Tue, 4 Mar 2014 11:40:48 +0000 (19:40 +0800)]
LU-4777 lfsck: redefine LFSCK_RBTREE_BITMAP_MASK
It should be the rbtree bitmap bits size, not the bytes size.
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I0c090702f3f2f5037926d6384b605751a0a9c948
Reviewed-on: http://review.whamcloud.com/9703
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Fan Yong [Tue, 4 Mar 2014 06:19:41 +0000 (14:19 +0800)]
LU-4775 lfsck: dump all visibile lfsck parameter correctly
We missed to handle "LPF_BROADCAST" and "LPF_ORPHAN" when dump
the lfsck parameters and caused the output via /proc interface
incomplete and bad formatted.
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I462c79e80be5b384a291b30980a95698a8dd9d19
Reviewed-on: http://review.whamcloud.com/9690
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Alex Zhuravlev [Thu, 13 Feb 2014 19:55:43 +0000 (23:55 +0400)]
LU-4611 osd: improve credits calculation
- llog catalog do not declare records twice (for old and new objects)
this might be an issue on ZFS with full debug enabled..
- llog to specify append by pos=-1, so OSD can take this into account
- object create/destroy should not include OI, this is calculated yet
- EA declaration to improve few specific cases
- osd_declare_write() to recognize overwrite optimistically,
using inode size and i_blocks
- osd_declare_write() to optimize very specific cases, like legacy
blockmaps with small offsets and allocated indirects
- index delete modify just a single block
preliminary testing on a local setup with 7 OSTs:
1360 credits before and 436 credits after.
llog declarations and index inserts (part of llog object creation)
still consume about 70% (7 OSTs):
create: 7/28, destroy: 1/4
attr_set: 2/2, xattr_set: 8/21
write: 37/179, punch: 14/56, quota 2/2
insert: 8/135, delete: 2/5
ref_add: 1/1, ref_del: 3/3
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Icb5c79df1f8ba248509b9d2561ac8843bb01f6af
Reviewed-on: http://review.whamcloud.com/9258
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Christopher J. Morrone [Thu, 20 Feb 2014 19:08:44 +0000 (11:08 -0800)]
LU-4624 statahead: Improve statahead debug messages
The statahead debug messages include the pid of the current
process in their body. This is both redudant (because all
lustre log messages contain the pid), and sometimes downright
misleading. For instance the messages would say something like
"stopping statahead thread 3446". One would probably think
that 3446 is the pid of the process that is being stopped,
but in fact it was the pid of the caller issuing the stop signal.
We remove all superfluous pids from the messages.
Next we have the ll_statahead_thread() and the ll_agl_thread() record
their respective pids in their respective ptlrpc_thread structures.
This allows to print the pid of the thread that we are trying to
stop (which is actually useful info) from other threads, such as those
calling ll_stop_statahead().
Change-Id: Id4def179e02e9097f3f22048192d45bdf15b0aa8
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/9360
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Niu Yawei [Tue, 18 Mar 2014 09:04:18 +0000 (05:04 -0400)]
LU-4778 llog: update catlog when plain log destroyed
When the empty plain log is destroyed while processing
catlog, we should call llog_cat_cleanup() to close the
plain log and update in catlog.
Intel-bug-id: INTL-64
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I97b1d5cc59e84fdb4888274b48f223a063293647
Reviewed-on: http://review.whamcloud.com/9702
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Nathaniel Clark [Thu, 13 Mar 2014 19:31:41 +0000 (15:31 -0400)]
LU-4200 mgs: fix counting of mgs exports
In only_mgs_is_running iterate over mgs exports and only count
non-self and non MDS-MDS exports.
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I145d897fe2f222dae5e998de322ea58ac9f75375
Reviewed-on: http://review.whamcloud.com/9650
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Niu Yawei [Wed, 27 Nov 2013 03:27:01 +0000 (22:27 -0500)]
LU-4028 quota: fix output of 'lfs quota'
Fix the output of 'lfs quota' when '-h' option is specified:
- total allocated block limit should be converted after calculation;
- show 1~3 digit after the decimal point;
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I90a8087dc15153c1b0b895c8081fe2882d6d01e6
Reviewed-on: http://review.whamcloud.com/8407
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Alex Zhuravlev [Sat, 27 Jul 2013 20:07:57 +0000 (00:07 +0400)]
LU-3489 osd-zfs: object iterator support
a trivial object iterator for ZFS using dmu_object_next().
the iterator skips all objects with no LMA EA. It does
implement simple prefetching algoright. It also enables
few tests in sanity-lfsck to be run with ZFS.
Test-Parameters: mdtfilesystemtype=zfs \
ostfilesystemtype=zfs mdsfilesystemtype=zfs \
testlist=sanity-scrub,sanity-lfsck
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Icf02342ac351a163e23575963d3366761c59b02b
Reviewed-on: http://review.whamcloud.com/7149
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Alex Zhuravlev [Tue, 11 Mar 2014 05:10:01 +0000 (09:10 +0400)]
LU-4743 osp: continue llog processing
when unexpected record type is met
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I3dd45f501192bc6dd3bfddb550fd455ae319b923
Reviewed-on: http://review.whamcloud.com/9574
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Christopher J. Morrone [Fri, 21 Feb 2014 22:46:47 +0000 (14:46 -0800)]
LU-4624 llite: Avoid statahead thread start/stop deadlocks
The statahead and statahead agl threads blindly set their
thread state to SVC_RUNNING without checking the state first. If, for
instance, another thread sets the state to SVC_STOPPING that
stop signal will now have been lost. Deadlock ensues.
We also partly improve the sai reference counting, because a race exists
where the ll_stop_statahead thread can drop the default reference, and
the statahead thread can exit and drop its reference as well. With no
references on the sai, the final put will poison and free the buffer. The
original do_statahead_enter() function may then continue to access
the buffer after it is freed because it did not take a reference of its
own. We add a local reference to address that.
Change-Id: I531f17966d49ba1e6ebd99abe9cb8b128eeac4f9
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/9358
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
John L. Hammond [Thu, 30 Jan 2014 17:07:13 +0000 (11:07 -0600)]
LU-4398 mdt: acquire an open lock for write or execute
In mdt_object_open_lock() opens for write or execute will always
acquire an open lock of the appropriate mode so that any conflicting
cached open locks on other clients will be canceled. Add a regression
test to sanityn.sh.
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I8092bca4c418ec99a25584abdfb635ffec19a26e
Reviewed-on: http://review.whamcloud.com/9063
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Nathaniel Clark [Fri, 14 Mar 2014 20:39:21 +0000 (16:39 -0400)]
LU-2524 tests: enable conf-sanity/58 on ZFS
Fix mount/unmount of MDS so test runs correctly on ZFS MDS.
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ifb9bd12fdecdc38132dfc487f565b67f7ade4123
Reviewed-on: http://review.whamcloud.com/9675
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Fan Yong [Fri, 28 Feb 2014 20:35:09 +0000 (04:35 +0800)]
LU-4767 lfsck: sub-task thread should not change lc_link
The lfsck sub-task thread (for double scan) should not change the
"lfsck_component:lc_link" which may cause the master lfsck engine
thread to access some unexpected RAM space by race.
LFSCK should not return -ENODEV to avoid to misguide ptlrpc.
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ifb80eaf7c1c4ba9bf7fbd5213e9eae09efed7e28
Reviewed-on: http://review.whamcloud.com/9661
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Fan Yong [Tue, 25 Feb 2014 18:43:26 +0000 (02:43 +0800)]
LU-4721 obdclass: handle local storage init/fini properly
1) In local_oid_storage_fini(), take the mutex on ls_device
before decreasing the 'los' reference to avoid others to
obtain the mutex earlier and freed the 'los' by race.
2) When llog init the local stroage for FID_SEQ_LLOG and
FID_SEQ_LLOG_NAME, it should record the handlers which
can be used to fini them to avoid releasing the handler
which is in using by others.
3) NOT forget the llog_ctxt_put() if something wrong during
the llog_osd_setup().
4) NOT put the object in lastid_compat_check() until all the
usages on such object have been done.
Test-Parameters: envdefinitions=SLOW=yes,ENABLE_QUOTA=yes testlist=sanity-scrub,sanity-scrub,sanity-scrub,sanity-scrub
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ibf139cf9783d06276a6c0ed764c40dec6e3f70cb
Reviewed-on: http://review.whamcloud.com/9573
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Bruno Faccini [Mon, 27 Jan 2014 18:55:31 +0000 (19:55 +0100)]
LU-3852 tests: disable test_251 from inside sanity-hsm
Due to test_251 frequent failures, subtest has been disabled at
the autotests config level. To be able to work on a patch to fix
the root cause (likely to be a pure consequence when running on
a too small-sized Lustre filesystem), subtest will be disabled
from inside sanity-hsm to allow autotests config changes to be
reverted and to re-enable it on a per-patch basis.
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I7dfae4d278cba049062480c276f9cad10d22e0e9
Reviewed-on: http://review.whamcloud.com/9014
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>
Emoly Liu [Mon, 3 Mar 2014 04:19:05 +0000 (12:19 +0800)]
LU-4704 test: enable acl 974 and 2561 series test
Add acl/{974,974_remote,2561,2561_zfs}.test to the
lustre/tests/Makefile.am nobase_noinst_DATA list, and fix acl/run
test to fail if the test scripts are missing.
acl/2561_zfs.test is a newly created test file for zfs. Since zfs
allocates a 512 byte dnode for empty file, its "ls -s" result is
different from ldiskfs'.
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I11be072eb1954fff3b7a19231b82345b3073a534
Reviewed-on: http://review.whamcloud.com/9541
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
John L. Hammond [Tue, 11 Mar 2014 20:15:16 +0000 (15:15 -0500)]
LU-4453 build: warn on deprecated cfs_* uses
In contrib/scripts/checkpatch.pl, use static dictionaries of
deprecated identifiers and headers rather than reading from the
kernel's Documentation/feature-removal-schedule.txt file. Populate the
deprecated identifiers dictionary with the cfs_{atomic,hlist,list}_*
macros, some out of fashion typedefs, and a few unsafe functions.
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I5af33e33693d6da59b98890ae2f27d95adcd8710
Reviewed-on: http://review.whamcloud.com/9585
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Reviewed-by: Li Xi <pkuelelixi@gmail.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>
John L. Hammond [Tue, 11 Mar 2014 20:45:17 +0000 (15:45 -0500)]
LU-4752 build: add gerrit_checkpatch.py to contrib/scripts/
Add a python script gerrit_checkpatch.py to contrib/scripts/ which
polls for recently updated changes, runs checkpatch.pl on them, and
pushes reviews back to gerrit.
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I9e3c9990288b8adc8f507a6b2294f2a53991a0e8
Reviewed-on: http://review.whamcloud.com/9586
Tested-by: Jenkins
Reviewed-by: Richard Henwood <richard.henwood@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Dmitry Eremin [Mon, 17 Mar 2014 20:33:19 +0000 (00:33 +0400)]
LU-4770 tests: tests scripts simplification
1. Xeon Phi don't support "mount -t lustre", we can use only
"mount.lustre" utility.
2. A "head" utility don't support shortcut "-1" option.
It should be specified as "-n 1".
3. A "sed" utility don't support expression "s/[-.]/ /3"
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ib5ba22451819772ebacb38706447403ae6668cc4
Reviewed-on: http://review.whamcloud.com/9668
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Oleg Drokin [Thu, 6 Mar 2014 01:56:38 +0000 (20:56 -0500)]
LU-4719 mdt: Don't try to print non-existent objects
mdt_dump_lmm() needs to make sure it's not trying to print
objects for a default striping from a directory, since there
are not objects in this case.
Return early if the D_INFO debug mask is disenabled, so it
doesn't iterate through the LOV/LMV layout at all. That is
just a waste of effort.
Change-Id: I2fffdaefad2a6b05247676441451fffdca620dcd
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/9520
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Matt Ezell <ezellma@ornl.gov>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
yangsheng [Tue, 18 Mar 2014 14:57:59 +0000 (22:57 +0800)]
LU-4252 osd: remove locking i_mutex in osd_punch
This piece of code intent to calm down the kernel
WARN_ON messsage show up since 3.9 upstream. So we
can just remove it for now. We'll add a ldiskfs
patch to resolve this issue while 3.9 server support
come up.
Change-Id: Ied941b98d213e3aff1c5ba7ec4c6436bdef903c7
Signed-off-by: yang sheng <yang.sheng@intel.com>
Reviewed-on: http://review.whamcloud.com/9644
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
John L. Hammond [Fri, 20 Dec 2013 20:01:58 +0000 (14:01 -0600)]
LU-2675 cleanup: const correct FID/OSTID/... helpers
Add a const qualifier wherever possible to the pointer parameters of
the inline helper functions in lustre_idl.h and lustre_fid.h. Change
the return type of several predicate functions from int to bool.
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ibe590d7184470cfe8a154fe0935a7d6bea261669
Reviewed-on: http://review.whamcloud.com/8641
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Oleg Drokin [Wed, 19 Mar 2014 07:22:40 +0000 (03:22 -0400)]
New tag 2.6.57
Change-Id: I58633c93179d37500989b54ecd53e3b715e5f2b2
Swapnil Pimpale [Thu, 13 Mar 2014 06:46:41 +0000 (12:16 +0530)]
LU-4753 endianness: Fix cpu_to_leXX size mismatch
In lod_prep_md_striped_create() and lod_xattr_set_lmv we use
cpu_to_le32() to transfer __u16 values. This patch replaces
cpu_to_le32() with cpu_to_le16().
Signed-off-by: Swapnil Pimpale <spimpale@ddn.com>
Change-Id: Icc33e3fac711b63cc9d67fcec82dd1b4fdc10cce
Reviewed-on: http://review.whamcloud.com/9637
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Emoly Liu [Tue, 11 Mar 2014 23:05:01 +0000 (07:05 +0800)]
LU-4748 test: get one single qos_threshold_rr number
In sanity.sh test_116b, if there are multiple MDTs on the same MDS,
we should get one single qos_threshold_rr number instead of the array,
otherwise it will cause resetting qos_threshold_rr failure.
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Ie779769f7ee35cd118fb6412fe3352a03254b9d3
Reviewed-on: http://review.whamcloud.com/9580
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
wang di [Mon, 16 Dec 2013 17:21:16 +0000 (09:21 -0800)]
LU-3534 out: only start transaction for modified update
Only start transaction, if the update will modify the
filesystem.
Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I9c516b5fe6f6bc743e1dbc69ab04fd7062b2311d
Reviewed-on: http://review.whamcloud.com/7983
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Brian Behlendorf [Tue, 14 May 2013 22:39:04 +0000 (15:39 -0700)]
LU-3338 llite: Limit reply buffer size
When allocating a reply buffer for the striping information don't
assume the unlikely worst case. Instead, assume the common case
and size the buffer based on the observed default ea/cookie size.
The default size is initialized to a single stripe and allowed to
grow up to an entire page if needed. This means that for smallish
filesystems (less than ~21 OSTs) where the worst case striping
information can fit in a single page there is effectively no
change. Only for larger filesystem will the default be less than
the maximum. This has a number of advantages.
* By limiting the default reply buffer size we avoid always
vmalloc()'ing the buffer because it exceeds four pages in size
and instead kmalloc() it. This prevents the client from
thrashing on the global vmalloc() spin lock.
* A reply buffer of exactly the right size (no larger) is allocated
in the overflow case. These larger reply buffers are still
unlikely to exceed the 16k limit where a vmalloc() will occur.
* Saves memory in the common case. Wide striped files exceeded
the default are expected to be the exception.
The reason this patch works is because the ptlrpc layer is smart
enough to reallocate the reply buffer when an overflow occurs.
Therefore the client doesn't have to drop the incoming reply and
send a new request with a larger reply buffer.
It's also worth mentioning that the reply buffer always contains
a significant amount of extra padding because they are rounded up
to the nearest power of two. This means that even files striped
wider than the default have a good chance of fitting in the
allocated reply buffer.
Also remove client eadatasize check in mdt xattr packing because
as said above client can handle -EOVERFLOW.
Change-Id: Ic6bab2aca208db1f4abcd87974a5879d06d0f2f1
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-on: http://review.whamcloud.com/6339
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Nathaniel Clark [Fri, 14 Mar 2014 14:09:46 +0000 (10:09 -0400)]
LU-4769 tests: conf-sanity/38 fix regression
stop_mds on zfs exports the pool. Use mount_fstype that accounts for
importing pool if needed.
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Id3fbfc61af032aaaca1030a5927bd509725a0f21
Reviewed-on: http://review.whamcloud.com/9669
Tested-by: Jenkins
Reviewed-by: Isaac Huang <he.huang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Li Xi [Fri, 7 Mar 2014 04:32:12 +0000 (12:32 +0800)]
LU-4728 mdt: fix NULL deference of mdt_fid_lock
When enabling hsm_control, mti_exp field of struct mdt_thread_info
could be NULL. ldlm_cli_enqueue_local will crash the kernel when
dereference it.
Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I4bdb8222aec378e396e7f544834f6fcf9fdaf777
Reviewed-on: http://review.whamcloud.com/9543
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Li Wei [Tue, 23 Jul 2013 15:19:49 +0000 (23:19 +0800)]
LU-3539 osp: Fix a series of UPDATE_OBJ endianness bugs
Current UPDATE_OBJ code have the following problems:
- Fields in "struct update_buf", except for ub_bufs, are sent in
senders' host endianness but expected to be little-endian by
receivers.
- Fields in "struct update", except for u_bufs, are sent in
little endianness but used without swabbing by receivers.
- Most u_bufs are sent in little endianness, but some are in
senders' host endianness.
- Sizes for a couple of string buffers in osp_md_declare_xattr_set()
and osp_md_xattr_get() are off-by-one.
- Error numbers in UPDATE_OBJ replies are not translated on both
sides.
This patch fixes them by sending all data in UPDATE_OBJ requests and
replies in host endianness, following Lustre's usual practices.
Change-Id: Id3ffb248af181b4beff2b62c9cb46fc6d9f9e673
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/7088
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Michael MacDonald [Tue, 11 Mar 2014 04:37:14 +0000 (00:37 -0400)]
LU-4751 hsm: Fix sanity-hsm tests for non-mrsh $PDSH
The workaround for starting backgrounded copytool
monitors via pdsh -Rmrsh is not compatible with non-mrsh
values of $PDSH. This commit adds a branch such that
the workaround is only used when required by the test
cluster environment.
Signed-off-by: Michael MacDonald <michael.macdonald@intel.com>
Change-Id: I35a57d26a3836cfd1bbb698d267a8fe9f31f86c8
Reviewed-on: http://review.whamcloud.com/9587
Tested-by: Jenkins
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
John L. Hammond [Thu, 20 Feb 2014 15:58:51 +0000 (09:58 -0600)]
LU-4320 mdt: fixup MDS_SWAP_LAYOUTS ELC handling
In mdc_ioc_swap_layouts() cancel *any* unused locks with LAYOUT or
XATTR IBITS set on the two files. (This matches the locks acquired in
mdt_swap_layouts(). Previously only locks that conflicted with a CR
LAYOUT lock were cancelled.) Add HABEO_CLAVIS to the target handler
flags for MDS_SWAP_LAYOUTS and call ldlm_request_cancel() from
mdt_swap_layouts() to do the intended early lock cancellation before
the handler acquires its own locks.
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I9d7ba9ac660b136dafa7e307177b92204a1257c3
Reviewed-on: http://review.whamcloud.com/9329
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
wang di [Fri, 28 Feb 2014 15:52:58 +0000 (07:52 -0800)]
LU-4682 llite: a few fixes for migration.
1. Clear the client dentry cache before migrating file/directory
to the remote MDT.
2. Do not return stripe information to client, it did not get
Layout lock.
3. A few cleanups from the review of 6662.
Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I209079dfa07241412878b30fd02e8152046c95c7
Reviewed-on: http://review.whamcloud.com/9522
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Jinshan Xiong [Thu, 6 Mar 2014 06:45:47 +0000 (22:45 -0800)]
LU-4591 lov: cancel ungranted sub lock
When the top lock is canceled due to error, we should cancel
ungranted sub lock otherwise the sublock state is undefined.
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I5c1df5220ce77ff434463d6443135c625e79ea04
Reviewed-on: http://review.whamcloud.com/9524
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Nathaniel Clark [Wed, 5 Mar 2014 17:36:52 +0000 (12:36 -0500)]
LU-4703 tests: Add test to check setxattr fails when it should
Setxattr was not correctly checking permissions in all cases. This
adds a test to sanity to ensure that trying to set xattrs on a file
fails when it should.
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Id95e13ec3bcbedb7cbb069c610d506d1f3d899a8
Reviewed-on: http://review.whamcloud.com/9508
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Mikhail Pershin [Tue, 25 Feb 2014 08:13:01 +0000 (12:13 +0400)]
LU-4384 ofd: remove INCOMPAT_SUPP, ROCOMPAT_SUPP defines
Removed ROCOMPAT/INCOMPAT_SUPP defines from MDT and OFD as
obsoleted. They are not used now. The tgt_server_data_init()
uses own way to set/check rocompat/incompat masks
Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Ib4b7b4b948e0563d1cc89faae2d423245ed4df0e
Reviewed-on: http://review.whamcloud.com/9375
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Henri Doreau [Wed, 19 Feb 2014 14:49:41 +0000 (15:49 +0100)]
LU-4545 hsm: Always report errors to coordinator.
Make sure feedback on processed items gets properly delivered to the
coordinator even if errors occur between action item delivery and
item processing initialization phase.
Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Change-Id: If1dfc88e4acb9fbd6bfb5b2d18ffdd4979944997
Reviewed-on: http://review.whamcloud.com/9310
Tested-by: Jenkins
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
John L. Hammond [Wed, 26 Feb 2014 23:48:13 +0000 (17:48 -0600)]
LU-2785 osc: remove unused obd methods
Remove the unused methods osc_punch(), osc_sync(), osc_brw(),
osc_enqueue(), osc_cancel(), osc_llog_init(), osc_llog_finish(), and
their supporting functions. Remove the file obd_ost.h, moving all of
the structures defined there to osc_request.c. Remove several orphaned
obd methods.
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I4148547c52625cb7b173f1b835c9e6e47e19e4f6
Reviewed-on: http://review.whamcloud.com/8547
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Michael MacDonald [Thu, 6 Mar 2014 17:25:24 +0000 (12:25 -0500)]
LU-4643 hsm: Improve sanity-hsm test_60
Small fix to further reduce the chance of false failures.
Also cleans up example HSM events in comments to address style
warnings.
Signed-off-by: Michael MacDonald <michael.macdonald@intel.com>
Change-Id: Icecb69ddc417cd33f80b38c9df623ac5fc0cc466
Reviewed-on: http://review.whamcloud.com/9534
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Michael MacDonald [Fri, 7 Mar 2014 16:18:10 +0000 (11:18 -0500)]
LU-4724 hsm: Safe copytool event logging
Protect against concurrent event log writes by multiple
threads within a copytool process. Fixes sanity-hsm test_71
failures.
Signed-off-by: Michael MacDonald <michael.macdonald@intel.com>
Change-Id: Ie18f7368f386acf0949a038e6bdb2ba7dca5c3a3
Reviewed-on: http://review.whamcloud.com/9553
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
James Simmons [Mon, 9 Dec 2013 17:14:52 +0000 (12:14 -0500)]
LU-3953 rpm: stop using unnecessary file lists
File lists are being generated for unconditonally packaged files.
RPM can handle both static and dynamic file lists, so let's use them.
It simplifies the spec file quite a bit.
There are also some files that are treated as being conditionally
included but the conditions are always true:
- lustre_types.h was removed in commit
adde80ff (v1.9.50)
- libcfsutil.a was made unconditional in commit
b279ac68 (v2.3.51)
- include/lustre/lustre_idl.h was added in commit
11330397 (v1.7.100)
I've removed the conditionals surrounding these and moved them into
the static file lists.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I3f08a40f3ffef468723b366848ed5645c4cb9de9
Reviewed-on: http://review.whamcloud.com/5489
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Andreas Dilger [Wed, 5 Mar 2014 18:59:30 +0000 (11:59 -0700)]
LU-4708 ldiskfs: do not copy NUL terminator from direntry
Do not try to copy a NUL terminator from the source dirent if there
is no dir_data for that record. This can happen with 1.x filesystems
that do not have a dirdata field in the dirent to store the FID.
The target buffer is expected to be NUL terminated, and this is
handled by using kzalloc() to allocate it and zero out the buffer,
but the source buffer may not have a trailing NUL byte if it is a
multiple of 4 bytes in length.
If the dirent is aligned at the end of the directory block/page this
can cause an oops by accessing beyond the end of the page, or will
otherwise copy a garbage byte to the target buffer.
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Iab52f916d7571b3d2587255280c72a52c60399c9
Reviewed-on: http://review.whamcloud.com/9510
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Jinshan Xiong [Thu, 6 Feb 2014 06:49:23 +0000 (22:49 -0800)]
LU-4381 lov: to not hold sub locks at initialization
Otherwise, it will cause deadlock because it essentially holds
some sub locks and then to request others in an arbitrary order.
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I00d51677aa94ec41856402fd8a027e09355786ee
Reviewed-on: http://review.whamcloud.com/9152
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Jinshan Xiong [Fri, 8 Nov 2013 03:58:19 +0000 (19:58 -0800)]
LU-3274 osc: allow to call brw_commit() multiple times
Sometimes the rq_commit_cb of BRW RPC can be called twice if that RPC
has already committed at reply time. This will cause inaccuracy of
unstable pages accounting and then assertion.
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ic596e1d44e2c27bf13701d2c6c9691562beb7e58
Reviewed-on: http://review.whamcloud.com/8215
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Bobi Jam [Thu, 13 Feb 2014 15:27:41 +0000 (23:27 +0800)]
LU-4627 llite: deed taking lli_trunc_sem during file write
File write before io loop will take lli_trun_sem read semaphore to
protect osc_extent, while after generic_file_aio_write() done, it
could possible need to kill suid or sgid, which will call
ll_setattr_raw() to change the inode's attribute, and it does not
involve size.
So the ll_truc_sem write semaphore should be constrained
around ll_setattr_ost() to not come across the lli_trunc_sem read
semaphore get from the normal file write path.
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ic7f9cfb6d76dcf7ad08635efab154e7cdc392ce8
Reviewed-on: http://review.whamcloud.com/9267
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Niu Yawei [Wed, 5 Mar 2014 12:17:39 +0000 (07:17 -0500)]
LU-4702 nodemap: fix nm_rbtree_postorder_for_each_entry_safe()
It should check if the 'pos' && 'n' is NULL.
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ibda34ec4cf1706f0651ed8c86a2e49e4a36cd2c0
Reviewed-on: http://review.whamcloud.com/9500
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Joshua Walgenbach <jjw@iu.edu>
Niu Yawei [Fri, 21 Feb 2014 03:07:14 +0000 (22:07 -0500)]
LU-4422 quota: fix s-q test_6
s-q test_6 should not fail by checking whether the file is
growing, because in most time, the test is waiting for quota
slave reconnecting to quota master.
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I2332febf639bdbe033c588b97a2c7a1e1e2f51e2
Reviewed-on: http://review.whamcloud.com/9339
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Andreas Dilger [Sat, 8 Feb 2014 01:02:57 +0000 (18:02 -0700)]
LU-2524 tests: clean up conf-sanity test_38 for ZFS
Clean up conf-sanity.sh test_38 so that it mounts the filesystem
locally instead of using debugfs. That allows it to be run on
ZFS instead of being skipped. Other minor style fixes.
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I68090d2082b298b81518e5ffde847f15b47953b4
Reviewed-on: http://review.whamcloud.com/9184
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Bobi Jam [Tue, 31 Dec 2013 08:07:37 +0000 (16:07 +0800)]
LU-4388 llite: issue OST_SYNC for fsync()
The last parameter @datasync of fsync() has following indication:
* if datasync=0, we'd always flush data and metadata
* if datasync=1, we'd always flush data while does not flush modifed
metadata unless that metadata is needed in order to allow a
subsequent data retrieval to be correctly handled. For example, a
change to the file size would require a metadata flush.
Lustre client can not tell the difference easily, and would issue
MDS_SYNC and OST_SYNC in all cases.
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I90fa8e761e91c8a04afba3fbfafe1f60f5839ad1
Reviewed-on: http://review.whamcloud.com/8684
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Bob Glossman [Thu, 6 Mar 2014 18:31:05 +0000 (10:31 -0800)]
LU-4726 ptlrpc: vfs_rename api changed
In newer kernels the vfs_rename api changed to take
more arguments. This patch creates a wrapper to let us adapt
to the different versions of vfs_rename in old and new kernels.
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ia5dafe64570fc404cd8a037fd442ca12e687ffc1
Reviewed-on: http://review.whamcloud.com/9537
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Li Xi [Thu, 27 Feb 2014 17:02:12 +0000 (12:02 -0500)]
LU-4199 libcfs: add wrapper of dump_trace()
dump_trace() is a X86 specific function. It will cause
compilation failure if the architecture is not X86. A more
general function dump_stack() should be called under that
environment.
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I8a5fffa6dec15dc8c779a22e24c10b364c77f9a0
Reviewed-on: http://review.whamcloud.com/8872
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Li Xi [Tue, 4 Mar 2014 08:48:43 +0000 (16:48 +0800)]
LU-4704 acl: fix permission problem of setfacl
Setxattr does not check the permission when setting ACL xattrs. This
will cause security problem because any user can walk around
permission checking by changing ACL rules.
Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I1e5c44674f2e22ecbe29a589a219bed52d037b9b
Reviewed-on: http://review.whamcloud.com/9473
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Li Dongyang [Tue, 4 Mar 2014 06:10:49 +0000 (17:10 +1100)]
LU-4703 mdd: do not skip xattr sanity check for all cases
xattr sanity check should be done at all times.
Otherwise we are risking letting a non root user setting
access acl on any file.
Signed-off-by: Li Dongyang <dongyang.li@anu.edu.au>
Change-Id: I5f44d38f2589f0758548d8ccae5efef27205f0a0
Reviewed-on: http://review.whamcloud.com/9469
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Michael MacDonald [Wed, 26 Feb 2014 21:18:32 +0000 (16:18 -0500)]
LU-4020 hsm: allow copytool event monitoring with JSON
Adds hooks into various llapi_hsm_* functions to emit JSON-formatted
event messages for consumption by a monitoring agent. The copytool
needs to be supplied with an optional --event-fifo argument to
enable this feature.
Incorporates the following work done by Bruno Faccini:
* Put all JSON routines in a separate file/lib to allow for
LGPL licensing.
* Major code cleanup to follow best practices and address review
concerns.
Signed-off-by: Michael MacDonald <michael.macdonald@intel.com>
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I1fecfb8e60ef091c6d37d16dc1faf40be45c5ee2
Reviewed-on: http://review.whamcloud.com/7790
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Alex Zhuravlev [Sun, 9 Feb 2014 09:04:57 +0000 (13:04 +0400)]
LU-4602 utils: do not include <linux/fs.h>
in some case (e.g. cross-compilation on OS/X) MS_* are
defined before linux/fs.h. so do not try to re-define
them.
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I9ea8ea4865aa0380d7556b88604ec563acb0562b
Reviewed-on: http://review.whamcloud.com/9190
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
James Nunez [Tue, 25 Feb 2014 22:34:46 +0000 (15:34 -0700)]
LU-4639 hsm: HSM requests not delivered
The total size of an HSM archive request may exceed the
desired (LNET) message. When this happens, it can hang
the client and not allow the archive request to succeed.
Before we know the total size of the hsm_action_items, we
need to limit the size of the reguest. Doing this limits
the number of items that can be sent in one archive request.
We’ve reduced the size allowed for the user archive request
to MDS_MAXREQSIZE/3.
sanity-hsm test 90 forms a list of files to archive. The
number of files in the list needed to be decreased to
match the limit described above.
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I3c19851b7fedf62854a5a1b4a8f27b9c49d8b313
Reviewed-on: http://review.whamcloud.com/9393
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Hongchao Zhang [Thu, 27 Feb 2014 14:00:11 +0000 (22:00 +0800)]
LU-4557 ldiskfs: init statfs variables after journal
in ext4_fill_super, the variables related to statfs
should be initialized after journal recovery is completed.
otherwise, if a large number of blocks were being allocated
before the filesystem crashed, then the blocks and inode
counters may become negative during use and report incorrect
values to statfs call.
Change-Id: Id7e3f61ce73f5499a6176c336c7931a47f6f76de
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/9277
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
wang di [Thu, 20 Feb 2014 14:47:38 +0000 (06:47 -0800)]
LU-4658 fld: reset rc to 0 before resend fld lookup req
Reset rc to 0 before resend fld lookup req, otherwise resend will
exit in fld_client_rpc before send request.
Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I030262615be3c360aeb399eae3f90736c15ebf98
Reviewed-on: http://review.whamcloud.com/9333
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Peng Tao [Thu, 27 Feb 2014 08:07:39 +0000 (16:07 +0800)]
LU-4689 hsm: count NULL terminator in hai_first/hal_size
If fsname is 8-byte aligned, hai_first fails to count the ending NULL
terminator causing hai to directly attached after fsname and future
hai_first will return a different position for first hai.
Signed-off-by: Peng Tao <bergwolf@gmail.com>
Change-Id: I5e9e1f48f99b4743b2d5b93397e06f6becabeb26
Reviewed-on: http://review.whamcloud.com/9431
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Fan Yong [Mon, 17 Feb 2014 00:44:00 +0000 (08:44 +0800)]
LU-3951 lfsck: OST-object inconsistency self detect/repair
When client sends object-based RPC to the OST, the RPC service
thread on the OST needs to verify whether the given parent FID
in the client RPC matches the parent FID information stored in
the OST-object.
When the client given PFID does not match the OST local stored
PFID, then the OST will return "-EINPROGRESS" to the client for
retry later to avoid the RPC service thread to be blocked for
long time. On the other hand, there will be a dedicated thread
to talk with the LFSCK for the PFID verification. If the client
given parent FID information is incorrect, then deny the access;
otherwise, if the OST local stored PFID attribute is invalid,
then the OST local stored PFID xattr will be repaired.
Other fixes:
1) Hold update lock on the .lustre/lost+found/MDTxxxx object
when add new name entry for handling orphan OST-object.
2) Hold dt_write_lock on the OST-object to be destroyed before
transaction start, the same as normal ofd_object_destroy does.
3) Simplify the lfsck_layout_recreate_lovea() implementation.
4) Make sanity-lfsck test_18 to be workable under both DNE and
non-DNE cases.
5) Other code cleanup.
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iefd26fe1782761ab16954a00aea0788c39534580
Reviewed-on: http://review.whamcloud.com/7667
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
wang di [Fri, 28 Feb 2014 17:32:37 +0000 (09:32 -0800)]
LU-4690 tests: disable test_4 in sanity.sh
Disable test_4 in sanity.sh because of LU-4690.
Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I1acac5ad5c049eed7618aebf473b385a150316d7
Reviewed-on: http://review.whamcloud.com/9440
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Oleg Drokin [Sat, 1 Mar 2014 03:40:39 +0000 (03:40 +0000)]
Revert "LU-4659 mdd: rename forgets updating target linkea"
This now breaks the build due to conflict with some DNE patch.
This reverts commit
f6c64625b87c06749e04a0a74960852bb9e50750.
Change-Id: Ie47a36b5e9df70183f3f27cec0f47402cebb102c
Reviewed-on: http://review.whamcloud.com/9445
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
Oleg Drokin [Sat, 1 Mar 2014 03:25:17 +0000 (03:25 +0000)]
Revert "LU-4199 llite: Init lli_lmv_md field of struct ll_inode_info"
In fact this member was already removed by some DNE patch.
So this causes build failures now.
This reverts commit
c330f3c9d58ce507b2111ab3744dbf71353374bc.
Change-Id: I7d99d727fd2ef1f5b8f17ff6e3d9e9da79eae577
Reviewed-on: http://review.whamcloud.com/9442
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
Dmitry Eremin [Wed, 19 Feb 2014 13:44:51 +0000 (17:44 +0400)]
LU-3953 build: configure output improvments
Cleanup from mess in configure output and multi-line messages.
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: If90b0962f1b44f28af41e30015cc2ab253d6b83e
Reviewed-on: http://review.whamcloud.com/9309
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Jinshan Xiong [Fri, 7 Feb 2014 07:03:25 +0000 (23:03 -0800)]
LU-4300 ldlm: ELC picks locks in a safer policy
Change the policy of ELC to pick locks that have no dirty pages,
no page in writeback state, and no locked pages.
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ie73130c4100e3b91c211635f6a72fe5dad994426
Reviewed-on: http://review.whamcloud.com/9175
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
James Simmons [Thu, 13 Feb 2014 01:01:42 +0000 (20:01 -0500)]
LU-3974 llite: invalidatepage api changed
Until recently invalidating pages from the buffer cache
was dependent only on the page passed in and the start
in the page to invalidate. Starting with the 3.11 kernel
you can also specify the length of the data in the page
to invalidate. This patch enables us to handle the new
case.
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Iedf458b20b2604bc3099d5ae38bf0ad07df83bd3
Reviewed-on: http://review.whamcloud.com/7729
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
James Simmons [Thu, 13 Feb 2014 00:51:44 +0000 (19:51 -0500)]
LU-3974 llite: use new struct dir_context
The readdir and nfs code over time has added more
parameters to be passed to be processed. For the 3.11
kernel a new struct dir_context was introduced to
minimize the impact of future expansion. This patch
addresses this change.
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ib42bf8cb06635a2a64e63b294d79e66ac82a1a5b
Reviewed-on: http://review.whamcloud.com/7747
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
yangsheng [Wed, 8 Jan 2014 11:44:15 +0000 (19:44 +0800)]
LU-4416 utils: loop_control device support
FC19 use loop_control device since 3.12.5 kernel.
Signed-off-by: yang sheng <yang.sheng@intel.com>
Change-Id: I40b241243507648b7dd9c6ddaf924c2324396e3f
Reviewed-on: http://review.whamcloud.com/8799
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
John L. Hammond [Wed, 11 Dec 2013 18:08:17 +0000 (12:08 -0600)]
LU-2675 obdclass: remove uses of lov_stripe_md
Remove the unused function llog_obd_add(). Remove the unused count and
lsm parameters from llog_cancel(). Move dump_lsm() from obdclass to
the only module that uses it (lov). Remove obd_lov.h.
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I38a5fdae210b6317a07d1601c1c93ae8071631ae
Reviewed-on: http://review.whamcloud.com/8545
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
James Simmons [Tue, 21 Jan 2014 17:06:59 +0000 (12:06 -0500)]
LU-3319 lprocfs: client side cleanups
Now that all the client side seq_file patches it is
time to handle the issue that people pointed out but
were not severe enough to prevent landing. This patch
addresses all the concerns as well and move all struct
lprocfs_seq_var to be initialized C99 style.
Change-Id: I89e8b719bd067ecf4e3cab481a2d4c62d5052af0
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/8943
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Ryan Haasken [Wed, 22 Jan 2014 19:34:15 +0000 (13:34 -0600)]
LU-4129 libcfs: Only dump log once per sec. to avoid EEXIST
Since the log file name contains the current time in seconds, dumping
the logs more than once per second causes EEXIST errors to be emitted.
Add a static variable to libcfs_debug_dumplog_internal that records
the time of the last Lustre log dump. If the current time in seconds
is equal to the last time, do not dump logs again.
Note that this is not thread-safe. However, in the rare case that two
threads try to access last_dump_time simultaneously, the worst thing
that could happen is that one of the threads will get an EEXIST error
when trying to write the log file. This is no worse than the current
situation, and it is not likely to happen.
Signed-off-by: Ryan Haasken <haasken@cray.com>
Change-Id: I7345635ab84333d6c1b455de6059d9d72e5a88f5
Reviewed-on: http://review.whamcloud.com/8964
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Andreas Dilger [Fri, 24 Jan 2014 05:07:55 +0000 (22:07 -0700)]
LU-4522 llite: quiet console permission error messages
Quiet some common console error messages for permission errors
that can be hit in common cases.
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I8ef2eec8c466e41bdaf08779440a186e442540e5
Reviewed-on: http://review.whamcloud.com/8988
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Li Xi [Wed, 29 Jan 2014 10:47:15 +0000 (02:47 -0800)]
LU-4199 llite: Init lli_lmv_md field of struct ll_inode_info
lli_lmv_md field of structure ll_inode_info is not inited to NULL,
which causes lmv_read_entry() to crash the kernel when trying to
access it.
Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I84e932a4246b4637d5ba240ea9193c8b347d62e4
Reviewed-on: http://review.whamcloud.com/9042
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
James Simmons [Thu, 13 Feb 2014 01:06:40 +0000 (20:06 -0500)]
LU-4199 libcfs: Handle nodemask on UMP machines
For UMP and SMP machines the struct cfs_cpt_table are
defined differently. In the case handled by this patch
nodemask is defined as a integer for the UMP case and
as a pointer for the SMP case. This will cause a problem
for ost_setup which reads the nodemask directly. Instead
we create a UMP version of cfs_cpt_nodemask and use that
in ost_setup.
Change-Id: I0dd80f79cf4589fc98182786d3f970432388f1fe
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/9219
Tested-by: Jenkins
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Li Xi <pkuelelixi@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Dmitry Eremin [Fri, 14 Feb 2014 17:23:56 +0000 (21:23 +0400)]
LU-4629 liblustre: remove suspicious check
Suspicious dereference of pointer 'inode' by passing argument 1 to
function 'llu_i2info' at line 387 before NULL check at line 391.
The pointer 'inode' can not be NULL. This checked on upper layer.
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Iea1e4d46f9faccf74a895002a8640959d63c395a
Reviewed-on: http://review.whamcloud.com/9324
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Dmitry Eremin [Tue, 25 Feb 2014 20:51:45 +0000 (00:51 +0400)]
LU-4629 lmv: fix issue found by Klocwork Insight tool
'plock.cookie' might be used uninitialized in this function.
sscanf format specification '%d' expects type 'int *' for 'd',
but parameter 3 has a different type '__u32*'
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ie800e69f345e815dc7437b89707cbaabac1af5f7
Reviewed-on: http://review.whamcloud.com/9390
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Dmitry Eremin [Fri, 21 Feb 2014 13:55:55 +0000 (17:55 +0400)]
LU-4629 gss: fix incorrect memset size
Function 'memset' is called for 'sizeof' applied to pointer
instead of 'sizeof' applied to pointed type.
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Icd1583cb2de90617f6d96baa5d39d28c0d5167ae
Reviewed-on: http://review.whamcloud.com/9345
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Dmitry Eremin [Fri, 21 Feb 2014 12:33:30 +0000 (16:33 +0400)]
LU-4629 utils: fix rsources leak
Resource acquired to 'fp' at line 3540 may be lost here.
Resource acquired to 'debug_log' at line 1690 may be lost here.
Resource acquired to 'mattr' at line 471 may be lost here.
Resource acquired to 'conn->oc_fd' at line 208 may be lost here.
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I47a1c44d8b1cfd8500ca71958814b8a23d6fff2b
Reviewed-on: http://review.whamcloud.com/9343
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Dmitry Eremin [Thu, 20 Feb 2014 10:17:38 +0000 (14:17 +0400)]
LU-4629 gss: fix uninitialized variable
'sg->page_link' is used uninitialized in many functions.
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I7f6d2870a8e95ce0614d617d08cf12f657a680ca
Reviewed-on: http://review.whamcloud.com/9325
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Swapnil Pimpale [Thu, 26 Dec 2013 10:37:43 +0000 (16:07 +0530)]
LU-3831 llite: Remove touch_atime() call in ll_readdir()
In 2.6.32 vfs_readdir() calls file_accessed() which checks for
O_NOATIME and calls touch_atime() appropriately. In 3.11, the same is
true of iterate_dir(). Hence removed the call to touch_atime() in
ll_readdir(). Also, removed the HAVE_TOUCH_ATIME_1ARG and
HAVE_F_PATH_MNT autocrud since ll_readdir() was the last caller of
touch_atime() in the lustre tree.
Signed-off-by: Swapnil Pimpale <spimpale@ddn.com>
Change-Id: Iedd3be29adae80427d71a7eb5a0f8c0d1f1d2924
Reviewed-on: http://review.whamcloud.com/8664
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Michael MacDonald [Tue, 25 Feb 2014 00:56:36 +0000 (19:56 -0500)]
LU-4643 hsm: Make sanity-hsm test_60 more robust
The first version of this test was fragile and could fail
intermittently when test infrastructure was not capable of
providing 1MB/sec in lustre bandwidth. This commit changes the
test to validate that a progress update occurs within the expected
window, rather than testing for a specific amount of data copied
under ideal conditions.
Signed-off-by: Michael MacDonald <michael.macdonald@intel.com>
Change-Id: Ief40a27452bc09d94101c7df00a46aa171a572f8
Reviewed-on: http://review.whamcloud.com/9376
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Lai Siyao [Mon, 24 Feb 2014 10:13:35 +0000 (18:13 +0800)]
LU-4659 mdd: rename forgets updating target linkea
MDD forgets updating target LinkEA if target is not unlinked, and
this will cause LinkEA inconsistency.
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I92b9019b9517513d0cf3f92cf09e7a546cee6027
Reviewed-on: http://review.whamcloud.com/9366
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
wang di [Thu, 20 Feb 2014 12:17:35 +0000 (04:17 -0800)]
LU-4516 mdc: missing lexxx_to_cpu in mdc_read_entry
A few missing lexx_to_cpu on some accesses to lu_dirpage/lu_dirent
in MDC.
Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I50d9d8916004ef460bd7a49376306e2d6bda2b99
Reviewed-on: http://review.whamcloud.com/9331
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Joshua Walgenbach [Tue, 11 Feb 2014 00:36:53 +0000 (01:36 +0100)]
LU-3527 nodemap: idmap management functions
This adds support for uid/gid mapping management to the nodemap
module. This mapping is added as four rbtrees in the nodemap
structure for forward and reverse mapping of UIDs and GIDs.
Postorder traversal for rbtrees have been added for cleanup
upon nodemap removal.
lctl commands have been added for adding, removing, and testing
maps.
Unit tests have been added to test the maps in all the relevant
modes (allow root access, trust client ids).
Signed-off-by: Joshua Walgenbach <jjw@iu.edu>
Change-Id: I9a8672f2883185a35d3553e5c2e91ae1d2cb094d
Reviewed-on: http://review.whamcloud.com/8125
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Reviewed-by: Ken Hornstein <kenh@cmf.nrl.navy.mil>
Joshua Walgenbach [Fri, 17 Jan 2014 21:21:31 +0000 (22:21 +0100)]
LU-3527 nodemap: add NID range management
The nodemap module defines a nodemap as a set of NID ranges
and a policy that is applied to them. This adds the range
management functions to nodemap. The definition of a range
is kept in the range_node structure.
The range_node structures are kept in a global rb_tree for
ease and speed of managing the range_nodes (each range_node
can be a member of eexactly one nodemap). The range_node
contains a pointer back to the nodemap of which it is a
member. The range_node also keeps the (inclusive) start and
end of the NID range is represents.
Additionally, each nodemap contains a linked list of the
pointers to the range_node in encompasses, for use in
management of the nodemap (deleting, checking ranges, etc)
to prevent walking the range_node tree.
Commands have been added to lctl to control the range management
and other nodemap policy features (squash ids, admin, trusted)
that will control its behavior with regards to ID mapping.
Unit tests have been added to sanity-sec.sh to check the
range management and policy controls.
Signed-off-by: Joshua Walgenbach <jjw@iu.edu>
Change-Id: I2b93f15de8f07a2bcef16fde27fb03a41a2063bc
Reviewed-on: http://review.whamcloud.com/8057
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Ken Hornstein <kenh@cmf.nrl.navy.mil>
wang di [Thu, 9 Jan 2014 13:30:36 +0000 (05:30 -0800)]
LU-2430 mdt: Add global rename lock.
Add global rename lock to protect migration process.
Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I363ab58150350e5bc558fcd655c5e7e7b653af10
Reviewed-on: http://review.whamcloud.com/8826
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Andrew Korty [Mon, 9 Dec 2013 23:14:51 +0000 (15:14 -0800)]
LU-4371 gss: Generalize testing of GSSAPI code
Rename sanity-gss.sh to sanity-krb5.sh and make sanity-gss.sh a
mechanism-agnostic test suite for just the GSSAPI code itself using
the gssnull security flavor, which in turn uses the null GSSAPI
mechanism.
Test-Parameters: testlist=sanity-gss
Signed-off-by: Andrew Korty <ajk@iu.edu>
Change-Id: I149ce66c4b044e20b9e8e1b93058ec4f62aba881
Reviewed-on: http://review.whamcloud.com/8525
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
wang di [Sun, 10 Aug 2014 09:58:24 +0000 (02:58 -0700)]
LU-2430 mdd: add lfs mv to migrate inode.
Add lfs mv to migrate the individual inode from one MDT to
another MDT, and this function will only migrate inode layout
on MDT but not touch data object on OST.
lfs mv -M 1 /mnt/lustre/test1 #move test1 to MDT1.
The directory will be migrated from top to the bottom, i.e.
migrating parent first, then migrating the child.
Add migrate into sanity 17n to check on-disk format.
Add sanity 230(c,d,e,f) sanityn 80 to for migration.
Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ib4456a1db8909bd96260c67fff48922081948dcd
Reviewed-on: http://review.whamcloud.com/6662
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Li Wei [Mon, 22 Jul 2013 14:35:45 +0000 (22:35 +0800)]
LU-3539 protocol: Change UPDATE_OBJ RPC format
This patch updates struct update for DNE phase II. See the DNE phase
II HLD for more information.
The comment describing the UPDATE_OBJ RPC format is already a bit out
of date. This patch simplies and updates it so that the overall
structure is easier to see while the details are commented in-line.
The related type names are modified to be more consistent with the
terminology used in the HLD and the comment.
Change OBJ_XXX-->OUT_XXX, and refine a few update structures to
object_update_xxx. And there are no logic changes in this patch.
Change-Id: I4355b4c863abb448713115f17b80402ec96009f8
Signed-off-by: Di Wang <di.wang@intel.com>
Signed-off-by: Li Wei <wei.g.li@intel.com>
Reviewed-on: http://review.whamcloud.com/7128
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Fan Yong [Wed, 12 Feb 2014 20:49:45 +0000 (04:49 +0800)]
LU-4604 lfsck: LFSCK async updates RPC flow control
Control the max in flight async updates RPCs count in OSP layer to
avoid too much in flight RPCs causes memory issue and OST overload.
Currently, we use a semaphore for that which is initialized as 512.
It also contains the layout LFSCK performance tests:
1) lfsck_layout performance under single MDS mode with kinds of
stripe counts and kinds of file sets, for routine check case
and repairing case.
2) lfsck_layout performance under DNE mode with kinds of file
sets, for routine check case and repairing case.
3) lfsck_layout impact on create performance.
This patch also fixes some bugs related with lu_device_type
operations smooth the lfsck performance test.
There was no protection when inc/dec lu_device_type::ldt_device_nr,
which may caused the ldt_device_nr to be wrong and trigger assert.
This patch redefine lu_device_type::ldt_device_nr as atomic type.
There was no protection when add/del lu_device_type::ldt_linkage
into/from the global lu_device_types list, which may caused bad
address accessing. This patch uses the existing obd_types_lock
to protect related operations.
We do NOT need lu_types_stop() any longer. Such function scans
the global lu_device_types list, and for each type item on it
which has zerod lu_device_type::ldt_device_nr, call its stop()
method. In fact, the lu_device_type::ldt_device_nr only will be
zero when the last lu_device_fini() is called, and at that time,
inside the lu_device_fini(), its stop() method will be called.
So it is unnecessary to call the stop() again via lu_types_stop().
Test-Parameters: mdtcount=2 ostcount=2 envdefinitions=SLOW=yes testlist=lfsck-performance
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ib9bae6ad0446e8705fa2767d080150e82a495e2f
Reviewed-on: http://review.whamcloud.com/8694
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Fan Yong [Wed, 12 Feb 2014 19:35:39 +0000 (03:35 +0800)]
LU-3336 lfsck: recreate the lost MDT-object
If the MDT-object does not exist but related OST-obejct still
alive, then when handle the orphan OST-object, the MDT-object
will be re-created under /.lustre/lost+found/MDTxxxx with the
given OST-object stripe and owner information. The other part
of the MDT-obejct layout EA will be empty which may be filled
by other orphan OST-object(s) as the LFSCK processing.
The name for the above re-created MDT-object will be named as
${MOUNT_POINT}/.lustre/lost+found/MDTxxxx/N-${FID}, the "FID"
is the MDT-object's fid. If the OST-object has zero PFID (for
some reason), then LFSCK will assign new FID to the parent.
If the MDT-object exists, but related layout EA slot is occupied
by other OST-object which may be created by former LFSCK running
for repairing dangling referenced MDT-object or may be not.
For the latter case, the LFSCK will create a new MDT-object with
the given OST-object under /.lustre/lost+found/MDTxxxx with name
"C-${FID1}-${FID2}-${idx}":
FID1: the new MDT-object's fid.
FID2: the original MDT-object's fid.
idx: index in the layout EA.
For the former one, then means the former LFSCK guess was wrong.
If such new created OST-object has never been modified which can
be indicated with lu_attr::la_ctime is zero and lu_attr::la_mode
has no "S_ISUID" set, then the LFSCK will destroy it and replace
it with the orphan OST-object; otherwise the LFSCK will keep the
new data, and create a new MDT-object to reference the orphan as
describe above with name "C-${FID1}-${FID2}-${idx}".
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ia7700f7b2e4e09d6b576d025295fdb5c10ca90e0
Reviewed-on: http://review.whamcloud.com/7811
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Fan Yong [Wed, 12 Feb 2014 19:35:18 +0000 (03:35 +0800)]
LU-3336 lfsck: namespace visible lost+found directory
The LFSCK will create the lost MDT-objects under a namespace visible
directory, then the admin can future handle those orphans with human
knowledge later.
The directory is named as "lost+found" under ".lustre", which is not
the same as backend (for ldiskfs case) "/lost+found". The latter one
is invisible from Lustre namespace. Under ".lustre/lost+found/", the
LFSCK will create sub-directory "MDTxxxx" for each MDT. (xxxx is the
MDT number)
The ".lustre/lost+found" uses special FID:
enum dot_lustre_oid {
FID_OID_DOT_LUSTRE = 1UL,
FID_OID_DOT_LUSTRE_OBF = 2UL,
FID_OID_DOT_LUSTRE_LPF = 3UL,
};
const struct lu_fid LU_LPF_FID = { .f_seq = FID_SEQ_DOT_LUSTRE,
.f_oid = FID_OID_DOT_LUSTRE_LPF,
.f_ver = 0x0000000000000000 };
Both ".lustre" and its child "lost+found" reside on the MDT0. As for
the "lost+found"'s children "MDTxxxx" uses normal FID and resides on
the corresponding MDT. The orphans will be linked under the "MDTxxx"
by the LFSCK on such MDT when it find the orphans.
The ".lustre/lost+found" is created by the MDT0 when mdd_prepare().
The ".lustre/lost+found/MDTxxxx" is created by the LFSCK on related
MDT when it needs to process orphans. (create on demand)
The permission for ".lustre/lost+found" is "r-x------" for root user.
Only the LFSCK can create sub-directories "MDTxxxx" under it. All the
other modifications are denied.
The permission for ".lustre/lost+found/MDTxxxx" is "rwx------" for
root user. Any modification is the same as operated on normal file.
The lost+found mechanism is not only for LFSCK phase II but also for
other LFSCK orphan handling. Since it is visible to Lustre namespace,
the admin can operate the orphans as other normal files easily.
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Idd37c87c2517175022127a2a605ebb3ddf1f223a
Reviewed-on: http://review.whamcloud.com/8305
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Fan Yong [Wed, 12 Feb 2014 19:32:41 +0000 (03:32 +0800)]
LU-3336 lfsck: regenerate lost layout EA
To find out orphan OST-objects, the LFSCK on OST side maintains
two bitmaps in RAM for the OST-object accessed during the LFSCK.
After the first cycle system scanning, the LFSCK got the bitmap
for the known OST-objects, and got another bitmap for which OST
objects have been referenced by MDT-objects. Then the LFSCK can
know which OST-objects are not referenced by any MDT-object via
comparing the two bitmaps.
If the MDT-object exists, then check the layout EA. If related
layout EA is empty, then fill the layout EA slot with the given
OST-object stripe information. If the given OST-object stripe
index exceeds current layout EA size, then extend the layout EA
and fill the gap slot(s) as empty, which can be filled by others.
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ie542289b397576be457b04792e845324d6926836
Reviewed-on: http://review.whamcloud.com/7810
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Fan Yong [Wed, 12 Feb 2014 09:21:32 +0000 (17:21 +0800)]
LU-3336 lfsck: orphan OST-objects iteration
During the second stage scanning, the LFSCK on the MDT(s) will scan
the orphan OST-objects via OSP level iteration which fetches remote
orphan OST-objects information via OBD_IDX_READ RPC, and shares the
existing framework/functions with others, such as quota.
Implement the sponsor (the master LFSCK engine on the MDT) logic
for the orphan OST-objects iteration.
Implement LFSCK layout rbtree iteration - lfsck_orphan_index_ops,
for slave LFSCK on OST. The lfsck_orphan_index_ops is registered
onto the rbtree object. The incoming OBD_IDX_READ RPC for orphan
OST-object scanning will iterate the rbtree via dt_index_read to
call the registered lfsck_orphan_index_ops.
Others:
1) Speed control during the second-phase scanning.
2) The LFSCK layout trace file (on the MDT) flags should be set
with LF_INCOMPLETE if LFSCK slave on OST restart or failed.
3) Some code cleanup.
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I67d5d870dbf9b80530f4d61ed1a3e5b5df70b1a0
Reviewed-on: http://review.whamcloud.com/8303
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Ann Koehler [Wed, 12 Feb 2014 17:14:00 +0000 (01:14 +0800)]
LU-4357 libcfs: restore __GFP_WAIT flag to memalloc calls
In 2.4, the flags passed to the memory allocation functions are
translated from CFS enumeration values types to the kernel GFP
values by calling cfs_alloc_flags_to_gfp(). This function adds
__GFP_WAIT to all flags except CFS_ALLOC_ATOMIC. In 2.5, when
the cfs wrappers were dropped, cfs_alloc_flags_to_gfp() was
removed and the CFS_ALLOC_xxxx was simply replaced with __GFP_xxxx.
This means that most memory allocation calls are missing the
__GFP_WAIT flag. The result is that Lustre experiences more ENOMEM
errors, many of which the higher levels of Lustre do not handle
robustly.
Notes GFP_NOFS = __GFP_WAIT | __GFP_IO. So the patch replaces
__GFP_IO with GFP_NOFS.
Patch does not add __GFP_WAIT to GFP_IOFS. GFP_IOFS was not used in
2.4 so it has never been used with __GFP_WAIT.
Signed-off-by: Ann Koehler <amk@cray.com>
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Ib241b39674129a27fea53c23c8ce3e74d165372a
Reviewed-on: http://review.whamcloud.com/9223
Tested-by: Jenkins
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Jian Yu [Wed, 12 Feb 2014 15:48:58 +0000 (23:48 +0800)]
LU-4460 mount: fix lmd_parse() to handle comma-separated NIDs
This patch reverts commit
3917e62018878dfffac59ceed70f20b0419945d3,
which cannot handle the upgrade situation that old mountdata already
contains comma-separated NIDs. The correct way to fix the original
issue is to parse comma-separated NIDs in lmd_parse().
The patch also updates disk2_4-ldiskfs.tar.bz2 to make the mountdata
of ost contain comma-separated NIDs so as to verify the patch under
upgrade situation.
Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes testlist=conf-sanity
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: If179618c9c89dc2168f748aeba59384ea31197ff
Reviewed-on: http://review.whamcloud.com/8918
Tested-by: Maloo <hpdd-maloo@intel.com>
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Sebastien Buisson <sebastien.buisson@bull.net>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Patrick Farrell [Wed, 22 Jan 2014 19:04:26 +0000 (13:04 -0600)]
LU-4471 mdd: mdd_unlink: do trans_start after sanity check
Currently, mdd_trans_start is called before
mdd_unlink_sanity_check. This means a remote directory
which has files in it can be removed on MDT0 before the
sanity check on MDT1 finds the files and errors, which
orphans the files on MDT1. This patch moves the sanity
check before mdd_trans_create and mdd_trans_start.
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I08882b682b9f0016577214821efec4759ee5c184
Reviewed-on: http://review.whamcloud.com/8827
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Oleg Drokin [Tue, 25 Feb 2014 17:44:39 +0000 (12:44 -0500)]
New tag 2.5.56
Change-Id: I9f095e4102fc46f7ed9829a6056ed83c0ac3f692
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
git://git.whamcloud.com / fs / lustre-release.git / log