Whamcloud - gitweb
fs/lustre-release.git
6 years agoLU-5285 mdt: reconstruct to handle non-existing objects 25/11025/3
Alex Zhuravlev [Wed, 9 Jul 2014 13:50:11 +0000 (17:50 +0400)]
LU-5285 mdt: reconstruct to handle non-existing objects

it's possible that reconstruction finds the object doesn't exist
because the original transaction was committed and the rep-ack
lock got released letting another clients to destroy the object.
we should handle this propely - return attributes to indicate the
object has been unlinked (nlink=0

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I3bc18ff0cdb3b7f6d28e47604cf3e142e3f5bc42
Reviewed-on: http://review.whamcloud.com/11025
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4971 ldlm: drop redundant ibits lock interoperability check 04/11004/4
Fan Yong [Mon, 2 Jun 2014 12:17:51 +0000 (20:17 +0800)]
LU-4971 ldlm: drop redundant ibits lock interoperability check

In very old release (older than Lustre-1.8), if the client talks with
the server that does not support ibits lock, then the client needs to
convert it as plain lock. Such interoperability check and convertion
is out of date for a long time. Drop it.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I635df2767ed3b859d93b7ff0680e30ef07d94747
Reviewed-on: http://review.whamcloud.com/11004
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5266 ldlm: granting the same lock twice on recovery 03/10903/2
Vitaly Fertman [Mon, 30 Jun 2014 19:36:58 +0000 (23:36 +0400)]
LU-5266 ldlm: granting the same lock twice on recovery

ASSERTION( lock->l_granted_mode != lock->l_req_mode ) is hit
when resending LDLM_FL_REPLAY.

Don't add lock to waiting list as it is already added or granted.

Change-Id: Ib8e5d2c7588f6cacd1723529e70d29f63742caad
Xyratex-bug-id: MRP-1944
Signed-off-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Reviewed-on: http://review.whamcloud.com/10903
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
6 years agoLU-5317 build: fix build of Xeon Phi client RPMs 44/11044/2
Dmitry Eremin [Thu, 10 Jul 2014 14:10:00 +0000 (18:10 +0400)]
LU-5317 build: fix build of Xeon Phi client RPMs

* disable DKMS build for Xeon Phi
* resolve issue with undefined reference to `libcfs_log_return'

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I383ca771fdd1d2cd22cc20a20d69b32af7e6e010
Reviewed-on: http://review.whamcloud.com/11044
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5310 build: fix build error in Centos 7 35/11035/2
Bob Glossman [Wed, 9 Jul 2014 21:03:18 +0000 (14:03 -0700)]
LU-5310 build: fix build error in Centos 7

Fix a code flaw added to master by recent commit
de2d5808bd2987f76d2486272e1a9c192ba277d4.
Causes a build failure in newer version gcc v4.8.2 used in Centos 7.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Icc0aa1994ece756cd70bc5d69101cbe1694f8df0
Reviewed-on: http://review.whamcloud.com/11035
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
6 years agoLU-5296 lod: don't skip attr_set for osp objects 89/10989/4
Niu Yawei [Fri, 4 Jul 2014 07:34:59 +0000 (03:34 -0400)]
LU-5296 lod: don't skip attr_set for osp objects

lod_attr_set() uses dt_object_exists() to check whether an osp
object (for OST object) is existing, which is incorrect for non-dir
object, and can lead to chown/chgrp being skipped on OST object.

The lsr_valid handling in osp_sync_add_rec() was got a problem in
commit 80f90fcde73e because it stored all of the passed attr flags
in struct llog_setattr64_rec, even though there are only fields for
storing the UID and GID.  Since the time stamps do not need to be
propagated to the OSTs (the MDT values are good enough), this is
not a problem.  Also, osp_sync_add_rec stored LA_UID/LA_GID flags
on disk, but they are not part of lustre_idl.h or lustre_disk.h and
are not guaranteed to be constant over time.  Instead, use the flags
OBD_MD_FLUID/OBD_MD_FLGID that are in lustre_idl.h.

Re-enable sanity-quota.sh test_34 so that this is actually tested.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I245e59141324b34071c9eb188dedbf615052f19d
Reviewed-on: http://review.whamcloud.com/10989
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4621 ofd: create same count of objects 26/10626/7
Hongchao Zhang [Sat, 24 May 2014 07:53:28 +0000 (15:53 +0800)]
LU-4621 ofd: create same count of objects

when replaying the creation request (precreation) from OSP,
OFD should create the same count of objects as it created
when it processed the request initially, else it could cause
"-ENOENT" problem for the following replay requests coming
from the clients.

Change-Id: I64f61f82f6a662ccc564599468466549a55b55d0
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes \
clientdistro=el6 ossdistro=el6 mdsdistro=el6 clientarch=x86_64 \
ossarch=x86_64 mdsarch=x86_64 \
clientcount=4 osscount=2 mdscount=2 \
austeroptions=-R failover=true iscsi=1 \
testlist=recovery-mds-scale
Reviewed-on: http://review.whamcloud.com/10626
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4478 ldiskfs: fix problem when ldiskfs_acct_on() fails 38/10938/2
Li Xi [Wed, 2 Jul 2014 04:15:03 +0000 (12:15 +0800)]
LU-4478 ldiskfs: fix problem when ldiskfs_acct_on() fails

ldiskfs_fill_super() does not handle the failure ldiskfs_acct_on()
returns correctly. This patch fixes the problem for RHEL6.5 kernel.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: Id6e9af30ac81b54c2873b8c2f2a8efd398398a8d
Reviewed-on: http://review.whamcloud.com/10938
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
6 years agoLU-4975 ofd: add comments for ofd_grant.c functions 86/10586/4
Johann Lombardi [Wed, 4 Jun 2014 14:38:47 +0000 (16:38 +0200)]
LU-4975 ofd: add comments for ofd_grant.c functions

- Fix up GPL header block to reference proper GPLv2 license URL.
- Remove mention of contacting Sun.
- Add introductionary comment block for the ofd_grant.c file.
- Add function comments blocks to all functions in the ofd_grant.c
  file

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: I51400354ae51a2d989f136c7dad389d88193a2e5
Reviewed-on: http://review.whamcloud.com/10586
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Richard Henwood <richard.henwood@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
6 years agoLU-4993 libcfs: convert nodemask_t to linux bitmask 32/10332/6
James Simmons [Fri, 27 Jun 2014 22:05:04 +0000 (18:05 -0400)]
LU-4993 libcfs: convert nodemask_t to linux bitmask

Starting in 3.14 kernels nodemask_t was changed from a
a unsigned long to a linux bitmap so more than 32 cores
could be supported. Using set_bit in cfs_cpt_table_alloc
no longer compiles so this patch backports bits of the
node management function that use a linux bitmap back
end. Cleaned up libcfs bitmap.h to use the libcfs layers
memory allocation function. This was pulling in lustre
related code that was not defined.

Change-Id: I7d08425bc09af2c3c04f7e5ed7d50e7c27d165df
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/10332
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4871 newline: Correct missing newline 00/10000/6
James Nunez [Thu, 17 Apr 2014 20:29:01 +0000 (14:29 -0600)]
LU-4871 newline: Correct missing newline

Several error messages are missing newline characters
at the end of the message. Newlines are added where
necessary and other minor corrections; no punctuation
at the end of an error message, add a return code to
the end of error messages, device name at the beginning,
etc.

There are just a couple of places where newlines are
removed and this is only in LDLM_DEBUG_NOLOCK. The definition
of LDLM_DEBUG_NOLOCK already has a newline in it and
resulted in double newlines printed.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I562a76a03753cfc8fd7bb822713280995d9cce45
Reviewed-on: http://review.whamcloud.com/10000
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Cliff White <cliff.white@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
6 years agoLU-5099 api: transfer object type via dt_insert API 47/10447/22
Fan Yong [Fri, 30 May 2014 04:06:29 +0000 (12:06 +0800)]
LU-5099 api: transfer object type via dt_insert API

When the LFSCK finds a dangling name entry, it may be required to
re-create the lost remote MDT-object. The LFSCK needs to know the
file type for the re-creating.

Generally, the LFSCK can get the file type from the name entry.
Unfortunately, when we insert the name entry into the directory
via dt_insert(), the sponsor does not transfer the object type.
It is not a problem for local case, because the OSD can get the
file type internally via checking the MDT-object directly. But
for remote case, when the OSD handles insert locally for remote
object, it cannot get the remote object type. On the other hand,
in DNE I, only directory can be remote, the OSD can assume that
the file type is S_IFDIR. But such assumption will be wrong when
enable remote object for normal file and will misguide the LFSCK
to re-create worng MDT-object.

This patch enhances the dt_insert() and dt_declare_insert() API
parameter to allow the insert operation sponsor to transfer the
file type directly.

The patch also changes the wire protocol for OUT_INDEX_INSERT
to transfer file type from one MDT to another.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I26a3b183f8c47a5e834a9708ae49c96d282b0067
Reviewed-on: http://review.whamcloud.com/10447
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5180 lfsck: linkea for orphan 33/10733/15
Fan Yong [Fri, 30 May 2014 03:59:19 +0000 (11:59 +0800)]
LU-5180 lfsck: linkea for orphan

To make the fid2path tools can work against any client visible
object, the LFSCK will generate linkEA not only for the orphan
under the directory /ROOT/.lustre/lost+found/MDTxxxx/ but also
for the objects ".lustre" and ".lustre/lost+found".

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ibf957e8e2ec93d0f8f8a7cf1ac45706fb79a41db
Reviewed-on: http://review.whamcloud.com/10733
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
6 years agoLU-5291 vvp: Make sure ft_flags is valid 56/10956/3
Paul Cassella [Thu, 3 Jul 2014 19:14:47 +0000 (14:14 -0500)]
LU-5291 vvp: Make sure ft_flags is valid

In ll_fault0, the 'fault' struct is mostly cleared before the call to
cl_io_loop, but ft_flags is not reset. It is ordinarily set by
the call to filemap_fault in vvp_io_kernel_fault, but if Lustre
returns before calling filemap_fault, it still has the old value of
ft_flags.

ll_fault0 will then consume the ft_flags field. If it has the
VM_FAULT_RETRY bit set, it will be used as ll_fault0() and
ll_fault()'s return value.

This is a problem when VM_FAULT_RETRY is in ft_flags:
When fault/filemap_fault return with that flag set, they have already
released the mmap semaphore, and do_page_fault does not need to
release it.
Incorrectly returning this flag from ll_fault means mmap_sem
is not upped in the kernel's do_page_fault().

In addition to clearing ft_flags, this patch does not use it unless
it is valid.  It's potentially misleading to return ft_flags in
"fault_ret" if ft_flags has not been set by filemap_fault.

This adds clarity, but does not change the current behavior:
When not valid, ft_flags is replaced by fault_ret, which is zero,
as is ft_flags when not set by filemap_fault.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: If3a7ad87ee94aff5ecc65429c79e0e595281546d
Reviewed-on: http://review.whamcloud.com/10956
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5289 mdc: cleanup intent if mdc_finish_enqueue() fails 63/10963/2
John L. Hammond [Thu, 3 Jul 2014 16:00:11 +0000 (11:00 -0500)]
LU-5289 mdc: cleanup intent if mdc_finish_enqueue() fails

In mdc_enqueue() clear the lock handle, lock mode, and request stored
in the lookup intent if mdc_finish_enqueue() fails.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I233588ab13df86be509a9763e00697d29dee1c7c
Reviewed-on: http://review.whamcloud.com/10963
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
6 years agoLU-4249 quota: race in qsd_upd_thread() 88/10988/2
Niu Yawei [Fri, 4 Jul 2014 03:51:10 +0000 (23:51 -0400)]
LU-4249 quota: race in qsd_upd_thread()

qsd_upd_thread() uses list_for_each_entry_safe() to process
list items one by one, however, it has to drop lock while
processing each item, that'll race with other list processing
thread.

The proper way is to check list head each time when it acquired
lock.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I55dfe3f65cc214c5d0fd8a9afdda795af94bbdce
Reviewed-on: http://review.whamcloud.com/10988
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
6 years agoLU-5223 lmv: build master LMV EA dynamically build via readdir 51/10751/17
Fan Yong [Sun, 25 May 2014 16:34:27 +0000 (00:34 +0800)]
LU-5223 lmv: build master LMV EA dynamically build via readdir

When creating a striped directory, the master object saves the slave
objects (or shards) as internal sub-directories. The sub-directory's
name is composed of ${shard_FID}:${shard_idx}. With the name, we can
easily to know what the shard is and where it should be.

On the other hand, we need to store some information related with the
striped directory, such as magic, hash type, shards count, and so on.
That is the LMV EA (header). We do NOT store the FID of each shard in
the LMV EA. Instead, when we need the shards' FIDs (such as readdir()
on client-side), we can build the entrie LMV EA on the MDT (in RAM) by
iterating the sub-directory entries that are contained in the master
object of the striped directroy.

Above mechanism can simplify the striped directory create operation.
For very large striped directory, logging the FIDs array in the LMV
EA will be trouble. It also simplify the LFSCK for verifying striped
directory, because it reduces the inconsistency sources.

Another fixing is about the lmv_master_fid in master LMV EA header,
it is redundant information, and may become one of the inconsistency
sources. So replace it with two __u64 padding fields.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: If7ddec892b75ff8f6e14d9f3f7a3ae0fa065239e
Reviewed-on: http://review.whamcloud.com/10751
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5095 hsm: Allow receiving messages to be non-blocking 93/10393/9
Frank Zago [Wed, 2 Jul 2014 16:35:18 +0000 (11:35 -0500)]
LU-5095 hsm: Allow receiving messages to be non-blocking

When using the HSM API for a copy tool, it's impossible to
properly exit an application because one thread might be
stuck in llapi_hsm_copytool_recv(), which in turn blocks
forever on a read.

So this patch expands the existing Lustre userspace HSM API
to be able to apply flags to the read side of the
communication pipe, so it can be opened with O_NONBLOCK,
and gives access to the pipe file descriptor used. (read side)

So now a program can do a poll/select, and then call
llapi_hsm_copytool_recv without blocking.

Notes: The flags passed in from the copy tool to
llapi_hsm_copytool_register are applied to the read side of
the pipe using fcntl, because we do not want to apply
the same flags to the read and write sides of the pipe.

No functionality is added for setting flags on the write
side of the pipe, since the kernel expects particular
behavior from that side of the pipe, and it seems unwise
to let user space modify it.

Additionally, this patch changes llapi_hsm_copytool_recv to
not return -EAGAIN when it finds a message for an archive
not serviced by that copytool.

Instead of returning -EAGAIN to the copy tool, which just
restarts the receive operation, this patch just loops inside
the recv function.

This is because this -EAGAIN conflicts with -EAGAIN
returned from the call to libcfs_ukuc_msg_get
which can occur when O_NONBLOCK is set.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I3e611d3c259de9a8c30f3939fb5e48ab88210c2e
Reviewed-on: http://review.whamcloud.com/10393
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5260 llite: removing ll_cl_context when error occurs 55/10955/2
Jinshan Xiong [Wed, 2 Jul 2014 22:46:29 +0000 (15:46 -0700)]
LU-5260 llite: removing ll_cl_context when error occurs

The bug is imported from commit f1b9122c of LU-5108.
It should have removed ll_cl_context from the list if failed to take
lli_write_mutex at:

if ((iot == CIT_WRITE) &&
    !(cio->cui_fd->fd_flags & LL_FILE_GROUP_LOCKED)) {
if (mutex_lock_interruptible(&lli->lli_write_mutex))
GOTO(out, result = -ERESTARTSYS);

Revise it to add ll_context right before cl_io_loop.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I6532bb8919c13692bd0ceb1ba0d755c1417e5095
Reviewed-on: http://review.whamcloud.com/10955
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5130 lod: Do not merge attr for striped directory 41/10841/6
Wang Di [Thu, 26 Jun 2014 06:49:27 +0000 (23:49 -0700)]
LU-5130 lod: Do not merge attr for striped directory

Does not need to merge attributes in lod_attr_get(),
because client will merge attributes itself for striped
directory (see lmv_merge_attr()), and also there are no
logic in MDD will reply on directory nlink/size/time.

Besides lod_attr_get is in the path of object initialization,
mdd_object_start()->mdd_la_get()->lod_attr_get(), if it tries
to get attribute from other MDTs, and if other MDTs are not
ready yet, it will cause unnecessary failures for object
initialization.

Change-Id: I56395a40fcd82142f99228a0776b5b7338c633ac
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10841
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5279 crypto: no vmalloc() for speed test buffer 82/10982/3
Andreas Dilger [Thu, 3 Jul 2014 19:25:56 +0000 (13:25 -0600)]
LU-5279 crypto: no vmalloc() for speed test buffer

Since the patch LU-5053 "libcfs: clean up cfs_crypto_hash code"
(commit 0723b8be34503b31dbfc6c877d7a855091c625b4) was landed the
cfs_crypto_performance_test() code can fail if CONFIG_DEBUG_VIRTUAL
is enabled.  This causes problems in SG handling because it expects
physical addresses:

    kernel BUG at arch/x86/mm/physaddr.c:20!
    RIP: 0010:[<ffffffff81050447>] __phys_addr+0x67/0x80

    sg_init_one+0x36/0x80
    cfs_crypto_hash_digest+0x94/0xf0 [libcfs]
    cfs_crypto_register+0x108/0x330 [libcfs]
    init_libcfs_module+0x0/0x3c0 [libcfs]
    sys_init_module+0xe3/0x260

Don't use vmalloc() to allocate the cfs_crypto_test_hashes()
test buffer.  Use alloc_page() and use cfs_crypto_hash_update_page()
to compute the hash instead of cfs_crypto_hash_digest(), since
tgt_checksum_bulk() computes the hash by page anyway.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I27ec3cf2931b6bf385dd91191abba8955b500c1e
Reviewed-on: http://review.whamcloud.com/10982
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Alexander Boyko <alexander_boyko@xyratex.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4610 tests: Turn on LFSCK debug messages 14/10814/3
James Nunez [Wed, 25 Jun 2014 01:21:00 +0000 (19:21 -0600)]
LU-4610 tests: Turn on LFSCK debug messages

Turn on LFSCK debug messages by default in the
Lustre test framework.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I02490fafbe590fa226faa3b84f37f2c8b4736e5d
Reviewed-on: http://review.whamcloud.com/10814
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
6 years agoLU-4416 osd-ldiskfs: add check for 3 arg ext4_journal_start 64/10164/13
Jeff Mahoney [Mon, 2 Jun 2014 13:05:57 +0000 (09:05 -0400)]
LU-4416 osd-ldiskfs: add check for 3 arg ext4_journal_start

Linux 3.9 added a type argument to ext4_journal_start_sb. The
ldiskfs patches shouldn't need to define whether the
implementation uses this version.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ia764126b27ebc766d39d1778278bdd8835f5a6ef
Reviewed-on: http://review.whamcloud.com/10164
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5259 ptlrpc: request gets stuck in UNREGISTERING phase 46/10846/3
Andriy Skulysh [Thu, 26 Jun 2014 10:37:36 +0000 (13:37 +0300)]
LU-5259 ptlrpc: request gets stuck in UNREGISTERING phase

Exit condition from UNREGISTERING phase is releasing of
both reply and bulk buffers.

Call ptlrpc_unregister_bulk() if ptlrpc_unregister_reply()
wasn't completed in async mode before switching to
UNREGISTERING phase.

Change-Id: Ia0420aecf0391e1a45063add2aa75bb4b3caa5a6
Xyratex-bug-id: MRP-1960
Signed-off-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Reviewed-on: http://review.whamcloud.com/10846
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Ann Koehler <amk@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5238 kernel: kernel update [RHEL6.5 2.6.32-431.20.3.el6] 75/10875/3
Bob Glossman [Fri, 20 Jun 2014 16:20:20 +0000 (09:20 -0700)]
LU-5238 kernel: kernel update [RHEL6.5 2.6.32-431.20.3.el6]

update RHEL6.5 kernel to 2.6.32-431.20.3.el6

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ife0d10b865c4367e2210b0dd28f32ea856b65c74
Reviewed-on: http://review.whamcloud.com/10875
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-3963 osd-zfs: convert to linux list api 98/10898/2
James Simmons [Mon, 30 Jun 2014 17:18:35 +0000 (13:18 -0400)]
LU-3963 osd-zfs: convert to linux list api

Move from the cfs_[h]list api to the native linux api for
all the code related to osd-zfs .

Change-Id: If6cece695c744309e39165086e3ac791fa534ede
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/10898
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4843 mdt: Add OBD_CONNECT_DIR_STRIPE flag 73/10773/7
Wang Di [Sat, 21 Jun 2014 07:59:25 +0000 (00:59 -0700)]
LU-4843 mdt: Add OBD_CONNECT_DIR_STRIPE flag

Add OBD_CONNECT_DIR_STRIPE to tell if the client supports
striped dir, so only new client (>= 2.6) can access striped
directory.

Change-Id: I5d5f1a6a9f5efdce0c6fe22106a8853cd78bb486
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10773
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
6 years agoRevert "LU-1538 tests: keep /sbin/mount.lustre until cleanup" 11/10911/2
Oleg Drokin [Tue, 1 Jul 2014 03:52:55 +0000 (03:52 +0000)]
Revert "LU-1538 tests: keep /sbin/mount.lustre until cleanup"

This change totally breaks my testing complaining about /sbin/mount.lustre being on a readonly filesystem.

This reverts commit 2ade2c4e1481c4bacf49e871cbd299cfc2ccc3b3.

Change-Id: Id8292f7e6328caca540808ab42f885289229f96b
Reviewed-on: http://review.whamcloud.com/10911
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4478 ldiskfs: fix problem when ldiskfs_acct_on() fails 29/8829/4
Li Xi [Tue, 14 Jan 2014 03:55:13 +0000 (11:55 +0800)]
LU-4478 ldiskfs: fix problem when ldiskfs_acct_on() fails

ldiskfs_fill_super() does not handle the failure that
ldiskfs_acct_on() returns correctly. This patch fixes
the problem.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I70afca523cee1d9fe7e0b9c5ffc122fbfd6c34aa
Reviewed-on: http://review.whamcloud.com/8829
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5262 osd: only allow set size for regular file 70/10870/9
Fan Yong [Sun, 25 May 2014 09:51:14 +0000 (17:51 +0800)]
LU-5262 osd: only allow set size for regular file

Otherwise, set size for other non-regular file (such as directory)
will cause the object data corruption.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I44092796961ef26d47ee9c210f52fa451084bd44
Reviewed-on: http://review.whamcloud.com/10870
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4933 osc: Automatically increase the max_dirty_mb 46/10446/7
Hongchao Zhang [Thu, 22 May 2014 04:26:25 +0000 (12:26 +0800)]
LU-4933 osc: Automatically increase the max_dirty_mb

When RPC size or the max RPCs in flight is increased, the actual
limit might be max_dirty_mb. This patch automatically increases
the max_dirty_mb value at connection time and when the related
values are tuned manually by proc file system.

this patch also changes the unit of "cl_dirty" and "cl_dirty_max"
in client_obd from byte to page.

Change-Id: Ieaea7660caa8008365b38f548b4ecba4027e080b
Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/10446
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
6 years agoLU-5053 libcfs: clean up cfs_crypto_hash code 90/9990/4
Andreas Dilger [Mon, 18 Nov 2013 08:47:44 +0000 (01:47 -0700)]
LU-5053 libcfs: clean up cfs_crypto_hash code

Remove <libcfs/libcfs_crypto.h> from <libcfs/libcfs.h> and only
include it into the places where it is actually needed.  This
works out to be the same places as <obd_cksum.h>, so put it there.

Fix the cfs_crypto_hash_* functions to take enum cfs_crypto_hash_alg
as the algorithm type, instead of an unsigned char.  Return
CFS_HASH_ALG_UNKOWN for unknown hash names instead of "0xFF".

Rename some variables to be consistent across functions.

Add comment blocks for cfs_crypto_hash_*() in linux-crypto.c.  Some
of these functions could be shared with user-crypto.c in a common
crypto.c file, but the code shuffling should be done separately.

Change cfs_crypto_hash_final() to always clean up the hash descrptor
instead of not doing this in error cases.  All of the callers were
just calling cfs_crypto_hash_final() immediately to clean up the
descriptor anyway, and the old behaviour is unlike other init/fini
functions, and prone to memory leaks and other incorrect usage.  The
callers can call cfs_crypto_digest_size() to determine the hash size
in advance if needed, and avoid complexity in cfs_crypto_hash_final().

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I79884eb2ee31a2b375420cf62af5ce8ff22e0e75
Reviewed-on: http://review.whamcloud.com/9990
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4694 hsm: Cleanup codes about return value 55/9455/3
James Nunez [Thu, 15 May 2014 19:46:59 +0000 (13:46 -0600)]
LU-4694 hsm: Cleanup codes about return value

errno migh be zero when llapi_hsm_action_get_fd() returns a failure.
ct_archive() does not handle it correctly, which might cause HSM take
the operation as successful even if it actually fails.

The patch also cleanup other codes about return value.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I625da68836f98346b104533a556c72f06232e3ef
Reviewed-on: http://review.whamcloud.com/9455
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Robert Read <robert.read@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4984 llite: check for integer overflow in hsm user request 15/10615/6
Nathaniel Clark [Thu, 5 Jun 2014 19:59:55 +0000 (15:59 -0400)]
LU-4984 llite: check for integer overflow in hsm user request

Check to make sure total size of request does not overflow when
calculated.  Return -1 from hur_len() if it does overflow.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I6a3ca50c53ff38dab4165f20dac6fa00eb888df2
Reviewed-on: http://review.whamcloud.com/10615
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
6 years agoLU-4018 tests: improve racer file_create workload 51/7951/4
Andreas Dilger [Tue, 15 Oct 2013 16:05:50 +0000 (10:05 -0600)]
LU-4018 tests: improve racer file_create workload

Use larger writes and smaller files, so that the racer file_create.sh
workload does not get bogged down with many threads stuck on sub-page
writes to the same file (causing excruciating 1kB-per-RPC sync writes
to disk on the OST).

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I173c155dfe4044f89deb8f2d8a7c3be4a28ce763
Reviewed-on: http://review.whamcloud.com/7951
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4921 lmv: try all stripes for unknown hash functions 41/10041/12
Wang Di [Thu, 26 Jun 2014 05:35:52 +0000 (22:35 -0700)]
LU-4921 lmv: try all stripes for unknown hash functions

For unknown hash type, LMV should try all stripes to locate
the name entry. But it will only for lookup and unlink, i.e.
we can only list and unlink entries under striped dir with
unknown hash type.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ifeed7131c24e48277a6cc8fd4c09b7534e31079f
Reviewed-on: http://review.whamcloud.com/10041
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
6 years agoLU-5270 obdclass: do not lock repetitively when geting jobid 90/10890/3
Li Xi [Sat, 28 Jun 2014 02:23:38 +0000 (10:23 +0800)]
LU-5270 obdclass: do not lock repetitively when geting jobid

When trying to get customized job ID, cfs_get_environ() tries to
lock/unlock mmap_sem for two times, which is unnecessary.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: Iae8a45d87508e3c465c0424d5fa54e294d4fb2b1
Reviewed-on: http://review.whamcloud.com/10890
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5227 build: build dkms using lbuild 47/10747/3
Minh Diep [Wed, 18 Jun 2014 18:23:03 +0000 (11:23 -0700)]
LU-5227 build: build dkms using lbuild

To include dkms build in lbuild
1. before build lustre, generate lustre-dkms.spec with configure
2. use rpmbuild to build the src
3. rebuild dkms using the src

Signed-off-by: Minh Diep <minh.diep@intel.com>
Change-Id: Iac71d20110fe0f84b3bfffb071f3e3b84c7f5f17
Reviewed-on: http://review.whamcloud.com/10747
Tested-by: Jenkins
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5240 obdclass: Increase minimum LU Cache for ZFS 02/10802/6
Nathaniel Clark [Tue, 24 Jun 2014 16:14:55 +0000 (12:14 -0400)]
LU-5240 obdclass: Increase minimum LU Cache for ZFS

If the lu cache is too small threads will thrash for data including
the OI tables and other important metadata.

Test-Parameters: mdsfilesystemtype=zfs mdtfilesystemtype=zfs ostfilesystemtype=zfs testgroup=review-zfs
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I1ce7ef9a160c8b028be0d3c9671a0b347cbe3331
Reviewed-on: http://review.whamcloud.com/10802
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Jenkins
6 years agoLU-5188 osp: return 1 if osp_sync_xxx_job issue RPC 28/10828/4
Wang Di [Wed, 25 Jun 2014 23:06:55 +0000 (16:06 -0700)]
LU-5188 osp: return 1 if osp_sync_xxx_job issue RPC

Return 1 if osp_sync_new_xxx_job() issue RPC, so
sp_sync_process_record() can decrease opd_syn_rpc_in_flight
and opd_syn_rpc_in_progress correctly if RPC is not
being sent, otherwise the opd_sync_thread will not be
stopped, caused LBUG (see LU-5244)

osp_sync_thread()) ASSERTION( count < 10 ) failed:

The problem is introduced by

http://review.whamcloud.com/10706

Change-Id: Iaf6dd6c4e1938a0c5d44a40c303eda1695b72105
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10828
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
6 years agoLU-5177 mdt: fix object leak and use after free 50/10750/3
Mikhail Pershin [Thu, 19 Jun 2014 06:35:02 +0000 (10:35 +0400)]
LU-5177 mdt: fix object leak and use after free

The mdt_intent_layout() and mdt_open_by_fid_lock() may exit without
object put and causing object leakage.

The mdt_md_create() passed possibly freed object to the
mdt_create_pack_capa()

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I634052c58ee8595871af987755fda5a9f2c942e1
Reviewed-on: http://review.whamcloud.com/10750
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5233 lod: Remove ldo_def_striping_set check 72/10772/4
Wang Di [Fri, 20 Jun 2014 15:15:04 +0000 (08:15 -0700)]
LU-5233 lod: Remove ldo_def_striping_set check

Only check ldo_stripenr/ldo_stripe_offset before create
striped directory, and ldo_def_striping_set is used to
mark whether the directory has default stripe, which can
be inherited to its children.

Reset ldo_stripenr to 0, if initialization fails, otherwise
it will leave some invalid object in cache.

Change-Id: Iba6985861fb1443bd4e7fcce3cd8c523b31a5940
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10772
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5123 procfs: u64 helper does not respect multiplier 35/10535/5
Stephen Champion [Sat, 31 May 2014 00:35:45 +0000 (17:35 -0700)]
LU-5123 procfs: u64 helper does not respect multiplier

lprocfs_write_frac_u64_helper should apply the multiplier argument to
the value if units are not specified by the user.  It has been
overriding the multiplier with '1' regardless.

Signed-off-by: Stephen Champion <schamp@sgi.com>
Change-Id: Iaacbb6923122479ba1ba4a5ee0088c670245d081
Reviewed-on: http://review.whamcloud.com/10535
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
6 years agoLU-5249 osd: check if FLD is fully intialized 06/10806/2
Wang Di [Tue, 24 Jun 2014 21:41:48 +0000 (14:41 -0700)]
LU-5249 osd: check if FLD is fully intialized

Check if FLD is fully initialized in OSD, before lookup
sequence in the local FLDB.

Change-Id: I07cbf293e617935588ce638f710868a05bdd0943
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10806
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5218 llog: keep llog ctxt indices constant 58/10758/4
Mikhail Pershin [Thu, 19 Jun 2014 19:49:10 +0000 (23:49 +0400)]
LU-5218 llog: keep llog ctxt indices constant

The llog context id table cannot be shrinked easily because that
will cause index shifting and incompatibility between old client
and new server and vice versa.

Patch moves llog_ctxt_id table to the lustre_idl.h because this is
wire protocol data, these values are added to the wirecheck and
checks are added in llog server code for wire data.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I003120223504d95164e4392b01c096e3e8b0fef0
Reviewed-on: http://review.whamcloud.com/10758
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5045 build: update OFED download site to https 55/10755/2
Minh Diep [Thu, 19 Jun 2014 15:21:15 +0000 (08:21 -0700)]
LU-5045 build: update OFED download site to https

OFED download site has changed to use https.

Signed-off-by: Minh Diep <minh.diep@intel.com>
Change-Id: Icb26444db5a3c239f595efb01cb03b1c1dd3e6de
Reviewed-on: http://review.whamcloud.com/10755
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5223 lod: generate linkEA for shard of striped directory 65/10765/2
Fan Yong [Thu, 15 May 2014 20:09:12 +0000 (04:09 +0800)]
LU-5223 lod: generate linkEA for shard of striped directory

Otherwise, it will misguide namespace LFSCK to regard it as the case
of linkEA corrupted or upgraded from 1.8 device, then cause improper
reparing or status reporting.

On the other hand, the linkEA for slave object of striped directory
will also be used for fid2path in subsequent patches.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I877b091eb330df0283de2ea21f2796387023da30
Reviewed-on: http://review.whamcloud.com/10765
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
6 years agoLU-5062 llite: Solve a race to access lli_has_smd in read case 60/10760/3
Jinshan Xiong [Thu, 19 Jun 2014 23:21:09 +0000 (16:21 -0700)]
LU-5062 llite: Solve a race to access lli_has_smd in read case

In vvp_io_read_lock(), it used to decide if to add read lock by
checking lli_has_smd. Accessing lli_has_smd is racy when an empty
file is turned into raid0, therefore, it may result in read requests
are issued without corresponding lock.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I70cb1888b42e0e9928f8346359e2b1f554fe4e8f
Reviewed-on: http://review.whamcloud.com/10760
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5203 zfs: Update spl/zfs version to 0.6.3 79/8979/14
Nathaniel Clark [Thu, 23 Jan 2014 20:02:51 +0000 (15:02 -0500)]
LU-5203 zfs: Update spl/zfs version to 0.6.3

Update SPL and ZFS version to official 0.6.3.
This also fixes the "search path" for libzfs.so.

Test-Parameters: mdsfilesystemtype=zfs  mdtfilesystemtype=zfs ostfilesystemtype=zfs testgroup=review-zfs
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I2530a7a1f7850e31aa99892f5a3e826baf69d20b
Reviewed-on: http://review.whamcloud.com/8979
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoRevert "LU-4398 mdt: acquire an open lock for write or execute" 90/10790/2
Oleg Drokin [Mon, 23 Jun 2014 18:02:55 +0000 (18:02 +0000)]
Revert "LU-4398 mdt: acquire an open lock for write or execute"

This is causing significant READ performance degradation in mdts: LU-5197

This reverts commit 708d85a652a77f85153790e6cca1b7a2b91947cf.

Change-Id: Ia591bed8a6f098ebad5bb8460c659516bcf24d97
Reviewed-on: http://review.whamcloud.com/10790
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5224 lnet: build failed with MLNX_OFED-2.2 49/10749/4
Wu Libin [Wed, 18 Jun 2014 12:13:14 +0000 (20:13 +0800)]
LU-5224 lnet: build failed with MLNX_OFED-2.2

Build lustre client can fail on the SLES11 SP3 with
MLNX_OFED-2.2 against the kernel-3.0.76-0.11, which will
cause the "'pm_qos_req' has incomplete type" error.This patch
put "#include <linux/compat-2.6.h>" in front of
"#include <net/sock.h>" to solve this problem.

This error come with the configure option
"--with-o2ib=/usr/src/ofa_kernel/default/", it affect the
search path of the compiler, which will search the header
files in this directory before the kernel header files
directory.

This patch change this way because "net/sock.h"(exactly is
"linux/netdevice.h") include header file "linux/pm_qos_params.h",
the pm_qos_params.h is exist both in ofa_kernel include
directory and kernel include directory, so we need to put the
linux/compat-2.6.h in front of net/sock.h to solve the
compatible problem.

Signed-off-by: Wu Libin <lwu@ddn.com>
Change-Id: I98df2a5faaecbd44528d3bd8a7ba796cc1d14cdc
Reviewed-on: http://review.whamcloud.com/10749
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-2675 libcfs: remove unused open flag conversions 42/10742/2
Andreas Dilger [Wed, 18 Jun 2014 06:49:42 +0000 (00:49 -0600)]
LU-2675 libcfs: remove unused open flag conversions

Remove the unused cfs_oflag2univ() and cfs_univ2oflag() conversion
routines.  These were intended to convert O_* open flags to and from
platform-specific flags at the user interface, but are not used
anywhere in the code.  The conversion of the client-specific open
flags to the Lustre protocol flags is done in mds_pack_open_flags()
and there is no need to ever convert back to client-specific flags.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I1f5f7fb7eefa2f8dff161f02b37bee6a9d500c1e
Reviewed-on: http://review.whamcloud.com/10742
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-1095 ofd: Silence 'slow create' warnings 94/10394/3
Brian Behlendorf [Tue, 20 May 2014 22:31:39 +0000 (15:31 -0700)]
LU-1095 ofd: Silence 'slow create' warnings

These messages were originally added to analyze a performance issue.
That issue is now much better understand and no longer needs to
be logged so verbosely.  They probably should have been disabled
prior to Lustre 2.4 being released but this was overlooked.

Change-Id: I395de36cdccc82afd4ed7b9c424d12d17b881f54
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/10394
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-3386 lproc: fix "imports" output format 44/10344/7
Andreas Dilger [Tue, 17 Jun 2014 13:53:41 +0000 (09:53 -0400)]
LU-3386 lproc: fix "imports" output format

There was no comma being printed in the import flags listing since
commit 62660c799b because the "first" marker wasn't being set false.

Fix the output format to put a space around the '[' and ']' list
delimiters.

Make the "unknown" connect flag be a single word.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Iee725b503ec771773d456b3436480c6e52500c1e
Reviewed-on: http://review.whamcloud.com/10344
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5234 llite: restore newer kernel support for ll_read_dir 61/10761/2
James Simmons [Thu, 19 Jun 2014 23:15:27 +0000 (19:15 -0400)]
LU-5234 llite: restore newer kernel support for ll_read_dir

During the merger of the patch for LU-4906 the support for
newer kernels was removed in the new code. This patch
restores this functionality so RHEL7 and SuSE12 clients can
function again.

Change-Id: Ieea4fb23cd644b2d6ce998cfdba8e0e15e481fc5
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/10761
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-3326 tests: sync after file deleted in run_*.sh 71/8071/9
Hongchao Zhang [Tue, 20 May 2014 03:38:52 +0000 (11:38 +0800)]
LU-3326 tests: sync after file deleted in run_*.sh

In run_*.sh, call sync to flush any dirty pages to OST
if MDT has not sent the object destroy request to it,
which will shrink the available disk space gotten before
running the tests and cause -EIO error.

Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Change-Id: Ia246122a5c3471b7cbea9cdaa7fad2ea4db856c5
Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,REQFAIL=100,DURATION=28800 \
clientdistro=el6 ossdistro=el6 mdsdistro=el6 clientarch=x86_64 \
ossarch=x86_64 mdsarch=x86_64 clientcount=4 osscount=2 mdscount=2 \
mdssizegb=10 ostsizegb=10 austeroptions=-R failover=true iscsi=1 \
testlist=recovery-mds-scale
Reviewed-on: http://review.whamcloud.com/8071
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4367 ptlrpc: add OBD_CONNECT_UNLINK_CLOSE flag 26/10426/12
Lai Siyao [Fri, 23 May 2014 03:32:09 +0000 (11:32 +0800)]
LU-4367 ptlrpc: add OBD_CONNECT_UNLINK_CLOSE flag

Add OBD_CONNECT_UNLINK_CLOSE flag for interop, once this is supported,
client packs file handle in unlink RPC, and MDT will close file before
unlink.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: If2f89a5268b95386e01cdc2f8613fd5c4cbd1d2c
Reviewed-on: http://review.whamcloud.com/10426
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5188 osp: Correctly check for invalid setattr record 06/10706/3
Oleg Drokin [Thu, 12 Jun 2014 23:13:25 +0000 (19:13 -0400)]
LU-5188 osp: Correctly check for invalid setattr record

Patch for LU-4345 (commit 80f90fcde73e2faff8b7b0ffc7c19bc52982e027 )
has a correct comment about lsr_valid member being either 0 or
having UID and GID fields set, but the check has a typo causing
it to check for lsr_valid to be both 0 and have the bits set which
is impossible.

The osp_sync_new_setattr_job() should return 0 for invalid record,
so that sync thread can continue processing on other records.

Change-Id: I32e6f261b4707391d6e96f8434aae7bf8f55bf8e
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/10706
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
6 years agoLU-5179 libcfs: do not leak mm_struct 59/10759/2
Oleg Drokin [Thu, 19 Jun 2014 20:20:43 +0000 (16:20 -0400)]
LU-5179 libcfs: do not leak mm_struct

cfs_get_environ might leak an mm_struct if LU-1735 race condition
hits.
Make sure to mmput in such a case.

Change-Id: I49499c14027358961743d8928ab2edbe54bf72f7
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/10759
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
6 years agoLU-2059 mgs: fix a bug in mgs_llog_open 89/10589/2
Mikhail Pershin [Wed, 4 Jun 2014 16:04:04 +0000 (20:04 +0400)]
LU-2059 mgs: fix a bug in mgs_llog_open

- don't calculate len for NULL ptr
- check the logname is PARAMS_FILENAME inside 'if (ptr == NULL ...)
clause so all cases with NULL ptr will end there.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Ib87ecb3245833fc8f9d069898a1b80c44bbfd9f0
Reviewed-on: http://review.whamcloud.com/10589
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4713 test: Add a check to test_237 in sanity.sh 88/10088/3
Swapnil Pimpale [Thu, 24 Apr 2014 17:42:08 +0000 (23:12 +0530)]
LU-4713 test: Add a check to test_237 in sanity.sh

This patch adds a check to test_237 in sanity.sh to check whether
write to the file succeeded or not.

Test-Parameters: clientdistro=sles11sp3 testlist=sanity envdefinitions=ONLY=237
Signed-off-by: Swapnil Pimpale <spimpale@ddn.com>
Change-Id: Id81f9bf01ead516c3b0e7f247ff71550d4a9847e
Reviewed-on: http://review.whamcloud.com/10088
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
6 years agoLU-5124 lod: use S_IS{REG,DIR} in lod_load_striping_locked() 02/10702/2
John L. Hammond [Thu, 12 Jun 2014 19:58:45 +0000 (14:58 -0500)]
LU-5124 lod: use S_IS{REG,DIR} in lod_load_striping_locked()

In lod_load_striping_locked() use S_ISREG(mode) rather than testing
(mode & S_IFREG) and similarly for S_ISDIR().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I040a50b485fc2b45d74daf403c041f05968e573b
Reviewed-on: http://review.whamcloud.com/10702
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
6 years agoLU-5205 mdc: Add config check for Xeon Phi build 34/10734/2
Yang Sheng [Tue, 17 Jun 2014 04:43:26 +0000 (12:43 +0800)]
LU-5205 mdc: Add config check for Xeon Phi build

Add config check for radix_tree_exception_entry()
to not broken Xeon Phi build.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I2f405e392003162e779ffbf9055e410917940df7
Reviewed-on: http://review.whamcloud.com/10734
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5220 llite: fix a flag bug of vvp_io_kernel_fault() 40/10740/4
Li Xi [Wed, 18 Jun 2014 02:24:42 +0000 (10:24 +0800)]
LU-5220 llite: fix a flag bug of vvp_io_kernel_fault()

After vvp_io_kernel_fault() locked the page, it should set
VM_FAULT_LOCKED.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: Ic63f1b830360b127df6ae3220d63bef3ebf0b89c
Reviewed-on: http://review.whamcloud.com/10740
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Jenkins
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4861 osc: a deadlock problem in osc completion ast 81/10581/2
Jinshan Xiong [Wed, 4 Jun 2014 00:34:21 +0000 (17:34 -0700)]
LU-4861 osc: a deadlock problem in osc completion ast

In osc_ldlm_completion_ast(), it tries to hold mutex of cl_lock
and update the lock even it's known that the lock is not granted,
this can introduce a deadlock case if the process is matching a
dlm lock with the calling stack:
  osc_lock_enqueue()
    -> osc_enqueue_base()
      -> ldlm_lock_match()
        -> osc_ldlm_completion_ast()
          -> cl_lock_mutex_get()
The enqueuing lock and the lock for completion_ast() are different
lock. This is a deadlock case because we're holding a lock's mutex
and then request the other.

It's not necessary to acquire the lock in completion_ast() if it's
known that the lock has not been granted.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ia8c0a4cb1e48c9f7fd921052174ce4d8d07713b1
Reviewed-on: http://review.whamcloud.com/10581
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4610 lfsck: fix lfsck, osd-ldiskfs, osd-zfs debug messages 79/10579/6
Fan Yong [Tue, 13 May 2014 14:13:59 +0000 (22:13 +0800)]
LU-4610 lfsck: fix lfsck, osd-ldiskfs, osd-zfs debug messages

Fix the debug subsystems for osd-ldiskfs, and osd-zfs to use S_OSD
instead of S_MDS for easier debug message filtering, except
osd_scrub.c which is using S_LFSCK instead of S_MDS.

Do not print a D_LFSCK message in osd_otable_it_preload(), since this
is being printed for every inode in the filesystem and fills the logs
if D_LFSCK is enabled.

Fix lfsck_pos_dump() to print the directory hash cookie in hex and
fix up sanity-lfsck test_6a to be able to handle this.

More D_LFSCK debug message for LFSCK check/repair processing. This
is the first step for logging all the LFSCK changes. The next step
will collect these LFSCK debug message and save them in some files
automatically.

Enable D_LFSCK debug by default.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Iff3143f22d3c4750ebe95fbeba8883a325300c1e
Reviewed-on: http://review.whamcloud.com/10579
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
6 years agoLU-4887 lfsck: run all LFSCK scan types by default 78/10278/7
Andreas Dilger [Tue, 13 May 2014 13:07:02 +0000 (21:07 +0800)]
LU-4887 lfsck: run all LFSCK scan types by default

When running "lctl lfsck_start" run all available scan types by
default (scrub, layout, and namespace currently) if none is given.
If '-t' is given, run only the specified scan types. 'scrub' is
always run.

Rename the lfsck_type constants to use a LFSCK_TYPE_ prefix to be
more descriptive.  Add LFSCK_TYPE_ constants to wirecheck.
Add ltn_ prefix to lfsck_type_name structure.

Print the available types from the lfsck_types_names[] array in
the help message instead of hard-coding them in the help message.

Don't return an error if "lctl lfsck_stop" is used when stopped.

Fix up lfsck_namespace /proc stats to use more descriptive names.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I106483ade0bf07c2920dd95a263acdbe4e500c1e
Reviewed-on: http://review.whamcloud.com/10278
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
6 years agoLU-4629 utils: fix input format specifier error 67/10667/4
Dmitry Eremin [Tue, 10 Jun 2014 20:09:38 +0000 (00:09 +0400)]
LU-4629 utils: fix input format specifier error

Width is not specified for 's' conversion specifier. This can result in
an overflow of the buffer provided in arguments of a call to 'sscanf'.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ic9f37c19461a28a51bd1a0a1a42685b137773a41
Reviewed-on: http://review.whamcloud.com/10667
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4629 libcfs: fix buffer overflow of string buffer 89/9389/19
Dmitry Eremin [Tue, 25 Feb 2014 18:34:13 +0000 (22:34 +0400)]
LU-4629 libcfs: fix buffer overflow of string buffer

Buffer overflow of string buffer due to non null terminated string.
Use strlcpy() when it's justifiable.
Use sizeof(var) instead of constants.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ib8d5493898a292e3c764170d0a0870bee26b23a0
Reviewed-on: http://review.whamcloud.com/9389
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-3544 xattr: xattr data may be gone with lock held 93/10693/2
Lai Siyao [Wed, 11 Jun 2014 14:44:01 +0000 (22:44 +0800)]
LU-3544 xattr: xattr data may be gone with lock held

Xattr cached data may be gone, but lock still held, in this case,
refetch xattr from server, otherwise client will return error.

Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes testlist=parallel-scale-nfs,parallel-scale-nfs
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Id6fbbef339fea540728b181b2ae91799b8151fbd
Reviewed-on: http://review.whamcloud.com/10693
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-3544 nfs: don't store parent fid 92/10692/2
Lai Siyao [Tue, 3 Jun 2014 08:40:00 +0000 (16:40 +0800)]
LU-3544 nfs: don't store parent fid

It's not necessary to store parent fid in lli_pfid, because MDT
can get it's parent fid from linkea, and now that DNE stripe
directory stores master inode fid in lli_pfid, stop storing parent
fid to avoid conflict.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I924c29a1d58b008026d37265978c81ca1aa21ebb
Reviewed-on: http://review.whamcloud.com/10692
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-3544 fid: do open-by-fid by default 76/7476/23
Lai Siyao [Wed, 28 Aug 2013 07:04:24 +0000 (15:04 +0800)]
LU-3544 fid: do open-by-fid by default

Currently client open-by-fid often packs name into the request,
but the name may be invalid, eg. NFS export, and even if it's
valid, it may cause inconsistency because this operation is done
on this fid, which is globally unique, but name not.

Since open-by-fid doesn't pack name, for striped dir we can't know
parent stripe fid on client, so we set parent fid the same as
child fid, and MDT has to find its parent fid from linkea (this is
already supported by MDT).

M_CHECK_STALE becomes obsolete.

Unset MDS_OPEN_FL_INTERNAL from open syscall flags, because these
flags are internally used, and should not be set from user space.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Id8d32a26497b2df944c7f9ef51a418ec136a6a9b
Reviewed-on: http://review.whamcloud.com/7476
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-2827 mdt: Also handle resend for layout-lock 78/10378/2
Bruno Faccini [Tue, 20 May 2014 15:50:23 +0000 (17:50 +0200)]
LU-2827 mdt: Also handle resend for layout-lock

This patch is an add-on to the generic fix for resends
handling known as :
Lustre-commit: I9a23c13313d5168d0c232fdc0acbdab330089356
Lustre-change: http://review.whamcloud.com/5978/
This patch adds handling of resend during layout-lock

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Ideb551779a884d77ded8036c513f62557b2dfcde
Reviewed-on: http://review.whamcloud.com/10378
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-2827 ldlm: wrong handling of ldlm resends 78/5978/10
Vitaly Fertman [Tue, 6 May 2014 19:18:03 +0000 (23:18 +0400)]
LU-2827 ldlm: wrong handling of ldlm resends

If a successful reply is lost for an intent
lock request, MDS will not correctly recover
from this situation on resend. Instead, a new
lock is created.

Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Reviewed-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Tested-by: Elena Gryaznova <elena_gryaznova@xyratex.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Change-Id: I9a23c13313d5168d0c232fdc0acbdab330089356
Tested-by: Andrey Bondarenko <andrey_bondarenko@xyratex.com>
Xyratex-bug-id: MRP-975 MRP-1798
Reviewed-on: http://review.whamcloud.com/5978
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-3188 osc: shorten IO calling path 92/10292/5
Bobi Jam [Mon, 12 May 2014 14:04:23 +0000 (22:04 +0800)]
LU-3188 osc: shorten IO calling path

By using osc_io_unplug_aync() for osc_queue_sync_pages() to shorten
the IO calling path, to reduce the chance of stack overflow.

This is revive of git commit 83ae17df2bdce837e62473aec27c03d67312c8ea.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I2ac32866f7adbc4701370704612c849a18a5d1ac
Reviewed-on: http://review.whamcloud.com/10292
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5221 vvp: release mmap_sem in error case 41/10741/2
Patrick Farrell [Wed, 18 Jun 2014 02:48:44 +0000 (21:48 -0500)]
LU-5221 vvp: release mmap_sem in error case

The mmap_sem is downed in vvp_mmap_locks, but in case of
error from cl_io_lock_alloc_add, it is not upped.

Credit to Paul Casella at Cray for finding this.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: If172e7cb4c4e6400143d76eee8aa595ae767a20c
Reviewed-on: http://review.whamcloud.com/10741
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5150 mdc: Handle empty but non-zero acl xattr 20/10620/2
Christopher J. Morrone [Fri, 6 Jun 2014 01:02:43 +0000 (18:02 -0700)]
LU-5150 mdc: Handle empty but non-zero acl xattr

We have found that posix_acl_access can have a value
of \002\000\000\000.  In that case body->aclsize is
non-zero, but the there are no actuall acls stored
in the xattr.

In mdc_unpack_acl(), it only checks IS_ERR() on the
pointer returned by posix_acl_from_xattr(), it does not
check for NULL.  Because of the above situation, the
xattr aclsize can be non-zero, but posic_acl_from_xattr()
still returns NULL.  Passing NULL to posix_acl_valid()
crashes the kernel.

We add a check to properly handle the NULL return value.

Change-Id: Ib4f623642b86db2c88923a27cd9e77c870f301af
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/10620
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4929 lfs: Add lfs dirstripe to the manpage 43/10643/10
Wang Di [Sat, 7 Jun 2014 20:18:11 +0000 (13:18 -0700)]
LU-4929 lfs: Add lfs dirstripe to the manpage

Add lfs-setdirstripe.1, lfs-getdirstripe and
lfs-mkdir.1 to the manpage.

Add -m mode to "lfs mkdir" help, and also mention lfs
setdirstripe can only be done on MDT0 with adminitrator
right.

lfs getdirstripe should honor -O MDTUUID to list the
stripe information of subdirectories located on that MDT.

Remove unnecessary error message in cb_getstripe.

Fix the default getdirstripe behavior, which should list all of
its subdirectories stripe pattern, like lfs getstripe.

Fix format of LMV stripe information.

Change-Id: I7ee6ef3d6df5df50e35f5874a27e176884d79704
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10643
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5164 osd: Limit lu_object cache 37/10237/3
Brian Behlendorf [Thu, 24 Apr 2014 20:47:47 +0000 (13:47 -0700)]
LU-5164 osd: Limit lu_object cache

For OSDs like ZFS to perform optimally it's import that they be allowed
to manage their own cache.  This maximizes the likelyhood that the ARC
will prefetch and cache the right buffers.  In the existing ZFS OSD
code a cached LU object pins buffers in the ARC preventing them from
being dropped.  As the LU cache grows it can consume the entire ARC
preventing buffers for other objects, such as the OIs, from being
cached and severely impacting the performance for FID lookups.

By default this patch will only limit the LU cache for ZFS OSDs.

NOTES:

* Setting LU_CACHE_NR_ZFS_LIMIT to 0 results in an LBUG on the MDS.
  This may be because an object is being used without a reference.
  Setting a minimum value of 256 was arbitrary, ideally we would
  set this value to 0.

* In order to be able to quickly determine the number of objects in
  the hash table the CFS_HASH_COUNTER flag is added.  This adds an
  atomic_inc/dec to the hash insert/remove paths but is not expected
  to have any measurable impact of performance.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Change-Id: Ia64838d50395f1d22e558631adbfa39d44e91606
Patch-Set: 3
Reviewed-on: http://review.whamcloud.com/10237
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4906 llite: read page from LMV/MDC for readdir 22/10622/8
Wang Di [Fri, 6 Jun 2014 08:49:37 +0000 (01:49 -0700)]
LU-4906 llite: read page from LMV/MDC for readdir

This patch will change readdir iteration from read entry to read
page. During rm -rf, the statahead thread will enqueue the lock for
each entry in mdc_read_entry, in the mean time, MDT will revoke
the lock when it deletes each entry, which causes ldlm lock ping
pong behavior, and a lot extra RPC.

With this patch, llite will read one page from MDC each time, then
stat ahead thread will hold the page and do getattr for all entries
in this page. Note: it does not need to hold ldlm lock after it
get the page, because stat ahead thread can live with -ENOENT
or ESTALE, if the entry becomes stale during this time.

With this patch, the performance of "rm -rf" will be improved
for current master.

1. without this patch on master

[root@mds tests]# ./createmany -o /mnt/lustre/test1/f- 10000
total: 10000 creates in 11.98 seconds: 835.02 creates/second
[root@mds tests]# ls /mnt/lustre/test1/ | wc
  10000   10000   68890
[root@mds tests]# time rm -rf /mnt/lustre/test1

real 1m4.496s
user 0m0.014s
sys 0m3.832s

2. with this patch on master

[root@mds tests]# ./createmany -o /mnt/lustre/test1/f- 10000
total: 10000 creates in 11.72 seconds: 853.50 creates/second
[root@mds tests]# time rm -rf /mnt/lustre/test1

real 0m8.151s
user 0m0.012s
sys 0m2.242s

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I189f0039b5624455b70d1381b43d452cc8766543
Reviewed-on: http://review.whamcloud.com/10622
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-1538 tests: keep /sbin/mount.lustre until cleanup 81/10481/2
Andreas Dilger [Wed, 28 May 2014 23:15:25 +0000 (17:15 -0600)]
LU-1538 tests: keep /sbin/mount.lustre until cleanup

Don't unmount /sbin/mount.lustre in the middle of running tests
on a local test system if it is not doing final cleanup.  Otherwise,
later mounts may fail.

The current /sbin/mount.lustre mountpoint is an empty stub that
returns success (0) if executed, but doesn't mount the filesystem.
Instead, create a mountpoint that prints an message if executed and
returns an error to the caller, so it is easier to debug problems.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ied7b69f536bad87333cf5c543384723415500c1e
Reviewed-on: http://review.whamcloud.com/10481
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4416 osd-ldiskfs: set I_LINKABLE for inc_nlink 75/10375/8
Jeff Mahoney [Fri, 6 Jun 2014 19:52:58 +0000 (15:52 -0400)]
LU-4416 osd-ldiskfs: set I_LINKABLE for inc_nlink

Linux commit f4e0c30c191 (allow the temp files created by open() to be
linked to) put in a check to warn if I_LINKABLE was not set
when trying to pull nlinks up from 0. This patch sets the flag
to properly allow it.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I2a71d0515f3735f8eb3e94d190e68e41a36e0f68
Reviewed-on: http://review.whamcloud.com/10375
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4629 obdclass: compile issues with variable not being initialized 05/10705/2
James Simmons [Thu, 12 Jun 2014 21:27:26 +0000 (17:27 -0400)]
LU-4629 obdclass: compile issues with variable not being initialized

One of the versions of gcc I have refuses to build obd_mount.c due to
index not be initialized in function lmd_make_exclusion before it is
used.

Change-Id: Ib549e595492e647364441c5d96df86ec513ad6c6
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/10705
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4629 ldlm: fix uninitialized variable 64/10664/3
Dmitry Eremin [Tue, 10 Jun 2014 17:46:55 +0000 (21:46 +0400)]
LU-4629 ldlm: fix uninitialized variable

'rc' might be used uninitialized in this function.

ldlm_lib.c:1902: 'rc' is declared.
ldlm_lib.c:1910: req->rq_export->exp_disconnected is false
ldlm_lib.c:1951: 'rc' is used, but is uninitialized.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I0cea444d564f3806ed1d926ca5f4909f7005e240
Reviewed-on: http://review.whamcloud.com/10664
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4579 ldlm: Properly display time on lock timeout 01/10601/3
Oleg Drokin [Thu, 5 Jun 2014 02:14:11 +0000 (22:14 -0400)]
LU-4579 ldlm: Properly display time on lock timeout

When lock timeout happens, need to printtime since last lock
activity, not since when the lock was granted.
As such set l_last_activity to current time when sending callbacks too.

Change-Id: I31d0de6d9add67865cfcb16af903647f7cb932a0
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/10601
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
6 years agoLU-5162 mdc: Add exception entry check for radix_tree 09/10709/2
Yang Sheng [Fri, 13 Jun 2014 03:33:02 +0000 (11:33 +0800)]
LU-5162 mdc: Add exception entry check for radix_tree

We need check exception entry after radix_tree lookup.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I9a8493e0de7d44f04b00d43919ed5f3ab6c1d0d9
Reviewed-on: http://review.whamcloud.com/10709
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5140 build: Make ko2iblnd buildable with Mellanox and newer kernels 71/10571/3
James Simmons [Thu, 5 Jun 2014 00:36:17 +0000 (20:36 -0400)]
LU-5140 build: Make ko2iblnd buildable with Mellanox and newer kernels

The libcfs layer handles different level of support for process
namespace support. It does this by examining the uidgid.h
header. Both newer kernels and the Mellanox external OFED stack
supply their own uidgid.h header. In the current ko2iblnd
driver it completely ignores the Mellanox uidgid.h version
which ends up conflicting with the native uidgid.h header. The
reason libcfs misses the Mellanox handling is because the libcfs
and lnet headers are placed before the Mellanox headers so the
definations are missed. This patch moves all the libcfs and lnet
headers after the Mellanox header section.

Change-Id: Ibd7425b4a4d5b0f59c48e2007d870c67adb5f63d
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/10571
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Shuichi Ihara <sihara@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4587 utils: remove llapi_ping_target() 66/10566/3
John L. Hammond [Tue, 3 Jun 2014 07:04:51 +0000 (02:04 -0500)]
LU-4587 utils: remove llapi_ping_target()

Remove llapi_ping_target(). This ioctl() based function was prototyped
in lustreapi.h but not provided by lustreapi. Replace its only use (in
lfs) with the equivalent /proc based function llapi_target_check(),
which is provided by lustreapi. Remove the now unnecessary calls to
ptl_initialize(), obd_initialize() and obd_finalize() in lfs main().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I02d7ea706f02827bbd4bd1b9bf0d216b0e2c131b
Reviewed-on: http://review.whamcloud.com/10566
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
6 years agoLU-3319 procfs: move mdd/ofd proc handling to seq_files 49/8049/18
James Simmons [Thu, 22 May 2014 18:25:52 +0000 (14:25 -0400)]
LU-3319 procfs: move mdd/ofd proc handling to seq_files

With 3.10 linux kernel and above proc handling now only
uses struct seq_files. This patch migrates the mdd/ofd
layer proc entries over to using seq_files.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I61b7df6bfd5efd0f12e3ca1a1813b7b62d493168
Reviewed-on: http://review.whamcloud.com/8049
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-3319 procfs: update zfs proc handling to seq_files 34/7934/7
James Simmons [Wed, 23 Apr 2014 13:31:09 +0000 (09:31 -0400)]
LU-3319 procfs: update zfs proc handling to seq_files

Migrate all zfs proc handling to using strictly seq_files.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I6dc7e65c3e74e7934a17939815ec3c334fac58c7
Reviewed-on: http://review.whamcloud.com/7934
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
6 years agoLU-5142 test: add version check to conf-sainty.sh test_43 75/10675/2
Emoly Liu [Fri, 6 Jun 2014 14:45:00 +0000 (22:45 +0800)]
LU-5142 test: add version check to conf-sainty.sh test_43

This patch adds Lustre version check codes into conf-sanity
test_43 to make the test work with servers which do not have
the following patches:

Lustre-commit: 3064318df90ce7019b83c8a7f80e93a9634c00fa
Lustre-change: http://review.whamcloud.com/5700
Lustre-commit: 917b0a17d4e50db9f966763925a2dc860d932a3b
Lustre-change: http://review.whamcloud.com/9221

Test-Parameters: envdefinitions=SLOW=yes,ONLY=43 testlist=conf-sanity ossjob=lustre-b2_5 mdsjob=lustre-b2_5 ossbuildno=61 mdsbuildno=61
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I9fad8d7002eb29b3da89213a6afc3c88de611b8f
Reviewed-on: http://review.whamcloud.com/10675
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
6 years agoLU-3496 ost: lprocfs batch update 56/6756/9
Andrew Perepechko [Tue, 3 Jun 2014 16:47:00 +0000 (20:47 +0400)]
LU-3496 ost: lprocfs batch update

Update lprocfs stats such as cache hits, misses, totals
using single calls to lprocfs_counter_add so as to
improve performance.

Non-zero number of hits/misses/totals for a single RPC
counts as a single sample for the respective hits/misses/totals
counter.

Signed-off-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Reviewed-by: Alexander Boyko <Alexander_Boyko@xyratex.com>
Reviewed-by: alexander_zarochentsev@xyratex.com
Reviewed-by: Vitaly Fertman <Vitaly_Fertman@xyratex.com>
Xyratex-bug-id: MRP-999
Change-Id: I8771e81b7e0356436a1ffcb7e9c51b87580750a9
Reviewed-on: http://review.whamcloud.com/6756
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
6 years agoLU-5160 build: warn on deprecated CFS_LIST_HEAD uses 49/10649/2
Emoly Liu [Thu, 5 Jun 2014 22:35:54 +0000 (06:35 +0800)]
LU-5160 build: warn on deprecated CFS_LIST_HEAD uses

Change checkpatch.pl script to report:
CFS_LIST_HEAD() is deprecated, use "struct list_head foo =
LIST_HEAD_INIT(foo);" instead.

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Ic667d4598c0740e2802416dd3800dacdaf378e73
Reviewed-on: http://review.whamcloud.com/10649
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
6 years agoLU-4975 ofd: add comments for ofd_capa.c functions 06/10606/4
Lai Siyao [Thu, 5 Jun 2014 15:18:20 +0000 (23:18 +0800)]
LU-4975 ofd: add comments for ofd_capa.c functions

- Fix up GPL header block to reference proper GPLv2 license URL.
- Remove mention of contacting Sun.
- Add introductionary comment block for the ofd_capa.c file.
- Add function comments blocks to all functions in the ofd_capa.c
  file
- Remove cfs_ prefix from list_head functions

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: I7210ed43c7d089e86d978e6033c3f4659bc41f52
Reviewed-on: http://review.whamcloud.com/10606
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Richard Henwood <richard.henwood@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-5126 libcfs: fix illegal page access of tracefiled() 24/10524/2
Li Xi [Fri, 30 May 2014 15:13:18 +0000 (23:13 +0800)]
LU-5126 libcfs: fix illegal page access of tracefiled()

After failure happens and put_pages_back() returns the pages,
tracefiled() should not go on itering on the page list.
Otherwise, some pages might be accessed illegally.

Signed-off-by: Li Xi <lixi@ddn.com>
Change-Id: I34dd91e3a7ca7d49e67c2bd587c481eb71873003
Reviewed-on: http://review.whamcloud.com/10524
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4976 osp: add comments for osp_trans.c functions 61/10361/6
Fan Yong [Sat, 3 May 2014 02:39:24 +0000 (10:39 +0800)]
LU-4976 osp: add comments for osp_trans.c functions

1) Rename some structures and functions names to avoid misguiding
   others, including:
1.1) osp_async_update_interpterer_t => osp_async_request_interpreter_t
1.2) osp_insert_async_update => osp_insert_async_request
1.3) osp_unplug_async_update => osp_unplug_async_request
1.4) osp_async_update_item => osp_async_request
1.5) osp_async_update_item_init => osp_async_request_init
1.6) osp_async_update_item_fini => osp_async_request_fini
1.7) osp_insert_async_update => osp_insert_async_request

2) Add introductory comment block for the osp_trans.c file. And add
   function comment blocks to all functions in the osp_trans.c file.

3) Typo fixing and code cleanup.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I3dc4e9d6613baa035d228f2254306b6a71b2d48c
Reviewed-on: http://review.whamcloud.com/10361
Reviewed-by: Richard Henwood <richard.henwood@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
6 years agoLU-4975 ofd: documenting the ofd_fs.c 17/10417/5
Mikhail Pershin [Wed, 21 May 2014 20:02:29 +0000 (00:02 +0400)]
LU-4975 ofd: documenting the ofd_fs.c

Fix up GPL header block to reference proper GPLv2 license URL.
Remove mention of contacting Sun, since they don't exist anymore.
Add introductory comment block for the ofd_fs.c file and add
function comment blocks to all functions in it.
Couple tiny helpers are moved inside caller code because they are
called just from one place and contains few lines of code.
This patch changes cfs_list_t->struct list_head also in several
places.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Ifa231f941c80c98bb9a0c20bdd56e1ae379f496a
Reviewed-on: http://review.whamcloud.com/10417
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
6 years agoLU-4423 llite: remove ll_readlink as unnecessary 23/10123/8
Oleg Drokin [Sun, 1 Jun 2014 14:27:20 +0000 (10:27 -0400)]
LU-4423 llite: remove ll_readlink as unnecessary

Taken from kernel upstrem by Al Viro:
lustre: generic_readlink() is just fine there, TYVM...

Linux-commit: 4efcc9ffcd4fc53f1f7de539842cdffa1f8e5ecc

Change-Id: I3693f5b7871f9a013c98f962eb9299ffbe32b705
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/10123
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
6 years agoLU-1199 build: Remove unused lbuild options. 22/9922/3
Ashley Pittman [Wed, 9 Apr 2014 14:05:00 +0000 (10:05 -0400)]
LU-1199 build: Remove unused lbuild options.

Remove the --tag, --ldiskfs, --reuserpm and --disable-timestamps
options entirely.

Signed-off-by: Ashley Pittman <apittman@ddn.com>
Change-Id: Icea6886730cc58f9703fb441a9cb94b58a790314
Reviewed-on: http://review.whamcloud.com/9922
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
6 years agoLU-4227 utils: mgs option is required to format a MDT/OST 87/8387/5
Emoly Liu [Mon, 9 Jun 2014 11:20:37 +0000 (19:20 +0800)]
LU-4227 utils: mgs option is required to format a MDT/OST

When running mkfs.lustre to format a MDT, --mgs or --mgsnode option
is required; while to format a OST, --mgsnode option is required.
conf-sanity.sh test_79 is added to verify the fix.

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Ia65b382aad2bc31496f3da62f15349ec38990ea0
Reviewed-on: http://review.whamcloud.com/8387
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>