git://git.whamcloud.com - fs/lustre-release.git/log

LU-7734 lnet: fix routing selection

Always prefer locally connected networks over routed networks.
If there are multiple routed networks and no connected networks
pick the best gateway to use. If all gateways are equal then
round robin through them.

Renamed dev_cpt to ni_dev_cpt to maintain naming convention.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Ie6a3aaa7a9ec4f5474baf5e1ec0258d481418cb1
Reviewed-on: http://review.whamcloud.com/21326

LU-7734 lnet: power8 compile fix

On Power8 the following error occured:

error: inlining failed in call to always_inline ‘lnet_get_numa_range’:
function body not available inline __u32 lnet_get_numa_range(void);

The reason for this was for the linux kernel you
must fill in the body of a inline function. Replace
this inline function with exposing the lnet_numa_range
module parameter like we do for portal_rotor. Also
treat all the lnet_numa_range handling as unsigned int.

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Signed-off-by: Olaf Weber <olaf@sgi.com>
Change-Id: Ic566e56eb1e333d145de21d1c197218c6425dc5b
Reviewed-on: http://review.whamcloud.com/21078
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Tested-by: Amir Shehata <amir.shehata@intel.com>

LU-7734 lnet: Routing fixes part 2

Fix lnet_select_pathway() to handle the routing cases correctly.
The following general cases are handled:
. Non-MR directly connected
. Non-MR not directly connected
. MR Directly connected
. MR Not directly connected
  . No gateway
  . Gateway is non-mr
  . Gateway is mr

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: If2d16b797b94421e78a9f2a254a250a440f8b244
Reviewed-on: http://review.whamcloud.com/21167

LU-7734 lnet: Routing fixes part 1

This is the first part of a routing fix.
- Fix crash in lnet_parse_get()
- Resolve deadlock when adding a route.
- Fix an issue with dynamically turning on routing
- Set the final destination NID properly when routing a msg

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I68d0e4d52192aa96e37c77952a1ebe75c1b770c5
Reviewed-on: http://review.whamcloud.com/21166

LU-7734 lnet: fix lnet_select_pathway()

Fixed the selection algorithm to work properly with > 1 local
networks. The behavior now is to iterate through all interfaces
on all networks

Also removed the health variable from struct lnet_peer_net since
it's never used.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Ib91748e80446585b6a9e1bc0f3af6894599d8aaa
Reviewed-on: http://review.whamcloud.com/20890
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Jenkins

LU-7734 lnet: configuration fixes

Fix cpt configuration from DLC to configure the proper list
of cpts in LNet. Check in LNet that no CPTs are outside the
available CPTs in the system.

Fix peer_rtr_credits name to peer_tx_credits to reflect the
actual value.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Ic4a3985a470ed901be6166df4079205677921817
Reviewed-on: http://review.whamcloud.com/20862
Tested-by: Jenkins

LU-7734 lnet: fix lnet_peer_table_cleanup_locked()

In lnet_peer_table_cleanup_locked() we delete the entire peer if the
lnet_peer_ni for the primary NID of the peer is deleted. If the next
lnet_peer_ni in the list belongs to the peer being deleted, then the
next pointer kept by list_for_each_entry_safe() ends up pointing at
freed memory.

Add a list_for_each_entry_from() loop to advance next to a peer_ni
that does not belong to the peer being deleted and will therefore
remain present in the list.

Signed-off-by: Olaf Weber <olaf@sgi.com>
Change-Id: I92bf219dc93a79f7d90035ccfbb38cd251138c04
Reviewed-on: http://review.whamcloud.com/20824
Tested-by: Jenkins
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Tested-by: Amir Shehata <amir.shehata@intel.com>

LU-7734 lnet: Fix lnet_msg_free()

Remove the ni_decref in lnet_msg_free(), since this function
gets called with no lnet_net_lock() held

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Ibfcbcea25287f4d22ae6146d7aa01f4279ffe969
Reviewed-on: http://review.whamcloud.com/20729

LU-7734 lnet: simplify and fix lnet_select_pathway()

In lnet_select_pathway() we restart selection if the DLC seq
counter changes. Provided we take a hold on the preferred
lnet_peer_ni, we only need to restart if an lnet_ni was added
or removed. Update the locations where lnet_incr_dlc_seq() is
called to take this into account.

A number of local variables must be reset whenever we goto
again. Do this immediately after the label for the global
variables, and immediately before the block that uses them
for the helper variables.

In the loop where NUMA distances are compared, use the NUMA
range for distances smaller than the NUMA range, simplifying
the subsequent comparisons between distances.

Remote the lo_sent output parameter. Instead do an early
return with LNET_CREDIT_OK.

Move the increment of the best_lpni->lpni_seq number after
the check that best_lpni isn't NULL.

When routing, the best_gw should be treated as the best_lpni
for the purpose of determining the CPT to lock.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Signed-off-by: Olaf Weber <olaf@sgi.com>
Change-Id: Ie71eebc2301601cf1c85c6248dbed06951b89274
Reviewed-on: http://review.whamcloud.com/20720

LU-7734 lnet: protect peer_ni credits

Currently multiple NIs can talk to the same peer_ni. The per-CPT
lnet_net_lock therefore no longer protects the lpni against
concurrent updates. To resolve this issue a spinlock is added
to the lnet_peer_ni, which must be locked when the peer NI
credits, delayed message queue, and delayed routed message queue
are modified. The lock is not taken when reporting credits.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Signed-off-by: Olaf Weber <olaf@sgi.com>
Change-Id: I52153680a74d43e595314b63487026cc3f6a5a8f
Reviewed-on: http://review.whamcloud.com/20702

LU-7734 lnet: proper cpt locking

1. add a per NI credits, which is just the total credits
   assigned on NI creation
2. Whenever percpt credits are added or decremented, we
   mirror that in the NI credits
3. We use the NI credits to determine best NI
4. After we have completed the peer_ni/ni selection we
   determine the cpt to use for locking:
cpt_of_nid(lpni->nid, ni)

The lpni_cpt is not enough to protect all the fields in the
lnet_peer_ni structure. This is due to the fact that multiple
NIs can talk to the same peer, and functions can be called with
different cpts locked. To properly protect the fields in the
lnet_peer_ni structure, a spin lock is introduced for the
purpose.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Ief7868c3c8ff7e00ea9e908dd50d8cef77d9f9a4
Reviewed-on: http://review.whamcloud.com/20701

LU-7734 lnet: peer/peer_ni handling adjustments

A peer can be added by specifying a list of NIDs
The first NID shall be used as the primary NID. The rest of
the NIDs will be added under the primary NID

A peer can be added by explicitly specifying the key NID, and then
by adding a set of other NIDs, all done through one API call

If a key NID already exists, but it's not an MR NI, then adding that
Key NID from DLC shall convert that NI to an MR NI

If a key NID already exists, and it is an MR NI, then re-adding the
Key NID shall have no effect

if a Key NID already exists as part of another peer, then adding that
NID as part of another peer or as primary shall fail

if a NID is being added to a peer NI and that NID is a non-MR, then
that NID is moved under the peer and is made to be MR capable

if a NID is being added to a peer and that NID is an MR NID and part
of another peer, then the operation shall fail

if a NID is being added to a peer and it is already part of that Peer
then the operation is a no-op.

Moreover, the code is structured to consider the addition of Dynamic
Discovery in later patches.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I71f740192a31ae00f83014ca3e9e06b61ae4ecd5
Reviewed-on: http://review.whamcloud.com/20531

LU-7734 lnet: Add peer_ni and NI stats for DLC

This patch adds three stats to the peer_ni and NI structures:
send_count, recv_count, and drop_count. These stats get printed
when you do an "lnetctl net show -v" (for NI) and
"lnetctl peer show" (for peer_ni).

Signed-off-by: Doug Oucharek <doug.s.oucharek@intel.com>
Change-Id: Ic41c88cbc68dba677151d87a1fab53a48d36ea29
Reviewed-on: http://review.whamcloud.com/20170
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Tested-by: Amir Shehata <amir.shehata@intel.com>

LU-7734 lnet: rename LND peer to peer_ni

Patch to rename LND peers to peer_ni to reflect the fact that these
constructs reflect an actual connection between a local NI and remote
peer NI.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I1c25a12eae61d8822a8c4ada2e077a5b2011ba22
Reviewed-on: http://review.whamcloud.com/19307
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Doug Oucharek <doug.s.oucharek@intel.com>

LU-7734 lnet: handle N NIs to 1 LND peer

This patch changes o2iblnd only, as socklnd already handles this
case. In the new design you can have multiple NIs communicating
to one peer. In the o2ilbnd the kib_peer has a pointer to the NI
which implies a 1:1 relationship.

This patch changes kiblnd_find_peer_locked() to use the peer NID
and the NI NID as the key. This way a new peer will be created for
each unique NI/peer_NI pair.

This is similar to how socklnd handles this case.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Ifab7764489757ea473b15c46c1a22ef9ceeeceea
Reviewed-on: http://review.whamcloud.com/19306
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Doug Oucharek <doug.s.oucharek@intel.com>

LU-7734 lnet: handle non-MR peers

Add the ability to declare a peer to be non-MR from the DLC
interface. By default if a peer is configured from DLC it is
assumed to be MR capable, except when the non-mr flag is set.

For non-MR peers always use the same NI to communicate with it.
If multiple NIs are used to communicate with a non-MR peer the
peer will consider that it's talking to different peers which could
cause upper layers to be confused.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Ie3ec45f5f44fa7d72e3e0335b1383f9c3cc92627
Reviewed-on: http://review.whamcloud.com/19305
Tested-by: Jenkins
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Olaf Weber <olaf@sgi.com>

LU-7734 lnet: Primary NID and traffic distribution

When receiving messages from a multi-rail peer we must keep track of
both the source NID and the primary NID of the peer. When sending a
reply message or RPC respone, the source NID is preferred. But most
other uses require identifcation of the peer regardless of which
source NID the message came from, and so the primary NID of the peer
must then be used.

An example for this is the creation of match entries. Another occurs
when an event is created: the initiator should be the primary NID, to
ensure upper layers (PtlRPC and Lustre) always see the same NID for
that peer.

This change also contains code to have PtlRPC use LNET_NID_ANY for
the 'self' parameter of LNetPut() and LNetGet() when it doesn't care
which NI it sends from, and to provide a local/peer NID pair when it
does. This can be broken out into a separate change.

Signed-off-by: Olaf Weber <olaf@sgi.com>
Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: If4391f2537a94f5784e8c61ae03aad266b2f8e7d
Reviewed-on: http://review.whamcloud.com/18938
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>

LU-7734 lnet: NUMA support

This patch adds NUMA node support. NUMA node information is stored
in the CPT table. A NUMA node mask is maintained for the entire table
as well as for each CPT to track the NUMA nodes related to each of
the CPTs. Following key APIs added:

cfs_cpt_of_node(): returns the CPT of particular NUMA node
cfs_cpt_distance(): calculates the distance between two CPTs

When the LND device is started it finds the NUMA node of the physical
device and then from there it finds the CPT, which is subsequently
stored in the NI structure.

When selecting the NI, the MD CPT is determined and the distance
between the MD CPT and the device CPT is calculated. The NI
with the shortest distance is preferred.

If the device or system is not NUMA aware then the CPT for the
device will default to CFS_CPT_ANY and the distance calculated
when CFS_CPT_ANY is used is largest in the system. IE, none
NUMA aware devices are least preferred.

A NUMA range value can be set. If the value is large enough
it amounts to basically turning off NUMA criterion completely.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I2d7c63f8e8fc8e8a6a249b0d6bfdd08fd090a837
Reviewed-on: http://review.whamcloud.com/18916
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Olaf Weber <olaf@sgi.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>

LU-7734 lnet: configure local NI from DLC

This patch adds the ability to configure multiple network interfaces
on the same network. This can be done via the lnetctl CLI interface
or through a YAML configuration. Refer to the multi-rail HLD for
more details on the syntax.

It also deprecates ip2nets kernel parsing. All string parsing and
network maching now happens in the DLC userspace library.

New IOCTLs are added for adding/deleting local NIs, to keep backwards
compatibility with older version of the DLC and lnetctl.

The changes also include parsing and matching ip2nets syntax at the
user level and then passing down the network interfaces down to the
kernel to be configured.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: I19ee7dc76514beb6f34de6517d19654d6468bcec
Reviewed-on: http://review.whamcloud.com/18886
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-7734 lnet: configure peers from DLC

This patch adds the ability to configure peers from the DLC
interface.

When a peer is added a primary NID should be provided. If none is
provided then the first NID in the list of NIDs will be used
as the primary NID.

Basic error checking is done at the DLC level to ensure properly
formatted NIDs. However, if a NID is a duplicate, this will be
detected when adding it in the kernel. Operation is halted, which
means some peer NIDs might have already been added, but not the
entire set. It's the role of the caller to backtrack and remove that
peer that failed to add.

When deleting a peer a primary NID or a normal NID can be provided.
If a standard NID is provided, then the peer is found, and the
primary NID is compared to the peer ni. If they are the same the
entire peer is deleted. Otherwise, only the identified peer ni is
deleted. If a set of NIDs are provided each one will be removed
from the peer identified by the peer NID in turn.

The existing show peer credits API can be used to show peer
information.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Iaf588a062b44d74305aa9aa7d31c7341c6c384b9
Reviewed-on: http://review.whamcloud.com/18476
Tested-by: Jenkins
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Olaf Weber <olaf@sgi.com>

LU-7734 lnet: Multi-Rail local_ni/peer_ni selection

This patch implements the local_ni/peer_ni selection algorithm.
It adds APIs to the peer module to encapsulate
iterating through the peer_nis in a peer and creating a peer.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Ifc0e5ebf84ab25753adfcfcb433b024100f35ace
Reviewed-on: http://review.whamcloud.com/18383
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Olaf Weber <olaf@sgi.com>
Tested-by: Jenkins
Tested-by: Doug Oucharek <doug.s.oucharek@intel.com>

LU-7734 lnet: Multi-Rail peer split

Split the peer structure into peer/peer_net/peer_ni, as
described in the Multi-Rail HLD.

Removed deathrow list in peers, instead peers are immediately
deleted. deathrow complicates memory management for peers to
little gain.

Moved to LNET_LOCK_EX for any operations which will modify the
peer tables. And CPT locks for any operatios which read the peer
tables. Therefore there is no need to use lnet_cpt_of_nid() to
calculate the CPT of the peer NID, instead we use lnet_nid_cpt_hash()
to distribute peers across multiple CPTs.

It is no longe true that peers and NIs would exist on
the same CPT. In the new design peers and NIs don't have a 1-1
relationship. You can send to the same peer from several NIs, which
can exist on separate CPTs

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Ida41d830d38d0ab2bb551476e4a8866d52a25fe2
Reviewed-on: http://review.whamcloud.com/18293
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Olaf Weber <olaf@sgi.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>

LU-7734 lnet: Multi-Rail local NI split

This patch allows the configuration of multiple NIs under one Net.
It is now possible to have multiple NIDs on the same network:
   Ex: <ip1>@tcp, <ip2>@tcp.
This can be configured using the following syntax:
   Ex: tcp(eth0, eth1)

The data structures for the example above can be visualized
as follows

               NET(tcp)
                |
        -----------------
        |               |
      NI(eth0)        NI(eth1)

For more details refer to the Mult-Rail Requirements and HLD
documents

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Id7c73b9b811a3082b61e53b9e9f95743188cbd51
Reviewed-on: http://review.whamcloud.com/18274
Tested-by: Jenkins
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Olaf Weber <olaf@sgi.com>

New tag 2.9.52

Change-Id: I7fd2714e14825f5966751a6d0a66313c1e3088b5
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8587 utils: Fix incorrect indenting in llapi_hsm_log_ct_progress

gcc6 highlights this case of incorrect indenting:
        if (progress_type == CT_RUNNING)
                rc = llapi_json_add_item(&json_items, "current_bytes",
                                         LLAPI_JSON_BIGNUM, &current);
                if (rc < 0)
                        goto err;

Just add the braces around, though logic-wise it's all fine.

Change-Id: I770857fe2f9ce29817558247ce0987842b8d06b4
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/24570
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

LU-8996 kernel: kernel update RHEL6.8 [2.6.32-642.13.1.el6]

Update RHEL6.8 kernel to 2.6.32-642.13.1.el6

Test-Parameters: trivial clientdistro=el6.8 mdsdistro=el6.8 ossdistro=el6.8 \
mdsfilesystemtype=ldiskfs mdtfilesystemtype=ldiskfs \
ostfilesystemtype=ldiskfs testgroup=review-ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I3d8ffa55ff050503a69c4db260ac6b915564349a
Reviewed-on: https://review.whamcloud.com/24878
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8929 lfsck: dumper gets current position properly

It is normal that the LFSCK iteration has been done when the
dump hanlder found the status as LS_SCANNING_PHASE1, it may
because of race, or the LFSCK failed to update the status.
Under such cases, the dump handler will use the position in
the last checkpoint as the current position. It may be not
100% accurate, but not serious issue.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I672258baa9d0b0aa8ec12249c13b2b147a274ab4
Reviewed-on: https://review.whamcloud.com/24869
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7481 utils: label lustre device correctly

Currently, the device label will be read before mounting Lustre,
the flags LDD_F_VIRGIN and LDD_F_WRITECONF will be set according
to the label (the corresponding original flags containing in the
lustre_disk_data will be ignored). But the device label could be
changed during mount by the journal recovery, and the device should
be also labeled to indicate the target device is supposed to start.

Test-Parameters: testlist=conf-sanity,conf-sanity,conf-sanity

Change-Id: I2df1d81f764a7d1ffa26afb0197d137c057a25e9
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: https://review.whamcloud.com/24845
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8932 lnet: define new network driver ptl4lnd

Assign an ID to the new network driver ptl4lnd developped by Bull
that implements a LND based on Portals 4 API. It is intended to be
used with BXI, the Bull interconnect hardware.

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: I38e505916899dc7f01b3ad0372c9f068fa06f308
Reviewed-on: https://review.whamcloud.com/24768
Tested-by: Jenkins
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-7429 tests: generate dangling name entry properly

There may be some creation after the dangling injection, such as
create ".lustre/lost+found/MDTxxxx" for LFSCK, create some quota
related local files. These creations may reuse the just released
local object/inode that is referenced by the dangling name entry.
That will fail the dangling injection as to the subsequent LFSCK
will not find dangling name entry. So before deleting the target
object for the dangling name entry, remove some other objects to
avoid the target object being reused by some potential creations.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I0e1bb60c1095119e10009fe2a8ce38687e3e7692
Reviewed-on: https://review.whamcloud.com/24763
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 ptlrpc: use 64-bit times for ptlrpc_sec

Here we use an unsigned long to store the timeout for gc,
which is probably safe until 2106, but this patch converts it
to use ktime_get_real_seconds() and time64_t for consistency.

Linux-commit: 8cc980713ec9e6847896891c54562ad815c33424

Change-Id: I9c66ac818239debe676b78fbee5764cd5b69028c
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/24710
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

LU-4423 ptlrpc: use 64-bit times in ptlrpc_enc_page_pool

ptlrpc_enc_page_pool computes time deltas using 'long' values from
get_seconds(). This is probably safe beyond y2038, but it's better
to go use monotonic times and 64-bit here for consistency.

Linux-commit: 80018a9edbc3180ae31a7197f9dacab975a7f5e2

Change-Id: I15511b04d8b4f7353ce109d8d9bb7887f551d880
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/24708
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

LU-4423 lnet: Better cookie gen

api-ni.c uses do_gettimeofday to get a 'cookie' or timestamp.
This patch replaces it with ktime_get_ns for the following reasons:

1. ktime_get_ns returns a __u64 which is safer than 'struct timeval'
which will overflow on 32-bit systems in year 2038 and beyond.
2. Improved resolution: nsecs instead of usecs.
3. Reduced compute: ktime_get_ns is faster than the multiply/add
combination used in this function

Linux-commit: 9056be30542bfff51190bdda67088f319cf4c9f5

Drop unneeded wrapper function Remove the function
lnet_create_interface_cookie() and replace its call
with the function ktime_get_ns().

Linux-commit: 7bcd831b8579212303ec7c30e975432b914493dc

The ln_interface_cookie is used to ensure that a node can tell whether
the following sequence of events has happened:

node sends GET or PUT to peer
node is rebooted
peer sends REPLY or ACK to node

The ln_interface_cookie is set once, when LNet starts, and remains
unchanged afterwards. To avoid accidentally obtaining the same cookie
after a reboot, the code generated ths cookie using ktime_get_ns().
Once generated, the value of the cookie is not interpreted, only
compared for equality. Olaf Weber reported that due to the use of
ktime_get_ns() a small chance exist of generating a cookie of identical
value across reboots. Using ktime_get_real_ns() removes any chance of
this from happening.

Change-Id: I159a0ff2573afb87f279a8e8f282b0ac076d9bf3
Signed-off-by: Tina Ruchandani <ruchandani.tina@gmail.com>
Signed-off-by: Shivani Bhardwaj <shivanib134@gmail.com>
Suggested-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/24682
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Olaf Weber <olaf@sgi.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 libcfs: Use swap() in cfs_hash_bd_order()

Use swap() function instead of using a temporary variable for swapping
two variables.

The Coccinelle semantic patch used to make this change is as follows:
//<smpl>
@@
type T;
T a,b,c;
@@
- a = b;
- b = c;
- c = a;
+ swap(b, c);
//<smpl>

Signed-off-by: Amitoj Kaur Chawla <amitoj1606@gmail.com>
Change-Id: I19aa52fe4e05fed2e03c1d8515731a5ce01b3d09
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/24576
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

LU-4315 docs: Fix Makefile.am to have one man page per line

The man pages for lfs(1) and lctl(8) are quite large. Splitting
them into one page per subcommand will allow for more detailed
information for each subcommand.

This patch modifies the Makefile.am so that the source man pages
are listed one per line. This will make it easier for subsequent
patches to add new pages. Existing pages that were missing from
the Makefile.am have been added to it, and an obsolete page has
been removed from the Makefile.am and been deleted.

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: If312fff9bc5e68176caf0a70a51876e69b1614d8
Reviewed-on: https://review.whamcloud.com/24371
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

LU-8926 llite: reduce jobstats race window

In the current code, lli_jobid is set to zero on every call
to lustre_get_jobid. This causes problems, because it's
used asynchronously to set the job id in RPCs, and some
RPCs will falsely get no jobid set. (For small IO sizes,
this can be up to 60% of RPCs.)

It would be very expensive to put hard synchronization
between this and every outbound RPC, and it's OK to very
rarely get an RPC without correct job stats info.

This patch only updates the lli_jobid when the job id has
changed, which leaves only a very small window for reading
an inconsistent job id.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I6c3a7f8683dc5f5d467940920938db18b0c20462
Reviewed-on: https://review.whamcloud.com/24253
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Chris Horn <hornc@cray.com>

LU-8855 llite: return small device numbers for compat stat()

The compat_sys_*stat*() syscalls will fail unless the devices majors
and minors are both less than 256. So in ll_getattr_it(), if we are in
32 bit compat mode then coerce the device numbers in to the expected
format.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I1bf13258902e13c76b9ebf3476fd1767712de0b3
Reviewed-on: https://review.whamcloud.com/23877
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-8821 mdt: avoid double find in mdt_path_current()

In mdt_path_current() avoid finding the object we are already holding
a reference to.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Iae4796047d2c5b02989d29baf2e7620545f7e45c
Reviewed-on: https://review.whamcloud.com/23701
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8066 obdclass: move lustre sysctl to sysfs

Backport from upstream the changes to port lustre
systctl to sysfs. Needed to re-export the function
lprocfs_read_frac_helper for later work. The
following patches were backported:

Fix class_procfs_init error return value. Dan Carpenter noticed
that procfs conversion patches introduced a bug where should
kobject_create_and_add, an error is not returned from
class_procfs_init.

Linux-commit: 3c4872f94359ed38a1392c0a9238c48a9aee6f8f

Move sysctl timeout to sysfs. This is the first step of
moving lustre sysctls from /proc/sys/lustre to /sys/fs/lustre

Linux-commit: e2424a1265f2772b66f068c205256e2aef5f74a0

Move max_dirty_mb from sysctl to sysfs. max_dirty_mb is
now a parameter in /sys/fs/lustre.

Linux-commit: df476a4d5de09d9324b108fc9c5ff2c00a0850d0

Move debug controls to sysfs. debug_peer_on_timeout,
dump_on_timeout and dump_on_eviction controls from
/proc/sys/lustre to /sys/fs/lustre

Linux-commit: 9e7fa14935901bcd09576b2866d5dd15f69caf83

Move AT controls from sysctl to sysfs. Adaptive Timeouts
controls are being moved from /proc/sys/lustre to
/sys/fs/lustre

Linux-commit: bcef118e7ed67e28edcaab9be9ca11412176c540

Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Change-Id: Id1b00bebf9ecca5284e9c71f4c0f91e56cbf391b
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/23428
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

LU-4423 ldlm: use 64-bit time for pl_recalc

The ldlm pool calculates elapsed time by comparing the previous and
current get_seconds() values, which is unsafe on 32-bit machines
after 2038.

This changes the code to use time64_t and ktime_get_real_seconds(),
keeping the 'real' instead of 'monotonic' time because of the
debug prints.

Linux-commit: 8f83409cf2382c968f96877368cd5b542b92af1d
Linux-commit: b8cb86fd95bb461c3496e1f4b4083b198c963a9c

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Change-Id: I81cca5b529dbf5615cf46461ad1c9179fdee7835
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/23350
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

LU-8710 ptlrpc: use current CPU instead of harcoded 0

fix crash if CPU 0 disabled.

Change-Id: I8ac5a10f544a1c8fc454bc64a6bb1d3607240be9
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/23305
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-8703 libcfs: remove usless CPU partition code

* remove scratch buffer and mutex which guard it.
* remove global cpumask and spinlock which guard it.
* remove cpt_version for checking CPUs state change during setup
because of just disable CPUs state change during setup.
* remove whole global struct cfs_cpt_data cpt_data.
* remove few unused APIs.

Change-Id: I0cc853d57952e76cf32801838a19e6872905aaa0
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/23303
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 obd: use ktime_t for calculating elapsed time

process_param2_config() tries to print how much time has passed
across a call_usermodehelper() function, and uses struct timeval
for that.

We want to remove this structure, so this is better expressed
in terms of ktime_t and ktime_us_delta().

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Change-Id: I90c61d9d49ee0d500772f1b370790e37859f18b2
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/23146
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

LU-7471 tests: Modified make_custom_file_for_progress fn

After executing tests like test_200 and if there is not
enough space the function make_custom_file_for_progress was returning
1 on error but this error was not getting caught properly.
Modified the code to catch the error properly.

Test-Parameters: trivial testlist=sanity-hsm

Seagate-bug-id: MRP-3026
Signed-off-by: Mikhail V. Pridushchenko <mikhail.v.pridushchenko@seagate.com>
Signed-off-by: Aditya Pandit <aditya.pandit@seagate.com>
Change-Id: I0db45e327d6f49d066b5c631096a459b8acd2758
Signed-off-by: Ashish Purkar <ashish.purkar@seagate.com>
Reviewed-on: https://review.whamcloud.com/17346
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

LU-3764 tests: clean up sanity test_116a code style

Clean up the code style for sanity.sh test_116a():
- proper indentation
- don't use $ for variables inside $((...))
- one statement per line

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I5fe502b85365e00ce34b90f8fabfb58320a07e51
Reviewed-on: https://review.whamcloud.com/7882
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Steve Guminski <stephenx.guminski@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9019 osd : remove struct timeval use in osd-ldiskfs

For brw_stats_show change the output to use a timespec64
type to avoid the overflow. Also change the format to
print the sub-second portion as 9 digits (nanoseconds)
for clarity, rather than printing six digits without
leading zeroes.

Both osd_write_prep() and osd_read_prep() want to report
the time it took to perform its operations in the
osd_get_page counter. This is currently done with
a call to do_gettimeofday with struct timeval which is
not 2038 safe. Move this operation to 64 bit time
handling.

Change-Id: I457ac799d855d2596220b6e0d0c5039e8b00021f
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/24896
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8562 osp: osp_precreate_thread gets stuck after disconnect

osp_precreate_thread() can get stuck because d->opd_got_disconnected
never gets reset. When opd_got_disconnected is set,
osp_precreate_cleanup_orphans() returns early with EAGAIN and can't
clear d->opd_pre_recovering. And because d->opd_pre_recovering can't
be cleared we always break out of the while loop where
d->opd_got_disconnected normally gets reset. So
osp_precreate_cleanup_orphans() is stuck always failing.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Change-Id: I0b4f4e2e55e7a8d7ffae633a4d3c578b4a484ae2
Reviewed-on: https://review.whamcloud.com/24758
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Sergey Cheremencev <sergey.cheremencev@seagate.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8840 osp: handle EA cache properly

For success case, dt_xattr_get() should return the EA size
instead of zero. If such EA does not exist, return -ENODATA.

More code cleanup for OSP EA cache to avoid potential reference
leak, buffer overflow, and so on.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I352b99b1ed08f1b15bdb8da2bf28689ae2d61c23
Reviewed-on: https://review.whamcloud.com/23782
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6245 libcfs: remove wrappers for timer functions

Use the linux timer_list api directly instead of going
throught a abstraction layer in libcfs.

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: Ida47b2746e68b7428cbc30b40fbdc8810b7d05f7
Reviewed-on: https://review.whamcloud.com/23733
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8732 tests: lnet-selftest Error inserting modules

Prevent locally derived module option MODOPTS_LIBCFS being
passed as part of environment variable to remote nodes.
This issue is more prominent in interop setups that have
different version LIBCFS library supporting different module
parameters.

Test-Parameters: trivial testlist=lnet-selftest
Signed-off-by: Abrarahmed Momin <abrar.habib@seagate.com>
Signed-off-by: Arshad Hussain <arshad.hussain@seagate.com>
Change-Id: Ie56103c55611bb62a7dfb314526c48781ab20bc4
Seagate-bug-id: MRP-2675
Reviewed-by: Ujjwal Lanjewar <ujjwal.lanjewar@seagate.com>
Reviewed-by: Ashish Purkar <ashish.purkar@seagate.com>
Reviewed-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Tested-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Reviewed-on: https://review.whamcloud.com/23250
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-9019 lnet: simplify lnet_eq_wait_locked

We can simplify the code by taking advantage of the behavior
of schedule_timeout_interruptible(). Instead of testing if
tms is less than zero we can pass in a signed long that
schedule_timeout_interruptible is expecting and for the case
of no timeout we can pass in MAX_SCHEDULE_TIMEOUT.

Change-Id: I15126aab500e975ec34ee270d3a1f9b83cf50902
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/23147
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6142 lnet: code style cleanups, improve error handling

During submission of the dynamic LNet work to the upstream
client Dan Carpenter suggested several improvements and a
few fixes. They are:

1) Rename the goto labels to be more clear.

2) Replace CLASSERT with BUILD_BUG_ON

3) Rework the loops to reduce the code indentation level
and to make them more clear.

4) Add test to ensure the ioctl data is really smaller
than what is reported in the ioctl header.

5) Fix one off error in lnet_get_peer_info

6) Handle errors instead of sucesses in lnet_dyn_[un]configure

Change-Id: If003d313ae587fd6311d9dfb419e8c6f59ef4705
Suggested-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/18469
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Olaf Weber <olaf@sgi.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7115 fld: don't retry for no LWP device

Just try again for LWP import since other client
import may run into deadloop.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: If1759b5ad7daa06abe6231669ad92e006ecb3059
Reviewed-on: https://review.whamcloud.com/17041
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8994 lmv: honour the specified stripe index

when creating the striped directory, specified
stripe index should always be used even the parent
has default stripe index.

Signed-off-by: Di Wang <di.wang@intel.com>
Change-Id: Ief00af3790282a0887b9d28c581ccf0070cc758f
Reviewed-on: https://review.whamcloud.com/24777
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LU-8753 osp: add rpc generation

Add rpc generation to make sure current update
request will not be sent until the remote llog
object got refresh.

Signed-off-by: Di Wang <di.wang@intel.com>
Change-Id: Iae678686b522d545b69510444805a1e411acfcfe
Reviewed-on: https://review.whamcloud.com/24364
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8871 kernel: kernel upgrade [SLES12 SP2 4.4.21-84]

Major linux version upgrade, from linux 3.12 to linux 4.4.
All new ldiskfs and kernel patches.
All new target and config files.

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
mdsdistro=sles12sp2 ossdistro=sles12sp2 mdsfilesystemtype=ldiskfs \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I354197d5426ebde3da6c814bfb7f97f0f04113eb
Reviewed-on: https://review.whamcloud.com/24257
Tested-by: Jenkins
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 ptlrpc: use 64-bit times for ptlrpc sec expiry

The exp_flvr_expire and imp_sec_expire are defined as 'unsigned long',
which doesn't overflow until 2106, but to be on the safe side, this
changes the code to use time64_t like we do everywhere else.

Linux-commit: 986ef135c2afce959971b0086fc04e0f9f654120

Change-Id: Id1a560504e9a9c673e0880ac4e3e80938b3789bc
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/24709
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Sebastien Buisson <sbuisson@ddn.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

LU-8945 ptlrpc : remove userland usage from ptlrpc

The reason for __REQ_LAYOUT_USER__ was to expose a
section of code in layout.c to userland for a utility
similar to wireshark. This was done before wireshark
existed but now that it does we no longer need to do
this type of hack. This also reduces lustre_acl.h to
strictly a kernel header now.

Test-Parameters: trivial

Change-Id: I649e36947789b8ebcb1204b65ac68b55a1d6ea49
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/24396
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8835 osc: handle 64 bit time properly in osc_cache_too_much

Use 64 bit time for cl_lru_last_used and the function
osc_cache_too_much(). The use of 64 bit time will
introduce an expensive 64 bit division operation. Since
the time lapse being calculated in osc_cache_too_much
will never be more than seventy years we can cast the
time lapse to an long and we can perform a normal 32 bit
divison operation instead.

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: Ic38db2d22436fc2a8aa431562e4f5d22ec18d842
Reviewed-on: https://review.whamcloud.com/23814
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

LU-6210 lod: Change positional struct initializers to C99

This patch makes no functional changes. Struct initializers in the
lod directory that use C89 or GCC-only syntax are updated to C99
syntax.

The C99 syntax prevents incorrect initialization if values are
accidently placed in the wrong position, allows changes in the struct
definition, and clears any members that are not given an explicit
value.

The following struct initializers have been updated:

lustre/lod/lod_dev.c:
struct lu_seq_range range
struct l_wait_info lwi
lustre/lod/lproc_lod.c:
static struct lprocfs_vars lprocfs_lod_osd_vars[]

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I6533b9cbfd9a98e903ba8e0f5c5d8e725226366d
Reviewed-on: https://review.whamcloud.com/23754
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8066 obd: Add debugfs root

This is just plumbing for migrating remaining procfs to
debugfs support

Linux-commit: 5c8c82f63a11c07a0687d2c71411166017012689

ldebugfs_remove is usually called on directories with files passed in
as attributes, so simple debugfs_remove failes on them as not empty
Switch to debugfs_remove_recursive.

This fixes a number of problems where a new filesystem is mounted after
being unmounted first.

Linux-commit: 6a491f2b80f2806221ba3a5a3e26fbe945f82d83

Turns out we mistakenly export some pretty-wide-reaching debugfs
functions as EXPORT_SYMBOL instead of EXPORT_SYMBOL_GPL as we should,
so this patch rectifies the situation.

Linux-commit: da33f1dd9f69c5a5d23e1254906b027af3362e36

Move /proc/fs/lustre/devices to debugfs. The devices file prints out
status information about all obd devices in the system in human
readable form.

Linux-commit: 4361a048743f900bb0710bd7cb36a650d7bef93a

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Iefe6ee69b607f627c5682513128706bf0c1c3e09
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/23427
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>

LU-8288 lfsck: handle dangling LOV EA reference

Originally, the layout LFSCK logic of handling dangling LOV EA
reference is as following:

During the first phase scanning, if the layout LFSCK find that
some LOV EA entry references an OST-object that does not exist,
then it will repair the inconsistency based on the LFSCK start
parameter "-c" option. If "-c" option is specified, the layout
LFSCK will think the OST-object lost, then it will create the
lost OST-object with the FID that is stored in the LOV EA slot.
But such repairing may be incorrect. Because the LOV EA may be
corrupted as to the LOV EA is invalid. Means the OST-object is
still on the OST. When moves to the second stage scaning, the
layout LFSCK will find the orphan OST-object that claims to be
as one of the MDT-object's stripe. And if someone has already
modified the new created OST-object before finding the orphan,
then the layout LFSCK cannot recover the original data back.

To avoid above trouble, the patch introduces new start option:
"--delay-create-ostobj" or short described as "-d". It allows
the layout LFSCK to postpone creating the "lost" OST-object
until all the orphan OST-objects handled. It will record the
dangling references in some new introduced layout LFSCK trace
files on disk during the layout LFSCK first stage scanning,
then travel those traces file after all orphan OST-objects
handled in the second-stage scanning. The side-effect of such
option is that as long as one OST does not join the layout
LFSCK or fail to complete the scanning, then reparing dangling
LOV EA will be skipped. For a large system with a lot of OSTs,
such condition may be a bit strict. The default value is 'off'.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ic222e1ad20c8011aa5f41cc43171d017ab5c464f
Reviewed-on: https://review.whamcloud.com/21562
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6142 lnet: change wire protocol typedefs to proper structure

The upstream kernel requires proper structures so
convert all the LNet wire protocols typedefs in
the LNet core.

Test-Parameters: trivial

Change-Id: I0287e0d16a1faa7407dc63f9cd607b94ff49d980
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/24566
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Reviewed-by: Olaf Weber <olaf@sgi.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>

LU-8752 lnet: Stop MLX5 triggering a dump_cqe

We have found that MLX5 will trigger a dump_cqe if we don't
invalidate the rkey on a newly alloated MR for FastReg usage.

This fix just tags the MR as invalid on its creation if we are
using FastReg and that will force it to do an invalidate of the
rkey on first usage.

Test-Parameters: trivial
Signed-off-by: Doug Oucharek <doug.s.oucharek@intel.com>
Change-Id: Id0de4f799d70c58011746520b20c8cfca6a29e6a
Reviewed-on: https://review.whamcloud.com/24306
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8569 lfsck: handle linkEA overflow

If the linkEA is marked as overflow (32 bits timestamp in header),
and even if some hard links are removed, we still cannot clear the
overflow timestamp, because the missed hard links entries are still
not in the linkEA. It is the namespace LFSCK's duty to add the missed
entries into the linkEA (if possible) and clear the overflow timestamp
if all the hard links entries are in the linkEA. But it is possible
that there are some new hard links added during the namespace LFSCK
scanning. So the namespace LFSCK needs to compare the linkEA overflow
timestamp with the namespace LFSCK latest reset time (scan the system
from the beginning) in the 2nd-stage scanning. If the latter one is
newer, then means all the hard links entries have been in the linkEA,
so the linkEA overflow timestamp can be cleared.

To avoid the trouble caused by clock drift among MDTs, the linkEA
overflow timestamp will be set as the MDT local time on which the
object resides even if the set linkEA operation is sponsored by
some remote MDT. So we can directly compare the linkEA overflow
time with the namespace LFSCK latest reset time.

The old interfaces between MDD/OUT and LFSCK via lfsck_in_notify()
for the linkEA overflow event are obsoleted and removed.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ia2c3f9d0a0ecb0b3285041f1365cb4e075a07dda
Reviewed-on: https://review.whamcloud.com/23741
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-7812 tests: address race condition for sanity:120{e,f}

By adding a delay between lock cancel and stat, it can avoid the
problem that the locks are still held by ldlm lock thread while
'cancel_lru_locks osc' is called.

Test-Parameters: trivial envdefinitions="ONLY=120e" testlist=sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity,sanity
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I2095cb1df05ff45c2671edaed5a61bcdd4a56266
Reviewed-on: https://review.whamcloud.com/24811
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8966 tests: rename sanity 39, 43, 44, 241 tests

The following tests are renamed to be run separately
from other tests in these groups:
test_39() to test_39a()
test_43() to test_43A()
test_44() to test_44A()
test_241() to test_241a()

Test-Parameters: trivial testlist=sanity
Signed-off-by: Elena Gryaznova <elena.gryaznova@seagate.com>
Change-Id: I7b841da0dc2d307e3174dddccc033c8dd37c27fd
Reviewed-on: https://review.whamcloud.com/24485
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8969 llite: sanity test_56x read failed: Invalid argument

In generic_file_read_iter() does not check EOF before invoke
direct_IO callback. So we have to check it by ourselves.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Ia30d2a42da81beb6c5217f081c18ce4d93553540
Reviewed-on: https://review.whamcloud.com/24552
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8973 tests: in sanity 103a use $tdir instead of $MOUNT

In sanity.sh test_103a we should use $tdir instead of $MOUNT
in case of any unclean testing directory.

Test-Parameters: trivial
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I891bbc0aab28bb58589b66405e0ad5c40fdad25f
Reviewed-on: https://review.whamcloud.com/24518
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8957 obdclass: fix exp_nid_hash race

Move exp_nid_hash unhash operation under exp_lock. To avoid
parallel export disconnect may race in there.

Change-Id: I02a9cf550b4edee7c42ba90d3f3926cd2584c24b
Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-on: https://review.whamcloud.com/24434
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8378 obd: move s3 in lmd_parse to inner loop

The upstream client complains with:

obdclass/obd_mount.c: In function lmd_parse:
obdclass/obd_mount.c:880: warning: variable set but not used

The solution is to move s3 to the inner loop
where it is only used.

Test-Parameters: trivial

Change-Id: I95280d08e5f00cf5fdae62acb21e2c08805a9948
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/23820
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>

LU-8635 tests: Use wait_update to expire jobstats

Instead of arbitarily adjusting sleep count, use wait_update()
with maximum of (left+5) seconds to expire old jobstats.

Test-Parameters: trivial
Seagate-bug-id: MRP-2781
Signed-off-by: Ashish Purkar <ashish.purkar@seagate.com>
Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: I75e017fc75d35faf1754faddfed168c0fbf432fe
Reviewed-on: https://review.whamcloud.com/22699
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8753 llog: remove lgh_write_offset

Remove lgh_write_offset for remote llog object,
since it is not easy to be refreshed when remote
transaction fails, which might cause llog file
corruption. Instead let's rely on the osp object
cache to invalidate and update the llog object
size.

Signed-off-by: Di Wang <di.wang@intel.com>
Change-Id: I38aa9ce2c5ac6485bf0cf6be023b8edde986b5d3
Reviewed-on: https://review.whamcloud.com/24008
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6854 llite: Do not set writeback for sync write pages

Sync-write pages don't have Dirty bit set, it will confuse newer
kernels by having writeback bit set.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: I2613919a9e942a7ee3dc48b250f7f0e5e4594e48
Reviewed-on: https://review.whamcloud.com/15610
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>

Revert "LU-8922 lod: check master stripes properly"

It causes frequent testing failures tracked in LU-8985

This reverts commit 1be03a1088d2aa583c2651435634ba336b48f939.

Test-Parameters: testlist=conf-sanity,conf-sanity,conf-sanity,conf-sanity,conf-sanity,conf-sanity

Change-Id: I444ce9e65c72db7db3f53692d9fe4f823943fade
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Reviewed-on: https://review.whamcloud.com/24750
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 lnet: use 64-bit time for selftest

The lustre selftest code has multiple time stamps that are kept
as 'time_t' or 'unsigned long' and can therefore overflow on
32-bit systems.

This changes the code to use time64_t instead.

Linux-commit: d9f79e6bf02c109f117132163239bfffa6475ccb

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Change-Id: I74d704d104e7398af411e73180fac57c0b943eb7
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/23352
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Frank Zago <fzago@cray.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8965 kernel: kernel update [SLES11 SP4 3.0.101-91]

Update SLES11 SP4 kernel to 3.0.101-91

Test-Parameters: mdsdistro=sles11sp4 ossdistro=sles11sp4 \
  clientdistro=sles11sp4 mdsfilesystemtype=ldiskfs \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs \
  testgroup=review-ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I8b7aa39f60626a511b58d1a7a9a31189c17b4f1a
Reviewed-on: https://review.whamcloud.com/24477
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Trent Geerdes <trent.geerdes@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6401 header: remove libcfs.h in interval_tree.h

The header libcfs.h for interval_tree.h is not needed
so we can just use the standard linux kernel headers
instead.h

Test-Parameters: trivial

Change-Id: I9c2c4f7e9bc227876818ff7213f392bdeb1112e0
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/24323
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Frank Zago <fzago@cray.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8889 llite: replace memory allocation with ll_env usage

Replace memory allocation with ll_env usage in ll_file_read
like in other functions.

Change-Id: I44445ab80cc856f45271e0888ea2538d2a44cdeb
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-on: https://review.whamcloud.com/24064
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

LU-6210 mgs: Change positional struct initializers to C99

This patch makes no functional changes. Struct initializers in the
mgs directory that use C89 or GCC-only syntax are updated to C99
syntax. Whitespace is corrected to match the coding style guidelines.

The C99 syntax prevents incorrect initialization if values are
accidently placed in the wrong position, allows changes in the struct
definition, and clears any members that are not given an explicit
value.

The following struct initializers have been updated:

lustre/mgs/mgs_handler.c:
struct lustre_handle lockh
struct lustre_handle conn
lustre/mgs/mgs_llog.c:
struct mgs_fsdb_handler_data d

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I36cbeaae50982b6f72d271997b229c46d085a6ee
Reviewed-on: https://review.whamcloud.com/23794
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6210 fid: Change positional struct initializers to C99

This patch makes no functional changes. Struct initializers in the
fid directory that use C89 or GCC-only syntax are updated to C99
syntax.

The C99 syntax prevents incorrect initialization if values are
accidently placed in the wrong position, allows changes in the struct
definition, and clears any members that are not given an explicit
value.

The following struct initializers have been updated:

lustre/fid/fid_lib.c:
const struct lu_seq_range LUSTRE_SEQ_SPACE_RANGE
const struct lu_seq_range LUSTRE_SEQ_ZERO_RANGE
lustre/fid/lproc_fid.c:
struct lu_seq_range tmp

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I73b6a37a0fcac29ba48410e7befb98f13bdb17cc
Reviewed-on: https://review.whamcloud.com/23789
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6210 osd: Change positional struct initializers to C99

This patch makes no functional changes. Struct initializers in the
osd directories that use C89 or GCC-only syntax are updated to C99
syntax.

The C99 syntax prevents incorrect initialization if values are
accidently placed in the wrong position, allows changes in the struct
definition, and clears any members that are not given an explicit
value.

The following struct initializers have been updated:

lustre/osd-ldiskfs/osd_scrub.c:
static const struct osd_lf_map osd_lf_maps[]
static const struct osd_lf_map osd_dl_maps[]
lustre/osd-zfs/osd_oi.c:
static const struct named_oid oids[]

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I1aa2bf64f0efcbd9c34f21f60c462f7a95c41f98
Reviewed-on: https://review.whamcloud.com/23770
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6210 ldlm: Change positional struct initializers to C99

This patch makes no functional changes. Struct initializers in the
ldlm directory that use C89 or GCC-only syntax are updated to C99
syntax.

The C99 syntax prevents incorrect initialization if values are
accidently placed in the wrong position, allows changes in the struct
definition, and clears any members that are not given an explicit
value.

The following struct initializers have been updated:

lustre/ldlm/ldlm_extent.c:
struct interval_node_extent limiter
struct interval_node_extent ext

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I219f5b2bd4b96d92eaf120f7080e762d72eb3105
Reviewed-on: https://review.whamcloud.com/23743
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6210 mdt: Change positional struct initializers to C99

This patch makes no functional changes. Struct initializers in the
mdt directory that use C89 or GCC-only syntax are updated to C99
syntax.

The C99 syntax prevents incorrect initialization if values are
accidently placed in the wrong position, allows changes in the struct
definition, and clears any members that are not given an explicit
value.

The following struct initializers have been updated:

lustre/mdt/mdt_lvb.c:
struct ldlm_valblock_ops mdt_lvbo
lustre/mdt/mdt_mds.c:
static struct lprocfs_vars lprocfs_mds_obd_vars[]

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: If8b3b6233e34f6e8406ab6b8e4e70845bf94bfc8
Reviewed-on: https://review.whamcloud.com/23702
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6210 obd: Change positional struct initializers to C99

This patch makes no functional changes. Struct initializers in the
obd directory that use C89 or GCC-only syntax are updated to C99
syntax. Whitespace is changed to match the code style guidelines.

The C99 syntax prevents incorrect initialization if values are
accidently placed in the wrong position, allows changes in the struct
definition, and clears any members that are not given an explicit
value.

The following struct initializers have been updated:

lustre/obdclass/lprocfs_jobstats.c:
static const struct seq_operations lprocfs_jobstats_seq_sops
lustre/obdclass/obd_config.c:
struct llog_process_cat_data
lustre/obdclass/obd_mount.c:
struct lustre_mount_data2 lmd2 (2 occurrences)

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: I74a7d6ebb250e8df0f24f60ce87933fddf325c30
Reviewed-on: https://review.whamcloud.com/23697
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6210 tests: Change positional struct initializers to C99

This patch makes no functional changes. Struct initializers in the
tests directory that use C89 or GCC-only syntax are updated to C99
syntax.

The C99 syntax prevents incorrect initialization if values are
accidently placed in the wrong position, allows changes in the struct
definition, and clears any members that are not given an explicit
value.

The following struct initializers have been updated:

lustre/tests/checkfiemap.c:
struct option long_opts[]
lustre/tests/llapi_layout_test.c:
static struct test_tbl_entry test_tbl[]
lustre/tests/mcreate.c:
struct option opts[]
lustre/tests/mmap_sanity.c:
struct test_case tests[]
lustre/tests/mpi/mdsrate.c:
struct option longOpts[]
lustre/tests/statmany.c:
struct option longopts[]

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Iadc141507664c97dcc3112c382d4e379a1bf8041
Reviewed-on: https://review.whamcloud.com/23568
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Frank Zago <fzago@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6210 snmp: Change positional struct initializers to C99

This patch makes no functional changes. Struct initializers in the
snmp directory that use C89 or GCC-only syntax are updated to C99
syntax.

The C99 syntax prevents incorrect initialization if values are
accidently placed in the wrong position, allows changes in the struct
definition, and clears any members that are not given an explicit
value.

The following struct initializers have been updated:

snmp/lustre-snmp.c:
struct variable7 clusterFileSystems_variables[]
struct oid_table osd_table[]
struct oid_table osc_table[]
struct oid_table mds_table[]
struct oid_table mdc_table[]
struct oid_table cli_table[]
struct oid_table lov_table[]
struct oid_table ldlm_table[]

Test-Parameters: trivial
Signed-off-by: Steve Guminski <stephenx.guminski@intel.com>
Change-Id: Iaedd0165bca2a2af2ce6a0f535602a85c6217da8
Reviewed-on: https://review.whamcloud.com/23463
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Frank Zago <fzago@cray.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8066 obd: Move /proc/fs/lustre root level files to sysfs

except devices, for now.

Linux-commit : 8b82844505699f10f80c7e554364e3d0618cd77d

Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Change-Id: Iacdba1e0f703eb90688c1b93a2ef81c0c8ad7a91
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/23425
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8066 obd: Generic helpers for sysfs

Add generic helpers to allow displaying of lustre-specific values
in /sys/fs/lustre

except devices, for now.

Linux-commit : 5e66f70e963e4ba2266071343f5a153c7db4327b

Change-Id: Idfbf419d8218546463b993a8031ef00609f8dce9
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/22209
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6401 headers: move lu_fid, ost_id funcs out of lustre_idl.h

Move lu_fid functions into lustre/lustre_fid.h
Move ost_id functions into lustre/lustre_ostid.h
Fix indenting, include new headers as needed

Signed-off-by: Ben Evans <bevans@cray.com>
Change-Id: Idcbb95f614829b434c48a476172eedb421fe488f
Reviewed-on: https://review.whamcloud.com/22712
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Frank Zago <fzago@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8765 ptlrpc: update replay cursor when close during replay

The replay cursor should be updated properly when close happened
during replay, otherwise, ptlrpc_replay_next() could run into a
dead loop due to an invalid replay cursor:

- replay cursor is moved to an open request during replay;
- application close that open file, so the rq_replay of the open
  request is cleared;
- ptlrpc_replay_next() calls ptlrpc_free_committed() to free
  committed/closed requests, the open request is removed from
  the committed list, so the replay cursor is changed to an
  empty list_head now. The open request won't be freed now since
  it's still held by the pending close request;
- ptlrpc_replay_next() continue to move the replay cursor to
  next and run into a dead loop at the end;

Another change in this patch is to remove the out of date comments
in ptlrpc_replay_next() and cover the whole process of finding
replay request within imp_lock, because:

1. With two separated replay lists and replay cursor introduced,
   finding replay request won't take much time as before, it's
   not necessary to do this "lock -> unlock -> lock -> unlock"
   trick anymore;

2. Nowadays there are various kind of non-replay requests are
   allowed during recovery, so ptlrpc_free_committed() may run in
   parallel to remove an open request while ptlrpc_replay_next()
   is iterating the open requests list;

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I358c4f55dbf38b0c9122c3aff40cf9ec96619d6a
Reviewed-on: https://review.whamcloud.com/23418
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 libcfs: use 64-bit times for cfs_srand seed

Several functions in Lustre call cfs_srand with do_gettimeofday
as the seed to get a pseudo-random number.
There is no bug here, but changing it to use ktime_get_ts64()
gets us closer to deprecating do_gettimeofday() and makes it slightly
more random.

Affected functions are:
lnet_shuffle_seed, init_lustre_lite and class_handle_init

Linux-commit: 1f4fc343c008981d3a91351c030e2c29f5cd1b1c

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Change-Id: I86d156afd71456d7d5412ebb8a2e519367d4f81d
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/23351
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Frank Zago <fzago@cray.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 ptlrpc: use 64-bit time for adaptive timeout

The adaptive timeout handling stores absolute times in 32-bit time_t
quantities, which will overflow in 2038.

This changes it to use time64_t.

Linux-commit: 0ac0478b65de1e5b8b62f608bdf1d8ba631acc4d

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Change-Id: I50d1090865af8970276d039281fc2ca0b87fa8af
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/23270
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 ldlm: use time64_t for l_last_activity

The l_last_activity struct member is used to keep track lock hold
times, and it is printed for debugging purposes. For the elapsed
time, we can use 'long' here, but it's better to use time64_t
for storing the real time to avoid an overflow in 2038.

Update server side code as well that is not present in upstream
client.

Linux-commit: bf6d21539d58f64812d71c956ecec3d67650b2ee

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Change-Id: Idbbc64afbb9307e6133c671a7aa2122318c62679
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/23141
Reviewed-by: Frank Zago <fzago@cray.com>
Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 libcfs: tracefile document seconds overflow

The lustre tracefile has a timestamp defined as

__u32 ph_sec;
__u64 ph_usec;

which seems completely backwards, as the microsecond portion of
a time stamp will always fit into a __u32 value, while the second
portion will overflow in 2038 or 2106 (in case of unsigned seconds).

Changing this would unfortunately change the format in an incompatible
way, breaking all existing user space tools that access the data.

This uses ktime_get_real_ts64() to replace the insufficient
do_gettimeofday() and then truncates the seconds portion to
an u32 type, along with comments to explain the result.

A possible alternative would be the use of ktime_get_ts64() to
read a monotonic timestamp that never overflows, but this would
trigger a check in user space 'hdr->ph_sec < (1 << 30)' that
attempts to ensure that the values are within a reasonable range.

Linux-commit: 65f28840e4c653b7d78feabcb65280c09d3d0d23

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Change-Id: I5fdf6d09d39fbb4da8ccba51a745b00996f116f7
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/23137
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Frank Zago <fzago@cray.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-4423 libcfs: use 64-bit inode timestamps internally

Lustre has 64-bit timestamps in its network data structures, but
on 32 bit systems, it converts them directly into time_t, which is
32 bit wide.

This changes the code to use 64-bit time stamps for files. The Linux
VFS code still uses time_t though, and will be changed in a separate
patch series.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Change-Id: I6d3e986fa4eac2fe76a6d66a55f2cdbfe8fdae55
Signed-off-by: Oleg Drokin <green@linuxhacker.ru>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/23136
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-6245 libcfs: add ktime_get_real_seconds support

In newer kernels time support is moving to u64 to avoid
the 2038 limitation. The function get_seconds() which
only returns a 32 bit value is being replaced by a new
function ktime_get_real_seconds(). This patch adds a
wrapper for older platforms that lack this function.
Only the libcfs/LNet layer has been ported in this
patch since it is a huge change.

Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Change-Id: Ic16859f1b052bfb72299cc85fe50364a9e70d660
Reviewed-on: https://review.whamcloud.com/22866
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Frank Zago <fzago@cray.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8314 lfs: improve getdirstripe interface

* remove unused options '-v' '-R' '-q' for "lfs getdirstripe".
* add "[--mdt-hash|-t]" option for "lfs getdirstripe", and print
hash_type by default.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Ib7eaf00744d0cd5dec13e683c39e2ecf1d54808b
Reviewed-on: https://review.whamcloud.com/24319
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

LU-8922 lod: check master stripes properly

When creating striped directory, it should
first check if the stripe has been created
on the current MDT, otherwise it might create
duplicate stripes on the master MDT, especially
when one MDT is deactived, and specified stripes
is larger than the active MDTs.

Signed-off-by: Di Wang <di.wang@intel.com>
Change-Id: I089f5cf517c71816151e8ece1b36b9064ee3b7f2
Reviewed-on: https://review.whamcloud.com/24247
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>