LU-14487 lustre: remove references to Sun Trademark. "lustre" is no longer a Trademark of Sun Microsystems. There is no need to acknowledge the trademark is every file, so just remove all these claims. Test-Parameters: trivial Signed-off-by: Mr NeilBrown <neilb@suse.de> Change-Id: I214670b39c5718f2b691193f268a64856e0cd743 Reviewed-on: https://review.whamcloud.com/41880 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org>
LU-12828 ldlm: FLOCK request can be processed twice Original request can be processed after resend request, so it can create a lock on MDT without client lock or unlock other lock. Make flock enqueue to use modify RPC slot. Change-Id: Icfee202fe2e389beda1116f78f8b933c7ea182fb Cray-bug-id: LUS-5739 Signed-off-by: Andriy Skulysh <c17819@cray.com> Signed-off-by: Vitaly Fertman <c17818@cray.com> Reviewed-by: Alexander Boyko <c17825@cray.com> Reviewed-by: Andrew Perepechko <c17827@cray.com> Reviewed-on: https://review.whamcloud.com/36340 Tested-by: jenkins <devops@whamcloud.com> Reviewed-by: Alexandr Boyko <c17825@cray.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-11444 ptlrpc: resend may corrupt the data Late resend if arrives much later than another modification RPC which has been already handled on this slot, may be still applied and therefore overrides the last one Send RPCs from client in increasing order for each tag and check it on server to check late resend. A slot can be reused by a client after kill while the server continue to rely on it. Add flag for such obsolete requests, here we trust the client and perform xid check for all in progress requests. Cray-bug-id: LUS-6272, LUS-7277, LUS-7339 Signed-off-by: Andriy Skulysh <c17819@cray.com> Reviewed-by: Vitaly Fertman <c17818@cray.com> Reviewed-by: Alexander Boyko <c17825@cray.com> Reviewed-by: Andrew Perepechko <c17827@cray.com> Change-Id: I97806577cf979f49a75379ffc55947cc3dcd02b1 Reviewed-on: https://review.whamcloud.com/35114 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Alexandr Boyko <c17825@cray.com> Reviewed-by: Mike Pershin <mpershin@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-6142 lustre: introduce CONFIG_LUSTRE_FS_POSIX_ACL Lustre (or any file system) should not conditionally compile code based on CONFIG_FS_POSIX_ACL. This config option enables library support. A file system can define it's own config option, which then selects CONFIG_FS_POSIX_ACL (if needed). It should act on its own option, not the library one. This patch makes that change. While lustre is out-of-tree, it cannot select CONFIG_FS_POSIX_ACL and must work with the configruation of the kernel it is being built against. So the new CONFIG_LUSTRE_FS_POSIX_ACL can only be selected if the based kernel has CONFIG_FS_POSIX_ACL selected. With that restriction this change becomes little more than an indirection. However it means that the out-of-tree code can be closer to the (eventual) in-tree code. Test-Parameters: trivial Signed-off-by: NeilBrown <neilb@suse.com> Change-Id: I3ba283f295d1c7217e7ff9917573be0fe92bb2ca Reviewed-on: https://review.whamcloud.com/36085 Tested-by: jenkins <devops@whamcloud.com> Reviewed-by: Neil Brown <neilb@suse.de> Reviewed-by: Petros Koutoupis <pkoutoupis@cray.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Arshad Hussain <arshad.super@gmail.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-6864 mdc: move RPC semaphore code to lustre/osp The "MDC RPC semaphore" is no longer used by MDC code since patch http://review.whamcloud.com/14374 "LU-5319 mdc: manage number of modify RPCs in flight" landed. It is only still used by the OSP currently. While there are plans to remove this from the OSP as well, it makes sense to move all of this code from MDC to OSP so that it will also be cleaned up when that functionality lands. Signed-off-by: Andreas Dilger <andreas.dilger@intel.com> Change-Id: I1c4c9a45f308a29600631ac7fed40e592b3ebbe5 Reviewed-on: https://review.whamcloud.com/32412 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com> Reviewed-by: James Simmons <uja.ornl@yahoo.com> Reviewed-by: Mike Pershin <mpershin@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-6401 uapi: migrate remaining uapi headers to uapi directory Move all the remaining lustre headers shared between user land and kernel space to the uapi directory. OBD_MAX_IOCTL_BUFFER can be removed since the utilites don't even acknowledge it value. Change-Id: I6dcdf3f824dae6bc8a0786f0e61787ca68c10bd8 Signed-off-by: James Simmons <uja.ornl@yahoo.com> Reviewed-on: https://review.whamcloud.com/25246 Tested-by: Jenkins Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Ben Evans <bevans@cray.com> Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-8648 all: remove all Sun license and URL references The link to the GPL license at Sun is no longer there. Also Sun Inc is gone so lets remove all references to it in our source code. Test-Parameters: trivial Change-Id: Ib47d07dcef49a1ef624d0d8a4a91b9731f486278 Signed-off-by: James Simmons <uja.ornl@yahoo.com> Reviewed-on: http://review.whamcloud.com/22800 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: John L. Hammond <john.hammond@intel.com> Reviewed-by: Frank Zago <fzago@cray.com>
LU-5319 mdc: manage number of modify RPCs in flight This patch is the main client part of a new feature that supports multiple modify metadata RPCs in parallel. Its goal is to improve metadata operations performance of a single client, while maintening the consistency of MDT reply reconstruction and MDT recovery mecanisms. It allows to manage the number of modify RPCs in flight within the client obd structure and to assign a virtual index (the tag) to each modify RPC to help server side cleaning of reply data. The mdc component uses this feature to send multiple modify RPCs in parallel. Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net> Change-Id: Ia707e39770e479648627611a99d0724e7070baec Reviewed-on: http://review.whamcloud.com/14374 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-6017 obd: remove destroy cookie handling Clients no longer need to track the max and default MDS cookiesizes so remove the obsolete obod o_valid flag OBD_MD_FLCOOKIE, the struct client_obd members cl_{default,max}_mds_cookiesize, the struct obd_trans_info and parameters of this type, the get info keys KEY_{DEFAULT,MAX}_COOKIESIZE, the cookiesize parameters from md_init_ea_size(), the files llite/*/{default,max}_cookiesize, and any code that needlessly handled these values. Signed-off-by: John L. Hammond <john.hammond@intel.com> Change-Id: I95f0dae370998006ec366a252891db531759ea7d Reviewed-on: http://review.whamcloud.com/12922 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Lai Siyao <lai.siyao@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
LU-5549 mdc: cl_default_mds_easize not refreshed The client_obd::cl_default_mds_easize field should track the largest observed EA size advertised by the MDT, subject to a reasonable upper bound. The MDC uses cl_default_mds_easize to calculate the initial size of request buffers. The default value should be small enough to avoid wasted memory and excessive use of vmalloc(), yet large enough to accommodate the common use case. In the current code, the default value is only updated if client_obd::cl_max_mds_easize is strictly less than mdt_body::mbo_max_mdsize. This condition is almost never met, because client_obd::cl_max_mds_easize is computed at client mount-time based on the number of OSTs in the filesystem, so the MDT won't ever observe and advertise an EA size larger than that. As a result, client_obd::cl_default_mds_easize indefinitely retains its initial value, which is computed at client mount-time based on the filesystem's default stripe width. Any getattr() requests for widely striped files will consequently allocate a request buffer that is too small, forcing reallocations on both the client and server side. To avoid this, update client_obd::cl_default_mds_easize independently of the value of client_obd::cl_max_mds_easize. In addition, this patch includes these changes: - Add comments to the client_obd structure to clarify what the cl_{default,max}_mds_{cookie,ea}size values mean. - Prevent mdc_get_info() from storing uninitialized data in client_obd::cl_max_mds_cookiesize. - Use 4096 as an upper bound for the default values. The former bound of PAGE_CACHE_SIZE is too large on 64k-page platforms (i.e. PPC), so it fails to prevent the vmalloc() spinlock contention described in LU-3338. The new value was chosen to be large enough to accommodate common use cases while staying well below the 16k threshold at which allocations start using vmalloc(). - Add test case 27E to ./lustre/tests/sanity.sh. Signed-off-by: Ned Bass <bass6@llnl.gov> Signed-off-by: Kyle Blatter <kyleblatter@llnl.gov> Change-Id: I363017844d6af3e6b67b7c03bd206226f9495116 Reviewed-on: http://review.whamcloud.com/11614 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Lai Siyao <lai.siyao@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-6068 misc: update Intel copyright messages 2014 Update the copyright messages for changes made in 2014. In some cases, changes made in 2013 did not generate an updated copyright message, so these files are updated accordingly. In some cases there were no licenses on newly-added files, so add them (either with new copyrights, or including copyrights from the source files). Some types of changes that are not actually changing the code (e.g. whitespace, mass changes to add or remote prefixes, patches updating the copyright messages, patch reversions and the reverted patch, patches that only delete code, etc.) have been excluded. The updatecw.sh script was updated to fix a number of problems: - it now handles comments in nroff (man page) files properly - remove need to handle non-Intel copyrights specially - allow subdirectory to be specified on command-line - improve script debugging when running with "sh -vx" - print patches changing the most files, to catch mass replacements There are still a few cases that need to be fixed up by hand (e.g. files split up and "git log --follow" can't find the original). Signed-off-by: Andreas Dilger <andreas.dilger@intel.com> Change-Id: Ib4875400a4908dc79ddc8b82ab0fe3ee903ebbe5 Reviewed-on: http://review.whamcloud.com/13174 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2675 mdc: remove mdc_sendpage() Remove the unused function mdc_sendpage() and supporting crud. Signed-off-by: John L. Hammond <john.hammond@intel.com> Change-Id: I09792c5b25f10351eead9e67d42de5821f4d7bc1 Reviewed-on: http://review.whamcloud.com/12893 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com> Reviewed-by: Bob Glossman <bob.glossman@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2675 lustre: move lustre_intent.h to lustre/include Move lustre_intent.h from lustre/include/linux to lustre/include. Signed-off-by: John L. Hammond <john.hammond@intel.com> Change-Id: I8b3947bc558e6721e4a238526497124d3ad193d0 Reviewed-on: http://review.whamcloud.com/11499 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Bob Glossman <bob.glossman@intel.com>
LU-2675 build: assume __linux__ and __KERNEL__ Assume that __linux__is defined everywhere and that __KERNEL__ is defined in most of lustre/. Signed-off-by: John L. Hammond <john.hammond@intel.com> Change-Id: Ia04e7ed4c3ab3e8ca205e14eaa1824536aedd1e3 Reviewed-on: http://review.whamcloud.com/11437 Reviewed-by: Bob Glossman <bob.glossman@intel.com> Tested-by: Jenkins Reviewed-by: Oleg Drokin <oleg.drokin@intel.com> Tested-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4906 llite: read page from LMV/MDC for readdir This patch will change readdir iteration from read entry to read page. During rm -rf, the statahead thread will enqueue the lock for each entry in mdc_read_entry, in the mean time, MDT will revoke the lock when it deletes each entry, which causes ldlm lock ping pong behavior, and a lot extra RPC. With this patch, llite will read one page from MDC each time, then stat ahead thread will hold the page and do getattr for all entries in this page. Note: it does not need to hold ldlm lock after it get the page, because stat ahead thread can live with -ENOENT or ESTALE, if the entry becomes stale during this time. With this patch, the performance of "rm -rf" will be improved for current master. 1. without this patch on master [root@mds tests]# ./createmany -o /mnt/lustre/test1/f- 10000 total: 10000 creates in 11.98 seconds: 835.02 creates/second [root@mds tests]# ls /mnt/lustre/test1/ | wc 10000 10000 68890 [root@mds tests]# time rm -rf /mnt/lustre/test1 real 1m4.496s user 0m0.014s sys 0m3.832s 2. with this patch on master [root@mds tests]# ./createmany -o /mnt/lustre/test1/f- 10000 total: 10000 creates in 11.72 seconds: 853.50 creates/second [root@mds tests]# time rm -rf /mnt/lustre/test1 real 0m8.151s user 0m0.012s sys 0m2.242s Signed-off-by: wang di <di.wang@intel.com> Change-Id: I189f0039b5624455b70d1381b43d452cc8766543 Reviewed-on: http://review.whamcloud.com/10622 Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: John L. Hammond <john.hammond@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2675 mdt: add mbo_ prefix to members of struct mdt_body Rename each member of struct mdt_body, adding the prefix mbo_. Signed-off-by: John L. Hammond <john.hammond@intel.com> Change-Id: If229b803a5ccdb27378f6fc44445c1c9bf120d8b Reviewed-on: http://review.whamcloud.com/10202 Tested-by: Jenkins Reviewed-by: Bob Glossman <bob.glossman@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Mike Pershin <mike.pershin@intel.com>
LU-4687 obdclass: unified flow control interfaces Unify the flow control interfaces for MDC RPC, FLD RPC and lfsck remote update (via OSP). We allow to adjust the maximum inflight RPCs count via /proc interface. Rename some variables/functions to make them easily understood. Signed-off-by: Fan Yong <fan.yong@intel.com> Change-Id: I096a1de17fe226bac2fe3a3c891365c2e2c51ef3 Reviewed-on: http://review.whamcloud.com/9562 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Niu Yawei <yawei.niu@intel.com> Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3338 llite: Limit reply buffer size When allocating a reply buffer for the striping information don't assume the unlikely worst case. Instead, assume the common case and size the buffer based on the observed default ea/cookie size. The default size is initialized to a single stripe and allowed to grow up to an entire page if needed. This means that for smallish filesystems (less than ~21 OSTs) where the worst case striping information can fit in a single page there is effectively no change. Only for larger filesystem will the default be less than the maximum. This has a number of advantages. * By limiting the default reply buffer size we avoid always vmalloc()'ing the buffer because it exceeds four pages in size and instead kmalloc() it. This prevents the client from thrashing on the global vmalloc() spin lock. * A reply buffer of exactly the right size (no larger) is allocated in the overflow case. These larger reply buffers are still unlikely to exceed the 16k limit where a vmalloc() will occur. * Saves memory in the common case. Wide striped files exceeded the default are expected to be the exception. The reason this patch works is because the ptlrpc layer is smart enough to reallocate the reply buffer when an overflow occurs. Therefore the client doesn't have to drop the incoming reply and send a new request with a larger reply buffer. It's also worth mentioning that the reply buffer always contains a significant amount of extra padding because they are rounded up to the nearest power of two. This means that even files striped wider than the default have a good chance of fitting in the allocated reply buffer. Also remove client eadatasize check in mdt xattr packing because as said above client can handle -EOVERFLOW. Change-Id: Ic6bab2aca208db1f4abcd87974a5879d06d0f2f1 Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Lai Siyao <lai.siyao@intel.com> Reviewed-on: http://review.whamcloud.com/6339 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: James Simmons <uja.ornl@gmail.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Bob Glossman <bob.glossman@intel.com>
LU-2430 mdd: add lfs mv to migrate inode. Add lfs mv to migrate the individual inode from one MDT to another MDT, and this function will only migrate inode layout on MDT but not touch data object on OST. lfs mv -M 1 /mnt/lustre/test1 #move test1 to MDT1. The directory will be migrated from top to the bottom, i.e. migrating parent first, then migrating the child. Add migrate into sanity 17n to check on-disk format. Add sanity 230(c,d,e,f) sanityn 80 to for migration. Signed-off-by: wang di <di.wang@intel.com> Change-Id: Ib4456a1db8909bd96260c67fff48922081948dcd Reviewed-on: http://review.whamcloud.com/6662 Tested-by: Jenkins Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: John L. Hammond <john.hammond@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4209 utils: fix O_TMPFILE/O_LOV_DELAY_CREATE conflict In kernel 3.11 O_TMPFILE was introduced, but the open flag value conflicts with the O_LOV_DELAY_CREATE flag 020000000 added to fix LU-812 in Lustre 2.4. O_LOV_DELAY_CREATE allows applications to defer file layout and object creation from open time (the default) until it can instead be specified by the application using an ioctl. Instead of trying to find a non-conflicting O_LOV_DELAY_CREATE flag or define a Lustre-specific flag that isn't of use to most/any other filesystems, use (O_NOCTTY|FASYNC) as the new value. These flags are not meaningful for newly-created regular files and should be ok since O_LOV_DELAY_CREATE is only meaningful for new files. I looked into using O_ACCMODE/FMODE_WRITE_IOCTL, which allows calling ioctl() on the minimally-opened fd and is close to what is needed, but that doesn't allow specifying the actual read or write mode for the file, and fcntl(F_SETFL) doesn't allow O_RDONLY/O_WRONLY/O_RDWR to be set after the file is opened. We will keep the 0100000000 flag for backward compatibility until 3.13 is the oldest client kernel that is supported, but drop the conflicting __O_TMPFILE value of 02000000 since that will cause an error when running on newer kernels. The 020000000 has only been used since Lustre 2.4.0 and always in conjunction with 0100000000, so any apps that used O_LOV_DELAY_CREATE directly instead of calling llapi_file_create*() will still work until Linux 3.13 is used. Signed-off-by: Andreas Dilger <andreas.dilger@intel.com> Change-Id: I565f3454616edc60c6acee01034aa5d773500c1e Reviewed-on: http://review.whamcloud.com/8312 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Bob Glossman <bob.glossman@intel.com> Reviewed-by: Peng Tao <bergwolf@gmail.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>