LU-2675 build: remove liblustre and libsysio Remove the lustre/liblustre and libsysio subdirectories. Signed-off-by: John L. Hammond <john.hammond@intel.com> Change-Id: I53c06b8c76955519a33a7b1292b0d87495105607 Reviewed-on: http://review.whamcloud.com/10657 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Bob Glossman <bob.glossman@intel.com> Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4906 llite: read page from LMV/MDC for readdir This patch will change readdir iteration from read entry to read page. During rm -rf, the statahead thread will enqueue the lock for each entry in mdc_read_entry, in the mean time, MDT will revoke the lock when it deletes each entry, which causes ldlm lock ping pong behavior, and a lot extra RPC. With this patch, llite will read one page from MDC each time, then stat ahead thread will hold the page and do getattr for all entries in this page. Note: it does not need to hold ldlm lock after it get the page, because stat ahead thread can live with -ENOENT or ESTALE, if the entry becomes stale during this time. With this patch, the performance of "rm -rf" will be improved for current master. 1. without this patch on master [root@mds tests]# ./createmany -o /mnt/lustre/test1/f- 10000 total: 10000 creates in 11.98 seconds: 835.02 creates/second [root@mds tests]# ls /mnt/lustre/test1/ | wc 10000 10000 68890 [root@mds tests]# time rm -rf /mnt/lustre/test1 real 1m4.496s user 0m0.014s sys 0m3.832s 2. with this patch on master [root@mds tests]# ./createmany -o /mnt/lustre/test1/f- 10000 total: 10000 creates in 11.72 seconds: 853.50 creates/second [root@mds tests]# time rm -rf /mnt/lustre/test1 real 0m8.151s user 0m0.012s sys 0m2.242s Signed-off-by: wang di <di.wang@intel.com> Change-Id: I189f0039b5624455b70d1381b43d452cc8766543 Reviewed-on: http://review.whamcloud.com/10622 Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: John L. Hammond <john.hammond@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2675 obd: decruft md_enqueue() and md_intent_lock() Remove the lmm and lmmsize parameters from both functions, storing that data in md_op_data when needed. Remove the unused lookup_flags parameter from md_intent_lock(), and the unused reqp parameter from md_enqueue(). Add a union ldlm_policy_data * parameter to md_enqueue(). Remove the unused function lmv_enqueue_remote(). Signed-off-by: John L. Hammond <john.hammond@intel.com> Change-Id: Ia914318a7c7cbeb59fe86d4a169559cd86f3ad57 Reviewed-on: http://review.whamcloud.com/10205 Tested-by: Jenkins Reviewed-by: wangdi <di.wang@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Revert "LU-2675 obd: decruft md_enqueue() and md_intent_lock()" This reintroduced back LU-4805, so reverting. This reverts commit a2877f1f6c0fd0ec5ece836a216f42af3ef913da. Change-Id: Id6728d4fa6a322a45579f0ce744d4c7161ac9d69 Reviewed-on: http://review.whamcloud.com/10203 Tested-by: Jenkins Reviewed-by: Oleg Drokin <oleg.drokin@intel.com> Tested-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2675 obd: decruft md_enqueue() and md_intent_lock() Remove the lmm and lmmsize parameters from both functions, storing that data in md_op_data when needed. Remove the unused lookup_flags parameter from md_intent_lock(), and the unused reqp parameter from md_enqueue(). Add a union ldlm_policy_data * parameter to md_enqueue(). Remove the unused function lmv_enqueue_remote(). Signed-off-by: John L. Hammond <john.hammond@intel.com> Change-Id: I5c216b24d8ac824b9487c3296f18b7a8511285e3 Reviewed-on: http://review.whamcloud.com/9850 Tested-by: Jenkins Reviewed-by: Lai Siyao <lai.siyao@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3963 cleanup: C89 and build cleanups use C99 initialization for a types, remove some platform depended code and use POSIX compatible instead. don't build a lhsmtool is none utils allowed. Signed-off-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com> Change-Id: I6e2837953902520240e9ceb251c92329d328d715 Reviewed-on: http://review.whamcloud.com/7803 Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Bob Glossman <bob.glossman@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4679 liblustre: remove dead code and clean headers In liblustre, remove statically unused functions, varaibles and macros. Also ensure that the headers needed by the c and h files are included at top level. Prune unneeded includes. In libsysio, include sys/queue.h in fs.h. Signed-off-by: John L. Hammond <john.hammond@intel.com> Change-Id: I878b442b6d0ae17d8d5d33375ca7596618323c52 Reviewed-on: http://review.whamcloud.com/9429 Reviewed-by: Lai Siyao <lai.siyao@intel.com> Tested-by: Jenkins Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3531 llite: move dir cache to MDC layer Move directory entries cache from llite to MDC, so client side dir stripe will use independent hash function(in LMV), which does not need to be tightly coupled with the backend storage dir-entry hash function. With striped directory, it will be 2-tier hash, LMV calculate hash value according to the name and hash-type in layout, then each MDT will store these entry in disk by its own hash. Signed-off-by: wang di <di.wang@intel.com> Change-Id: I14bb6bd81aad6fd59dcc22cf4bcea9d341dca2a1 Reviewed-on: http://review.whamcloud.com/7043 Tested-by: Jenkins Reviewed-by: John L. Hammond <john.hammond@intel.com> Tested-by: Maloo <hpdd-maloo@intel.com> Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-1346 libcfs: replace cfs_ memory wrappers Replace memory relevant wrappers with kernel API. Affected primitives: CFS_PAGE_SIZE, CFS_PAGE_SHIFT, cfs_num_physpages, cfs_copy_from_user, cfs_copy_to_user, cfs_page_address, cfs_kmap/cfs_kunmap, cfs_get_page, cfs_page_count, cfs_page_index, cfs_page_pin, cfs_page_unpin, cfs_memory_pressure_get/set/clr, CFS_NUM_CACHEPAGES, CFS_ALLOC_XXX flags, cfs_alloc/free, cfs_alloc/free_large, cfs_alloc/free_page, CFS_DECL_MMSPACE, CFS_MMSPACE_OPEN, CFS_MMSPACE_CLOSE, CFS_SLAB_XXX flags, cfs_shrinker_t, cfs_set/remove_shrinker, CFS_DEFAULT_SEEKS, cfs_mem_cache_t, cfs_mem_cache_alloc/free/create/destroy, cfs_mem_is_in_cache manual changes: 1. cfs_alloc_flags_to_gfp() is removed 2. remove kmalloc/kfree etc. from linux-mem.c and linux-mem.h 3. remove page_address/kmap/kunmap etc. from linux-mem.h 4. remove page_cache_get/page_cache_release from echo_internal.h. They are defined already in user-mem.h 5. change kmem_cache_create/destroy prototype to kernel's and modify all callers to match them 6. define _SPL_KMEM_H and related macros to avoid using spl's sys/kmem.h that redefines slab allocator 7. change kmem_virt to is_vmalloc_addr as provided by kernel, so that we don't use any spl's sys/kmem.h functions 8. clean up include files a little bit in osd-zfs 9. various coding style cleanup NUMA allocators(cfs_cpt_xxx) are not changed in this patch. gnilnd is not converted, as requested by James Simmons. Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com> Signed-off-by: Peng Tao <tao.peng@emc.com> Change-Id: Iadfbb0d5a0e31c78dd6c811e5ffdb468fa7e6f44 Reviewed-on: http://review.whamcloud.com/2831 Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2771 dlmlock: compress out unused space * lustre/include/lustre_dlm.h: Remove all bit fields and the unused weighing callback procedure. respell LDLM_AST_DISCARD_DATA as LDLM_FL_AST_DISCARD_DATA to match other flags. * .gitignore: ignore emacs temporary files * autogen.sh: rebuild the lock bits, if autogen is available. * contrib/bit-masks/lustre_dlm_flags.def: define the ldlm_lock flags * contrib/bit-masks/lustre_dlm_flags.tpl: template for emitting text * contrib/bit-masks/Makefile: construct the .c and .h files The .c file is for constructing a crash extension and is not preserved. * contrib/bit-masks/.gitignore: ignore built products * lustre/contrib/wireshark/packet-lustre.c: use built files instead of local versions of the defines. In the rest of the modified sources, replace flag field references with bit mask references. * lustre/osc/osc_lock.c: removed osc_lock_weigh, too Signed-off-by: Bruce Korb <bruce_korb@xyratex.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com> Reviewed-by: Keith Mannthey <Keith.Mannthey@intel.com> Change-Id: I104522e4012deb0dd8b9d12c2d1f3fd3851708cb Reviewed-on: http://review.whamcloud.com/5312 Reviewed-by: Keith Mannthey <keith.mannthey@intel.com> Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: <bruce.korb@gmail.com>
LU-1346 libcfs: remove cfs_ file wrappers Replace file relevant wrappers with kernel API. Affected primitives: file, dentry, dirent, kstatfs, filp_size, filp_poff, filp_open, do_fsync, filp_close, filp_read, filp_write, filp_fsync, get_file, fget, fput, file_count, flock_t, flock_type, flock_set_type, flock_pid, flock_set_pid, flock_start, flock_set_start, flock_end, flock_set_end. completion, init_completion, fini_completion, wait_for_completion, complete Change some API implementations of darwin/winnt to make it be consistent with linux kernel API such as filp_open/filp_close etc. Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com> Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: John L. Hammond <john.hammond@intel.com> Change-Id: Ibe00df71c658aeb5dda854481f6ab5c181b3de7b Reviewed-on: http://review.whamcloud.com/2830 Tested-by: Hudson Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Peng Tao <bergwolf@gmail.com> Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-2446 build: Update Whamcloud copyright messages for Intel Update the copyright messages for master branch for Intel. This is the result of the updatecw.sh script. Signed-off-by: Keith Mannthey <keith.mannthey@intel.com> Change-Id: Id8dc454e7a5b6dffef64aa6efefeaaaea966d780 Reviewed-on: http://review.whamcloud.com/4812 Tested-by: Hudson Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Bob Glossman <bob.glossman@intel.com>
LU-1347 build: remove the vim/emacs modelines Delete all of the vim/emacs modelines. Add copyright information for a few files. Signed-off-by: Liang Zhen <liang@whamcloud.com> Change-Id: Ic22ce8d8cc7786ffdb19937eb9f93a2d49aa2581 Reviewed-on: http://review.whamcloud.com/2658 Tested-by: Hudson Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1146 build: batch update copyright messages Update copyright messages for all files modified by Whamcloud. Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Change-Id: I9654b780c6e4726d369e2a893f7e210d4078819b Reviewed-on: http://review.whamcloud.com/2242 Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-991 llite: cleanup md_readpage interface Use md_op_data to transfer the parameters of readpage on client metadata stack. Signed-off-by: Wang Di <di.wang@whamcloud.com> Change-Id: I103a555094e678711aa2f2e5d73ec848a452aed0 Reviewed-on: http://review.whamcloud.com/2001 Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Fan Yong <yong.fan@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-506 kernel: FC15 - fix GCC 'set-but-unused' warnings Eliminate a number of unused variables in the code that caused GCC-4.6.0 to warn about "set-but-unused variables". In some cases the variables are only used in conditionally-compiled code, so they are used in some way just to quiet the compiler. Signed-off-by: Yang Sheng <ys@whamcloud.com> Change-Id: I8b9b744c787fff4057722420866adea86d8240f5 Reviewed-on: http://review.whamcloud.com/1330 Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Niu Yawei <niu@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-5 readdir read multiple pages per rpc add support for readdir to read multiple pages per rpc: * because client has no idea how many directory pages it can read, it tries to read maximum pages each time, but will only store pages read from mds into page cache. * add a flag LDF_COLLIDE to mark a dir page hash collides with the next page, and client will remove this page from page cache after processing. * upon readpage bulk io failure, client won't be evicted, and client will resend the bulk request. * support large page size on client: MDS_READPAGE RPC will fill reply with page size LU_PAGE_SIZE(4k), and if client page is bigger than LU_PAGE_SIZE, several pages will be integrated into one dir page with CFS_PAGE_SIZE. Signed-off-by: Lai Siyao <laisiyao@whamcloud.com> Change-Id: Id6bc36fbcec79993d49bbe9a535851e5e3ebd876 Reviewed-on: http://review.whamcloud.com/604 Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
LU-376 Positive LL_DIR_END_OFF to indicate the tail of dir hash/offset 1) Keep 'MDS_DIR_END_OFF' unchanged (0xfffffffffffffffeULL) to simplify dir hash/offset related interoperability issues. 2) Introduce positive "LL_DIR_END_OFF" (0x7fffffffffffffffULL) on client to indicate the tail of dir hash/offset for up layer callers, like llseek(), readdir(), and so on. 3) Support new 2.x client to talk with old 2.0 server with 32bit hash. Change-Id: I74c148482f91ed91dfffd608f98971566f1257ad Signed-off-by: nasf <yong.fan@whamcloud.com> Reviewed-on: http://review.whamcloud.com/886 Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: Hudson Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-163 MDS returns 32/64-bit dir name hash according to client type 1) liblustre client, 1.8.5 or older client do not support 64-bit dir name hash. 2) register Lustre version 'get_name()' for NFS processing. 3) 'll_dir_seek()' processes name hash-based dir seek operation. 4) re-define "DIR_END_OFF" as "0x7fffffffffffffffULL" to always return positive dir name hash for successful seek. 5) small cleanup for 'll_dops_init()' to drop unnecessary statahead process. Signed-off-by: nasf <yong.fan@whamcloud.com> Change-Id: I0e3177c98c6e8814c39551ab54edfa3f861cbc8c Reviewed-on: http://review.whamcloud.com/432 Tested-by: Hudson Reviewed-by: Oleg Drokin <green@whamcloud.com>
b=20563 i=zam i=andreas i=rahul fix values of LUSTRE_SEQ_MAX_WIDTH and LUSTRE_SEQ_META_WIDTH. this patch fixes fid to inode mapping so that it will be independent of LUSTRE_SEQ_MAX_WIDTH. there is fix for seq flush also.