LU-3462 ldiskfs: Subsume ldiskfs's build system into lustre Remove ldiskfs's independant build system (configure, spec file, etc.) and merge its build into lustre's build system. Move the ldiskfs/ldiskfs directory up one level. The extra directory level is no longer needed without the independant ldiskfs build system. Add a make dependency on ldiskfs's "sources" target in ldiskfs/autoMakefile.am rather than explicitly launching a separate make instance from the top level autoMakefile.am. Remove unused "sources" targets in libcfs, lnet, and lustre/obdclass. Improve the handling of the --disable-ldiskfs option, and make it interact with other configure options more cleanly. Change-Id: Id0787f27269e938032e480c7c627327c0adf2672 Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov> Reviewed-on: http://review.whamcloud.com/6850 Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Minh Diep <minh.diep@intel.com> Reviewed-by: Brian J. Murrell <brian.murrell@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-1617 build: ignore automatically generated files Do not list automatically generated files in "git status" output, so that it does not hide checkpatch.pl output in the commit message. Signed-off-by: Andreas Dilger <andreas.dilger@intel.com> Change-Id: Iab5c5be25de7dc54225fb2aa6540a4d75e500c1e Reviewed-on: http://review.whamcloud.com/6229 Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com> Reviewed-by: Keith Mannthey <keith.mannthey@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2989 build: some make targets are broken Add make targets "distclean" and "maintainer-clean" in ldiskfs/ldiskfs/Makefile. The global "make distclean" now executes whithout an error. Also lustre/target/.gitignore file is created to hide message from GIT. Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com> Change-Id: Ic8841d11960d5aacd8293ecfcaf8c628dd743c45 Reviewed-on: http://review.whamcloud.com/5766 Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Alexey Shvetsov <alexxy@gentoo.org> Reviewed-by: Faccini Bruno <bruno.faccini@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-992 build: remove obsolete rhel5 server configurations For 2.4 RHEL5 server side support has been removed but some technical debt remained before. This patch removes the no longer needed rhel5 kernel configurations and the CONFIG_LDISKFSDEV_* options that were left over from the rhel5 ext3 days. Signed-off-by: James Simmons <uja.ornl@gmail.com> Change-Id: I345b1b5d182602b82be018b1e18a7bdbe939f578 Reviewed-on: http://review.whamcloud.com/5930 Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-1812 fsfilt: ext_pblock renamed to ext4_ext_pblock For kernels 2.6.35 and above ext_pblock was renamed to ext4_ext_pblock. With no more RHEL5 kernel support for ldiskfs we also clean up more macros. see kernel commit bf89d16f6ef5389f1b9d8046e838ec87b9b3f8b9 for pblock change. Signed-off-by: James Simmons <uja.ornl@gmail.com> Signed-off-by: chas williams - CONTRACTOR <chas@cmf.nrl.navy.mil> Change-Id: I3ce7f27f6fd6826380e6f2f54b2d50d09d36f78a Reviewed-on: http://review.whamcloud.com/5001 Tested-by: Hudson Reviewed-by: Bob Glossman <bob.glossman@intel.com> Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-1812 fsfilt: ext4_free_blocks() has changed slightly ext4_free_blocks() now takes a buffer_head and explicit flags instead of just metadata. Test the presence of the buffer_head argument to determine which ext4_free_blocks() is available. see kernel commit e6362609b6c71c5b802026be9cf263bbdd67a50e Signed-off-by: James Simmons <uja.ornl@gmail.com> Signed-off-by: chas williams - CONTRACTOR <chas@cmf.nrl.navy.mil> Change-Id: I925df73a054613469866ec025ae412ead0ce9e56 Reviewed-on: http://review.whamcloud.com/4991 Reviewed-by: Bob Glossman <bob.glossman@intel.com> Tested-by: Hudson Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com> Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-1199 build: Cleanup ldiskfs kernel config defines Make it more clear what the purpose of the "CONFIG_LDISKFS_*" defines are with a comment, and separate them from options that do not necessarily originate in the kernel source. All of these options are needed for ldiskfs and some are needed by osd-ldiskfs in the lustre file system. To handle these shared options a extra autoconf header file is create containing only the values of interest to osd-ldiskfs. Signed-off-by: James Simmons <uja.ornl@gmail.com> Change-Id: I8e71b6cf471c4317b20fdce14d66f8b2883a226e Reviewed-on: http://review.whamcloud.com/5675 Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Bob Glossman <bob.glossman@intel.com> Reviewed-by: Minh Diep <minh.diep@intel.com> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-1199 build: Assume that ldiskfs PDO is now standard PDO (Parallel Directory Operations) is now a standard ldiskfs feature. The old patch series files that lacked PDO support have been removed from the tree, so now we can drop all of the places that conditionally use PDO, and just assume that PDO is standard. Also remove support for rhel5 series file in ldiskfs, since that no longer exists in the tree. Change-Id: I76b5b31a4614c1ade3b4b5890a111cdabc4afa63 Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov> Reviewed-on: http://review.whamcloud.com/5004 Tested-by: Hudson Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Liang Zhen <liang.zhen@intel.com> Reviewed-by: James Simmons <uja.ornl@gmail.com>
LU-1199 build: Refactor ldiskfs's build system Lustre has a complicated build system that combines automake files and the kernel's Makefile.in build system in a rather serpentine system that has long chains of makefile includes, making it very difficult for the reader to ever know exactly which targets and reles will either be required or available under any specific set of conditions. The good news is that none of that is required by ldiskfs because ldiskfs is only built for kernel-space Linux. We only need one fairly straight forward build path. To simplify the build system, we: - Remove Rules.in - Remove all autoMakefile.am files - Remove confusing build/Makefile - Combine the three .m4 files into one, since that was just a hold-over from Lustre's build system. - Cleanup and improve some of the m4 macros - Remove the multi-level build/ subdirectory tree and replace with single-level config subdirectory. - Fix the ldiskfs debuginfo package (actually contains the patched source of ldiskfs!) - Simplify autogen.sh - Update .gitignore files Change-Id: I9e6112c150c1e0f9076d5e689b69534eab7a964b Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov> Reviewed-on: http://review.whamcloud.com/4728 Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Brian J. Murrell <brian.murrell@intel.com> Reviewed-by: Minh Diep <minh.diep@intel.com> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-1617 build: skip generated files in .gitignore Ignore automatically generated files via .gitignore so that a large number of spurious files are not listed at "git commit" time. Also consolidate many duplicate ignored files in the top-level .gitignore file, since we don't need to list the exceptions in every directory like the original .cvsignore files that they were generated from. Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Change-Id: I5f9c60dc498f98abcd8c91042fc72408a915500c Reviewed-on: http://review.whamcloud.com/3364 Reviewed-by: Yu Jian <yujian@whamcloud.com> Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-723 ldiskfs: Drop support for ext3 based ldiskfs Building ldiskfs from ext3 is no longer supported, thus this change attempts to clean things up and only support building ldiskfs from ext4 sources. The main changes made, involved replacing the @BACKFS@ autoconf variable with 'ext4', and removing the HAVE_EXT4_LDISKFS macro. It is now safe to assume that ldiskfs is build based entirely on ext4. Signed-off-by: Prakash Surya <surya1@llnl.gov> Change-Id: Ia0eff6e4c1b755806ccfbf554e4a36201f9f1a64 Reviewed-on: http://review.whamcloud.com/1643 Tested-by: Hudson Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-50 ldiskfs: pdirops patch for ldiskfs Single directory performance is a critical for HPC workloads. In a typical use case an application creates a separate output file for each node and task in a job. As nodes and tasks increase, hundreds of thousands of files may be created in a single directory within a short window of time. Today, both filename lookup and file system modifying operations (such as create and unlink) are protected with a single lock for an entire ldiskfs directory. PDO project will remove this bottleneck by introducing a parallel locking mechanism for entire ldiskfs directories. This work will enable multiple application threads to simultaneously lookup, create and unlink in parallel. This patch contains: - pdirops support for ldiskfs - N-level htree directory - integrate with osd-ldiskfs Change-Id: I269c0e3112e68f3acd79e860dab052a68c7d7aaa Signed-off-by: Liang Zhen <liang@whamcloud.com> Reviewed-on: http://review.whamcloud.com/375 Tested-by: Hudson Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-723 Enhance lustre/ldiskfs build system Enhance the lustre/ldiskfs build system so it is more robust, flexible, and consistent with lustre/zfs build system. This change is being made in the interest of standardizing the infra-structure around backend filesystems. This change does not effect the current behavior of the --with-ldiskfs, --enable-ext4, or --with-ldiskfsprogs configure options. However, it does remove the obsolete --with-ldiskfs-inkernel configure option which was only used by LLNL. It also adds the --with-ldiskfs-obj configure option which improves flexibility. And the --enable-ldiskfs-build configure option to support building against the lustre-ldiskfs-devel package. The behavior of these options is consistent with their zfs counterparts, see commit 8c7266c for further details. --enable-ext4 enable ldiskfs build using ext4 --enable-ldiskfs-build enable ldiskfs configure/make --with-ldiskfs=path set path to ldiskfs source --with-ldiskfs-obj=path set path to ldiskfs objects --with-ldiskfsprogs use alternate names for ldiskfs-enabled e2fsprogs Sample ./configure results when building lustre and ldiskfs using the kernel-devel and kernel-debuginfo-common packages. checking whether to enable ldiskfs... yes checking ldiskfs source directory... /home/behlendo/src/git/lustre/ldiskfs checking ldiskfs object directory... /home/behlendo/src/git/lustre/ldiskfs checking ldiskfs module symbols... Module.symvers checking ldiskfs source release... 3.3.0 checking whether to use ext3 or ext4 source... ext4 checking ext4 source directory... /usr/src/debug/.../fs/ext4 checking whether to build ldiskfs... yes checking for /home/behlendo/src/git/lustre/ldiskfs/configure... yes checking for /usr/src/debug/.../fs/ext4/dir.c... yes checking for /usr/src/debug/.../fs/ext4/file.c... yes checking for /usr/src/debug/.../fs/ext4/inode.c... yes checking for /usr/src/debug/.../fs/ext4/super.c... yes checking if ext4_ext_walk_space() takes i_data_sem... yes checking if LDISKFS_SINGLEDATA_TRANS_BLOCKS takes sb as argument... yes checking if ldiskfs_discard_preallocations defined... yes checking if ldiskfs_ext_insert_extent needs 5 arguments... yes In the context of this change additional cleanup has been done and all of the ldiskfs specific code relocated to lustre-build-ldiskfs.m4. Note that this change moves us closer to supporting patchless Lustre servers with ldiskfs. Once the remaining kernel patches for Lustre are dropped you will be able to build Lustre using the distribution provided kernel-devel and kernel-debuginfo-common packages. This change also incorperates ORI-340, commit f604951, which ensures that the Module.symvers file will cleanly include the symbols for all enabled Lustre backends. While the only backend supported by master right now is ldiskfs this brings the master and orion branchs in to sync in this regard. Change-Id: I6f13f266944ec6967f4d7705a30b83ab8e577b15 Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Prakash Surya <surya1@llnl.gov> Reviewed-on: http://review.whamcloud.com/1566 Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-264 Integrate upstream ext4 MMP chnages Integrate the changes made to the MMP patch when it was submitted upstream for the 2.6.40 kernel. Mostly cosmetic changes were done (improved comments, etc), along with moving the MMP functionality into a separate mmp.c file in order to avoid increasing the size of super.c too much. Signed-off-by: Niu Yawei <niu@whamcloud.com> Change-Id: I6909f3e44b077d65d653994ed1ac3ee11fcacb7f Reviewed-on: http://review.whamcloud.com/865 Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Tested-by: Hudson Tested-by: Maloo <whamcloud.maloo@gmail.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-73 RHEL6 support. Change-Id: If694f1e25cb19438bcc30cfa8fc42c01156c9f8a Signed-off-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com> Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com> Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Reviewed-on: http://review.whamcloud.com/394 Reviewed-by: Johann Lombardi <johann@whamcloud.com> Tested-by: Hudson Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-73 RHEL6 support. Include client, ldiskfs, kernel patches. Change-Id: Ice16b8bf40c2e37df9af9f399316917097e8ee8f Signed-off-by: Bobi Jam <bobijam@whamcloud.com> Reviewed-on: http://review.whamcloud.com/307 Tested-by: Hudson Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com> Reviewed-by: Brian J. Murrell <brian@whamcloud.com>
Introduce .gitignore files. The top level .gitignore file is new and is an attempt at pulling in some of the common items that you might get for free (by default)with CVS. The other subdir/.gitignore files are translated versions of their corresponding .cvsignore file. Because CVS does not descend into a subdir when applying a ingore rule we have to prepend a "/" to the pathname to get git to behave the same way.
b=20298 (Merge head ldiskfs and b1_8 ldiskfs) i=adilger i=girish This is related to unifying the ldiskfs branch between b1_8 and HEAD. For that adding the patches to series files which is required for HEAD. Also some other changes required in makefile and config file.