LU-17522 build: Distribute clang build infrastructure Macro files: lustre-toolchain.m4 lustre-compiler-plugins.m4 and directory: cc-plugins Should be included in distributed files, unconditionally. Test-Parameters: trivial Fixes: d684885098 ("LU-16961 clang: plugins and build system integration") Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: I6ddedd82c6180ffd1c4134fda6af6df6bd23dd34 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53991 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com> Reviewed-by: Timothy Day <timday@amazon.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-16967 build: Separate lnet LND deb packaging Enable separate packaging of lnet lnd kernel modules into separate packages with build profile multiple-lnds: lustre-lnet-module-socklnd for socklnd.ko lustre-lnet-module-gnilnd for kgnilnd.ko, profile gnilnd lustre-lnet-module-kfilnd for kkfilnd.ko, profile kfilnd lustre-lnet-module-o2iblnd for o2iblnd.ko, profile ext_o2ib lustre-lnet-module-in-kernel-o2iblnd for ko2iblnd.ko, profile int_o2ib Test-Parameters: trivial HPE-bug-id: LUS-11711 Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: I3a5ca03fa410238f66083289db0899c8b4bfab5c Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52397 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com> Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com> Reviewed-by: Frank Sehr <fsehr@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-17347 debs: also move .ddeb files into debs/ When building debian packages, the resulting packages are moved into a 'debs/' subdir. Don't miss the debug symbol packages 'dbgsym', which are suffixed .ddeb. Also add .buildinfo file. Test-Parameters: trivial Change-Id: I52d0bddfaafc67c4a2a2dbc786d7f320c0b979f8 Signed-off-by: Aurelien Degremont <adegremont@nvidia.com> Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53378 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Minh Diep <mdiep@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-16967 build: Add in-kernel-ko2iblnd driver Add in-kernel-ko2iblnd.ko for users of in-kernel OFED and only build ko2iblnd.ko if an external OFED is available This allows for building and packaging both an external (MOFED or OPA) o2ib driver and an in-kernel o2ib driver. Packaging rules will be written so that only enable one of the o2iblnd drivers can be installed. In the case of the in-kernel-ko2iblnd.ko driver a symlink named ko2iblnd.ko will be created to point to the in-kernel based o2ib driver which allows for a reasonable migration path for the majority of users. It is useful for dist build and test to be able to build both in-kernel IB and external OFED in the same build. This also means there would be some install/configure adjustments that ought to have some discussion. Test-Parameters: trivial HPE-bug-id: LUS-11711 Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: I8105fad0b20c36705d7e14e3ae976bf3d81e9f1b Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51915 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Frank Sehr <fsehr@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-17216 ofd: make enable_health_write tunable enable_health_write should be tunable rather than a compilation option. This allows us to test it more easily and gives admins the option to try it out without having to recompile their Lustre servers. It will still be disabled by default. Add sanity/70 test to run a simple check to ensure enable_health_write and health_check don't explode. It's not a thorough check. But it at least checks that the interfaces appear to work. Signed-off-by: Timothy Day <timday@amazon.com> Change-Id: I7b1832f8acf578b891386e28c5af760070a6862c Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52782 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-17065 build: Remove snmp support Last patched in 2012 and not well tended to before that, snmp support can probably be removed. Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com> Signed-off-by: Timothy Day <timday@amazon.com> Change-Id: I1831aed42e560531e57a6ff8aa978f3e5286fd44 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52194 Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com>
LU-17128 build: fix lnet.service missing issue on Ubuntu 22.04 The lnet.service file was in the lustre-client-utils package built on Ubuntu 20.04, but it was missing from the package built on Ubuntu 22.04. This is caused by the dpkg-buildpackage change introduced by dpkg version 1.21.1ubuntu2.1 installed by default on Ubuntu 22.04. To fix this issue, we need to specify build profiles explicitly to dpkg-buildpackage via -P|--build-profiles option instead of just setting the environment variable DEB_BUILD_PROFILES. Test-Parameters: trivial clientdistro=ubuntu2204 Signed-off-by: Jian Yu <yujian@whamcloud.com> Change-Id: I9975ef357f0aba722c56d27eaa9b2cfbccc9c524 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52404 Reviewed-by: Colin Faber <cfaber@ddn.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com>
LU-17095 build: avoid modules.order nonexistence failure The modules.order is a temporary output file generated by kbuild while running "make" command. Sometimes, there is a race condition that causes the file not created and makes make command fail as follows: cat: ...//modules.order: No such file or directory This patch creates an empty modules.order file to avoid the error. Test-Parameters: trivial Change-Id: If779a727731f18e9409c35c0cd0deddd79559d3a Signed-off-by: Jian Yu <yujian@whamcloud.com> Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52323 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-17096 debian: add obd_test.ko, llog_test.ko to lustre-tests The obd_test.ko module was missing from the lustre-tests Debian package. Hence, it wasn't being installed on the Ubuntu clients during testing. This caused sanity/55a and sanity/55b to consistently fail. Add llog_test.ko to lustre-tests also. It's not unheard of to use Ubuntu for Lustre server. So the package may as well include llog_test.ko. Also, update debian/.gitignore. Test-Parameters: trivial testlist=sanity env=ONLY=55,ONLY_REPEAT=50 clientdistro=ubuntu2204 Test-Parameters: trivial testlist=sanity env=ONLY=55,ONLY_REPEAT=50 Signed-off-by: Timothy Day <timday@amazon.com> Change-Id: I050de4563478996828886ca623fa96b58f9fef5e Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52398 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Thomas Stibor <thomas@stibor.net> Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-17058 build: add help and checkpatch as make targets Add `make help` to print out available make targets. The output is styled after the Linux kernel `make help`. Add `make checkpatch` to run checkpatch.pl script against most recent commit. Update README to mention `make help`. Test-Parameters: trivial Signed-off-by: Timothy Day <timday@amazon.com> Change-Id: I65ce84040502994ae7caa0c8b72d808442f6b79e Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/52142 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com> Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-16961 clang: plugins and build system integration Clang has a plugin system. Compiler extensions can be created by making a shared library and loading it via the "-fplugin" options. This makes it simple to implement custom warnings and static analyzers. This patch adds a plugin to detect functions that should have been made static. This plugin has been run over the majority of the Lustre tree and patches have been submitted for all warnings. The plugin did not return any false positives in my testing. It also add the "--enable-compiler-plugins" configure option, which automatically builds and sets up the in-tree C compiler plugins. The option force-enables the plugin regardless of which compiler is in use. This behavior could be changed if there is ever a need to support GCC specific plugins. Also, add the configure checks needed to support building C++ in the Lustre tree. Clang and GCC plugins (and the compilers themselves) are written in C++. The license for the plugin mirrors that of the LLVM project itself. This leaves the door open for contributing this plugin upstream in the future. This isn't being upstreamed now because it lacks any significant user community. Hence, the plugin does not appear to meet the requirements for upstreaming based on https://clang.llvm.org/get_involved.html. Test-Parameters: trivial Signed-off-by: Timothy Day <timday@amazon.com> Change-Id: I747ed91b53e765cc58e91a3eb9ec6c12b9908a96 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51659 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-16517 build: pass extra configure options to "make debs" While running "make debs", the configure command in debian/rules ignores some user defined configure options. This patch fixes the issue by adding the detection of the extra options into debian/rules. Test-Parameters: trivial clientdistro=ubuntu2204 Change-Id: Ia9db4e05abf33834cb3c853f4f0829dadc8d7400 Signed-off-by: Jian Yu <yujian@whamcloud.com> Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50464 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Tested-by: Shuichi Ihara <sihara@ddn.com> Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com> Reviewed-by: Shuichi Ihara <sihara@ddn.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-16666 doc: remove unmaintained doxyfiles These Doxygen related files had old urls, some using http rather than https. Some code scanning tools will flag this. Rather than update these, remove all of the old doxyfiles. They are very out of date, Doxygen throws many errors when you try to use them, and they do not seem to generate usable documentation. Test-Parameters: trivial Signed-off-by: Timothy Day <timday@amazon.com> Change-Id: I5a24d4754582ecee558c4e87385b8835d2675adc Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50415 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Aurelien Degremont <adegremont@nvidia.com> Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-6142 build: add SPD/GPL license to build files Update the file header to have the SPDX license and use the standard format. Convert spaces to tabs. Test-Parameters: trivial Signed-off-by: Timothy Day <timday@amazon.com> Change-Id: Id02218aa5b435bc0de96a39d3daa53a83a51c857 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50347 Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Oleg Drokin <green@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com>
LU-16115 build: Linux 5.17 external module support Linux commit v5.16-rc3-26-g129ab0d2d9f3 Added quotes around "$(CONFIG_CC_VERSION_TEXT)", however .config stores CONFIG_CC_VERSION_TEXT with quotes thus breaking the GNU make Makefile for external modules. We need to supply a non-quoted value and override the value in .config before it is used. Test-Parameters: trivial HPE-bug-id: LUS-11190 Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: I997b6987ef37a8c5b9d8f0984e81d9402a2ea705 Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/48360 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Jian Yu <yujian@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: jsimmons <jsimmons@infradead.org> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-15968 build: update libssl3 In Ubuntu 22.04 libssl1.1 has been superceded by libssl3 Test-Parameters: trivial clientdistro=ubuntu2004 Change-Id: Ic2504c3a40d5c756c2da7b81edb1501fb6b44712 Signed-off-by: Minh Diep <mdiep@whamcloud.com> Reviewed-on: https://review.whamcloud.com/47751 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Sebastien Buisson <sbuisson@ddn.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-15596 build: set TARBALL with m-a build helper for debs debs built using module-assistant tool by default scan for packages and may find a random lustre tarball Specify the correct tarball with the TARBALL environment variable to avoid default search picking an incorrect tarball source. HPE-bug-id: LUS-10783 Test-Parameters: trivial Fixes: 1d97ac16d8 ("LU-14948 build: Warn about /usr/src/lustre.tar.bz2") Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: Iaa5a31aaa81e11ee97ae2ea27811c7a4399a0efa Reviewed-on: https://review.whamcloud.com/46639 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Alexey Lyashkov <alexey.lyashkov@hpe.com> Reviewed-by: Petros Koutoupis <petros.koutoupis@hpe.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-14948 build: Warn about /usr/src/lustre.tar.bz2 When /usr/src/lustre.tar.bz2 exists, make debs (and dkms-debs) will fail with an error like: Extracting the package tarball, /usr/src/lustre.tar.bz2, ... ../../generic.sh: line 73: debian/rules: Permission denied BUILD FAILED! Add the current git hash to the lustre tarball, as well as attempt to remove the conflict from /usr/src. Failing that, give a warning to ask the user to remove the conflicting file. HPE-bug-id: LUS-10308 Test-Parameters: trivial Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: I4aaa803cb81c2ed8ffc0182bb49ea0bff5064df4 Reviewed-on: https://review.whamcloud.com/44677 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Andreas Dilger <adilger@whamcloud.com> Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-14937 build: re-use config cache in 'make rpms/debs' Idea is to pass along the value of the -C or --cache-file options from the initial ./configure to the one launched as part of the rpm or deb build. But all the environment variables related cache info must be removed otherwise the config cache file cannot be reused. Test-Parameters: trivial Signed-off-by: Sebastien Buisson <sbuisson@ddn.com> Change-Id: Iab4ae2815961ba10132d9cb44f82ca58d313e908 Reviewed-on: https://review.whamcloud.com/44659 Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com> Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-14785 build: changelog updates should not dirty version When building lustre debs the final version should not include 'dirty' due to an update of the changelog HPE-bug-id: LUS-10152 Test-Parameters: trivial Signed-off-by: Shaun Tancheff <shaun.tancheff@hpe.com> Change-Id: I59f4da4b3006302e3598cfa56a0364b052f885ef Reviewed-on: https://review.whamcloud.com/44069 Reviewed-by: James Simmons <jsimmons@infradead.org> Reviewed-by: Li Dongyang <dongyangli@ddn.com> Reviewed-by: Thomas Stibor <thomas@stibor.net> Tested-by: jenkins <devops@whamcloud.com> Tested-by: Maloo <maloo@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com>