Whamcloud - gitweb
LU-16807 ksocklnd: ksocklnd_ni_get_eth_intf_speed() must use only rtnl lock 28/51028/8
authorJames Simmons <jsimmons@infradead.org>
Fri, 19 May 2023 12:41:11 +0000 (08:41 -0400)
committerOleg Drokin <green@whamcloud.com>
Wed, 31 May 2023 19:15:42 +0000 (19:15 +0000)
A kernel with debug enable is reporting the following:

include/linux/inetdevice.h:221 suspicious rcu_dereference_check() usage!
other info that might help us debug this:
rcu_scheduler_active = 2, debug_locks = 1
5 locks held by lctl/42289:
 #0: ffffffffa04263f8 ((libcfs_ioctl_list).rwsem){++++}-{3:3}, at: __blocking_notifier_call_chain+0x44/0xa0
 #1: ffffffffa04fa7b0 (lnet_config_mutex){+.+.}-{3:3}, at: lnet_configure+0x1d/0xc0 [lnet]
 #2: ffffffffa04f06e8 (the_lnet.ln_api_mutex){+.+.}-{3:3}, at: LNetNIInit+0x4c/0x960 [lnet]
 #3: ffffffffa04f0788 (&the_lnet.ln_lnd_mutex){+.+.}-{3:3}, at: lnet_startup_lndnet+0x11e/0xa90 [lnet]
 #4: ffffffff834a0cd0 (rtnl_mutex){+.+.}-{3:3}, at: rtnl_lock+0x1b/0x30

stack backtrace:
CPU: 2 PID: 42289 Comm: lctl Kdump: loaded Tainted: G        W  O     --------- -  - 4.18.0rh8.5-debug #2
Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
  Call Trace:
  ? dump_stack+0x119/0x18e
  ? lockdep_rcu_suspicious+0x141/0x14f
  ? ksocklnd_ni_get_eth_intf_speed.isra.1+0x308/0x360 [ksocklnd]
  ? mark_held_locks+0x6a/0xc0
  ? ktime_get_with_offset+0x233/0x2b0
  ? trace_hardirqs_on+0x4e/0x220
  ? ksocknal_tunables_setup+0xed/0x200 [ksocklnd]
  ? ksocknal_startup+0x4ff/0x1180 [ksocklnd]

The function __ethtool_get_link_ksettings() has a hard requirement
to have the in_dev device protected by the rtnl mutex. At the
same time we are aquiring in_dev using the rcu lock. The in_dev
needs to be stabilized by the same lock. So use rtnl functions
instead.

Test-Parameters: trivial testlist=sanity-lnet
Change-Id: Iec795d62eb33002950cc962f29f9b93905b3bb3f
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51028
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Neil Brown <neilb@suse.de>
lnet/klnds/socklnd/socklnd_modparams.c

index 163eaf6..68e2bea 100644 (file)
@@ -211,11 +211,11 @@ static int ksocklnd_ni_get_eth_intf_speed(struct lnet_ni *ni)
                if (!(flags & IFF_UP))
                        continue;
 
-               in_dev = __in_dev_get_rcu(dev);
+               in_dev = __in_dev_get_rtnl(dev);
                if (!in_dev)
                        continue;
 
-               in_dev_for_each_ifa_rcu(ifa, in_dev) {
+               in_dev_for_each_ifa_rtnl(ifa, in_dev) {
                        if (strcmp(ifa->ifa_label, ni->ni_interface) == 0)
                                intf_idx = dev->ifindex;
                }