Whamcloud - gitweb
LU-14391 utils: handle very large YAML data sets. 32/54132/3
authorJames Simmons <jsimmons@infradead.org>
Thu, 22 Feb 2024 00:57:21 +0000 (19:57 -0500)
committerOleg Drokin <green@whamcloud.com>
Mon, 4 Mar 2024 20:05:30 +0000 (20:05 +0000)
commit985fc4fa2cafc12128f45fe1e6e643a746c3fc72
treeb89b80111a6e012594897b8bcf31f2f01471b2aa
parent1b3f3430fd8173eae72ee7f5c473e19ef2d2f3b0
LU-14391 utils: handle very large YAML data sets.

Some functionality for Lustre and even LNet can return huge
amounts of Netlink data that can overwhelm the internal libyaml
buffers. To resolve this we can create a resizable internal
buffer to collect all the Netlink data that is formated into
YAML. After the message has been completed we can feed this
data in chunk sizes the smaller internal libyaml library can
handle. The libyaml library internal buffer is a rolling buffer
so it will updated when we exceed its internal size. This will
allows collecting every single type of Lustre stat in one go
and for sites that have very large LNet router setups.

Test-Parameters: trivial
Change-Id: I20fdbb19b0f3de3ab52e8ad568c6926f61f627b9
Signed-off-by: James Simmons <jsimmons@infradead.org>
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/54132
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Chris Horn <chris.horn@hpe.com>
Reviewed-by: Timothy Day <timday@amazon.com>
Reviewed-by: Frank Sehr <fsehr@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
lnet/utils/lnetconfig/liblnetconfig_netlink.c