LU-19014 memcg: fix client hang in balance_dirty_pages()

LU-19014 memcg: fix client hang in balance_dirty_pages()

Two nodes (at least) append write a shared file in Lustre with
memcg enabled.
The client randomly hung in balance_dirty_pages() with the
following call trace:
[<0>] balance_dirty_pages+0x2ee/0xd10
[<0>] balance_dirty_pages_ratelimited_flags+0x27a/0x380
[<0>] generic_perform_write+0x150/0x210
[<0>] vvp_io_write_start+0x516/0xc00 [lustre]
[<0>] cl_io_start+0x5a/0x110 [obdclass]
[<0>] cl_io_loop+0x97/0x1f0 [obdclass]
[<0>] ll_file_io_generic+0x4d2/0xe50 [lustre]
[<0>] do_file_write_iter+0x3e9/0x5d0 [lustre]
[<0>] vfs_write+0x2cb/0x410
[<0>] ksys_write+0x5f/0xe0
[<0>] do_syscall_64+0x5c/0xf0

After analyze the core dump of the hung system, we found that the
bdi_writeback data structure (wb) corresponded to the memcg has
pending dirty pages (in state WB_registered | WB_has_diry_io), but
can not write-out the dirty pages and loop in balance_dirty_pages
function.

This is a bug in Lustre memcg code. In OSC/MDC layer, it will stop
to flush dirty pages once found that there are no any unstable
pages.
However, there may be some dirty pages queued in the cache. In
this case, the client should still write back the dirty pages.
Thus the wb stat accounting will be updated and the write process
can continue instead of looping endless.

Moreover, there are some problem in the current Lustre CLIO engine.
When the system or a certain memcg is under memory pressure, the
client just queues the dirty page in page cache or in the current
active extent (OES_ACTIVE osc_extent) when vvp_io_write_commit()/
cl_io_commit_async() is called in ->write_end(). The queued pages
can not be written back even the kernel is trying to flush dirty
pages in writeback via ->ll_writepages().
The client is looping in the following call sequences:

loop:
->write_begin()
->write_end()
->balance_dirty_pages()
  ->Launch file writeback in background but cannot flush any
    dirty pages.
  ->The current process is paused a certain time (i.e. 200ms) as
    the corresponding @wb is dirty exceeded.
-> GOTO loop:

The write progress is very slow: write a page and sleep/pause for
a period of time alternately.

We fix this hang in ->ll_write_end(). When detect the corresponding
@wb is dirty exceeded, the client will submit the dirty pages into
OSC writeback cache. The state of current extent will change from
OES_ACTIVE to OES_CACHE and this kind of extents can be written
back. Moreover, we mark the current extent as urgent, thus it can
be flushed much more quickly.

Fixes: 8aa231a99 ("LU-16713 llite: writeback/commit pages under memory pressure")
Signed-off-by: Yingjin Qian <qian@ddn.com>
Change-Id: Iecee60484f1b65fad6f4c9eac7bd4d2c53f38b8d
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/59223
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Zhenyu Xu <bobijam@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>