Whamcloud - gitweb
LU-10472 osd-ldiskfs: T10PI between RPC and BIO 66/32266/15
authorLi Xi <lixi@ddn.com>
Sun, 8 Apr 2018 12:21:13 +0000 (08:21 -0400)
committerOleg Drokin <green@whamcloud.com>
Tue, 6 Nov 2018 07:13:24 +0000 (07:13 +0000)
commitccf3674c9ca3ed8918c49163007708d1ae5db6f5
treebe2c28ee66ca24fa9323d46f8bd27f2a3aa9aa3b
parentc50bd08b35757d20e606c10bb5b4474011dd07f6
LU-10472 osd-ldiskfs: T10PI between RPC and BIO

When OST recieves bulk write RPC, the T10PI guard tag will be
generated during the process of calculating RPC checksum with
T10PI type. Guard tags of each sector will be copied to the
BIO integrity payload to avoid recalculating of guard tags.

When OST reads data from disk, the T10PI guard tags will be
copied from BIO integrity payload. These guard tags will be
reused for calculation the RPC checksum with T10PI type, thus
no recalcuating of guard tags is needed either.

However, if the data that the client is reading is cached
in memory, the guard tags need to be calculated based on the
cached data, since there is no place to plug the guard tags
to the page cache on OSS.

Some modification to Linux kernel is needed:

1) We can pass “struct bio *” and  to the integrity
generate/verify methods, and struct blk_integrity_exchg
has bi_idx which is the current bio_vec index.

2) bio_integrity_prep accepts optional pointers to integrity
generation/verification methods. The optional methods take
priority over the ones registered by the device.

These two modification enable Lustre (and other file systems) to
integrate with BIO for integrity verification/generation. Any private
data need during data integrity generation/verification process can
be attached to bio->bi_private. Instead of calculating guard tags,
Lustre generation method will copy the guard tags from existing
buffer. And instead of (or besides of) data integrity verification,
Lustre verification method will copy the guard tags to internal
buffer for further usage.

Besides of these changes, two Linux kernel patches are applied:

1) The first problem is that bio_integrity_verify() doesn't verify
the data integrity at all. In that function, after reading the data,
bio->bi_idx will be equal to bio->bi_vcnt because of bio_advance(),
so bio_for_each_segment_all() should be used, not
bio_for_each_segment(). And also, bio_advance() should not change
the integrity data bio_integrity_advance() unless the BIO is being
trimmed.
Linux-commit: 63573e359d052e506d305c263576499f06355985

2) The second patch fixes a problem of the sd_dif_complete(). When
sector offset is larger then 2^32, the mapping from physical
reference tag to the virtual values expected by block layer will be
wrong.
Linux-commit: c611529e7cd3465ec0eada0f44200e8420c38908

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Change-Id: Ia6c1d586284b0d9884116e1a753fd88e066366fe
Reviewed-on: https://review.whamcloud.com/32266
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Peter Jones <pjones@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
15 files changed:
lustre/autoconf/lustre-core.m4
lustre/include/lustre_compat.h
lustre/include/obd.h
lustre/include/obd_support.h
lustre/kernel_patches/patches/block-integrity-allow-optional-integrity-functions-rhel7.patch [new file with mode: 0644]
lustre/kernel_patches/patches/block-pass-bio-into-integrity_processing_fn-rhel7.patch [new file with mode: 0644]
lustre/kernel_patches/patches/fix-integrity-verify-rhel7.patch [new file with mode: 0644]
lustre/kernel_patches/patches/fix-sd-dif-complete-rhel7.patch [new file with mode: 0644]
lustre/kernel_patches/series/3.10-rhel7.series
lustre/osd-ldiskfs/Makefile.in
lustre/osd-ldiskfs/osd_handler.c
lustre/osd-ldiskfs/osd_integrity.c [new file with mode: 0644]
lustre/osd-ldiskfs/osd_internal.h
lustre/osd-ldiskfs/osd_io.c
lustre/target/tgt_handler.c