LU-10460 osd-zfs: Add tunables to disable sync

LU-10460 osd-zfs: Add tunables to disable sync

This patch allows replacing the call to txg_wait_synced(),
which blocks waiting for a full pool sync, with a smaller
tunable delay.  This delay is intended to stand in for the time
it would have taken to synchronously write the dirty data to
the intent log.

This allows testing ZFS behaviour as if there were a low-latency
ZIL device enabled to handle sync IO operations.  Setting the
delay to zero disables sync operations on the server completely.
However, be aware that no data is guaranteed to be written to
disk if the tunables are enabled, and this patch is solely for
performance analysis.  By default the tunables are set to -1,
which leaves the system using the normal sync behaviour.

Two new tunables are introduced to control the delay, the
osd_object_sync_delay_us and osd_txg_sync_delay_us module options.
These values default to -1 which preserves the safe full sync
pool behavior.  Setting these values to zero or larger will
replace the pool sync with a delay of N microseconds.

The initial test results obtained by running sanityN test 16
(fsx) are encouraging.  If the zil_commit() time can be kept to
less than 10ms we should see a significant performance improvement.
These tests were run in a pristine centos 6.4 VM and the results
are averaged over four runs.

osd_txg_sync_delay_us     -1    -1     -1     -1      -1
osd_obj_sync_delay_us     -1     0   1000  10000  100000
--------------------------------------------------------
SanityN test 16 (secs)  24.3   7.3    7.6   10.1    34.4

Lustre-change: https://review.whamcloud.com/7761
Lutre-commit: 00c4bd85b372f68c3e75fdab9658c8f0074113be

Change-Id: Iff9b66888edc79a5e1585fa3ce8377be068748f2
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Darby Vicker <darby.vicker-1@nasa.gov>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/31163
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>