LU-7988 hsm: change coordinator start/stop mechanisms

LU-7988 hsm: change coordinator start/stop mechanisms

Instead of using cdt_state and cdt_thread.t_flags to keep the state of
the coordinator, only use cdt_state. cdt_thread.t_flags is now only
used to signal the coordinator, not to stop it. cdt_state is used to
control the coordinator behaviour.

Split mdt_hsm_cdt_stop() into 2 functions. One to actually signal the
coordinator to stop, and another one to cleanup its ressources. The
coordinator is now responsible to clean its own ressources instead of
having two different paths depending on whether Lustre is shutting
down, or a user is stopping it through /proc.

Protect cdt_state with a spin lock, and add a transition table to
catch invalid transitions.

Removed mdt_opts.mo_coordinator as it is just a subset of cdt_state.

Test-Parameters: trivial testlist=sanity-hsm
Signed-off-by: frank zago <fzago@cray.com>
Signed-off-by: Quentin Bouget <quentin.bouget@cea.fr>
Change-Id: I7b0a878792d287b781578a7afe4e2c2cf7dec5cc
Reviewed-on: https://review.whamcloud.com/22667
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Jean-Baptiste Riaux <riaux.jb@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>