b=12421

i=brian
i=wangdi

Adds stats-collect to RPM build.

diff --git a/lustre-iokit/Makefile.am b/lustre-iokit/Makefile.am
index 919458e..c2bcb86 100644 (file)
--- a/lustre-iokit/Makefile.am
+++ b/lustre-iokit/Makefile.am
@@ -1,4 +1,4 @@
-SUBDIRS = obdfilter-survey sgpdd-survey ost-survey ior-survey
+SUBDIRS = obdfilter-survey sgpdd-survey ost-survey ior-survey stats-collect
 
 EXTRA_DIST = lustre-iokit.spec
 

diff --git a/lustre-iokit/configure.ac b/lustre-iokit/configure.ac
index 9592dad..49b6fc4 100644 (file)
--- a/lustre-iokit/configure.ac
+++ b/lustre-iokit/configure.ac
@@ -1,5 +1,5 @@
 AC_INIT
-AM_INIT_AUTOMAKE(lustre-iokit,1.1)
+AM_INIT_AUTOMAKE(lustre-iokit,1.2)
 AC_PATH_PROGS(BASH, bash)
 AC_PATH_PROGS(PERL, perl)
 RELEASE="`date +%Y%m%d%H%M`"
@@ -12,4 +12,5 @@ sgpdd-survey/Makefile
 obdfilter-survey/Makefile
 ost-survey/Makefile
 ior-survey/Makefile
+stats-collect/Makefile
 )

diff --git a/lustre-iokit/lustre-iokit.spec.in b/lustre-iokit/lustre-iokit.spec.in
index df50b5f..3c06193 100644 (file)
--- a/lustre-iokit/lustre-iokit.spec.in
+++ b/lustre-iokit/lustre-iokit.spec.in
@@ -17,7 +17,7 @@ BuildArch: noarch
 Requires: python > 2.2, sg3_utils
 
 %description
-This package includes four tools:
+This package includes five tools:
 sgpdd-survey:
 A test of the 'bare metal' performance, bypassing as much of the kernel as we can. Uses the sgp_dd utility. 
 
@@ -32,6 +32,9 @@ ost-survey
 This survey tests the client-to-disk performance of individual OSTs, and 
 ranks then for comparison.
 
+stats-collect
+This script will collect IO stats on a defined set of nodes. 
+
 ior-survey:
 A script to run the IOR benchmark. The latest version can be downloaded from http://www.llnl.gov/asci/purple/benchmarks/limited/ior/
 
@@ -56,13 +59,19 @@ make install DESTDIR=$RPM_BUILD_ROOT
 /usr/bin/ost-survey
 /usr/bin/sgpdd-survey
 /usr/bin/parse-sgpdd
+/usr/bin/lstats.sh
+/usr/bin/gather_stats_everywhere.sh
+/usr/bin/config.sh
 %doc obdfilter-survey/README.obdfilter-survey
 %doc ior-survey/README.ior-survey
 %doc ost-survey/README.ost-survey
 %doc sgpdd-survey/README.sgpdd-survey
+%doc stats-collect/README.lstats.sh
 
 
 %changelog
+* Tue Jul 24 2007 Cliff White
+- Added stats-collect
 * Mon Apr 9 2007 Cliff White
 - Merged with existing, changed to .in format.
 * Thu Oct 4 2006  Kalpak Shah

diff --git a/lustre-iokit/stats-collect/Makefile.am b/lustre-iokit/stats-collect/Makefile.am
new file mode 100644 (file)
index 0000000..3cb3f61
--- /dev/null
+++ b/lustre-iokit/stats-collect/Makefile.am
@@ -0,0 +1,3 @@
+bin_SCRIPTS = config.sh gather_stats_everywhere.sh lstats.sh
+CLEANFILE = $(bin_SCRIPTS)
+EXTRA_DIST = README.lstats.sh config.sh gather_stats_everywhere.sh lstats.sh

diff --git a/lustre-iokit/stats-collect/README.lstats.sh b/lustre-iokit/stats-collect/README.lstats.sh
new file mode 100644 (file)
index 0000000..28e4d12
--- /dev/null
+++ b/lustre-iokit/stats-collect/README.lstats.sh
@@ -0,0 +1,75 @@
+Overview
+--------
+These script will be used to collect profile info of lustre client and server.
+It will be run on a single(control) node, and collect all the profile info and 
+create a tarball on the control node. 
+
+lstat.sh : The stat script for single node, which will be run on each profile 
+          node.
+gather_stats_everywhere.sh : collect stats script.
+config.sh : the config for gather_stats_everywhere.sh.
+
+Requirements
+-------
+1) Lustre is installed and setup on your profiling cluster.
+2) ssh/scp to these node names works without requiring a password.
+
+Configuration
+------
+Configuration is very simple for this script, all of the profiling config VARs are
+in config.sh
+
+XXXX_INTERVAL: the profiling interval
+where value of interval means:
+   0 - gather stats at start and stop only
+   N - gather stats every N seconds
+if some XXX_INTERVAL isn't specified, related stats won't be collected
+XXXX can be: VMSTAT, SERVICE, BRW, SDIO, MBALLOC, IO, JBD, CLIENT 
+
+As for ior-collect-stat.sh, you can modify the various IOR and MPI 
+parameters inside ior-collect-stat.sh 
+
+Running
+--------
+1) The gather_stats_everywhere.sh will be run in three mode
+   
+   a)sh gather_stats_everywhere.sh config.sh start 
+     It will start collect stats on each node provided in config.sh
+   
+   b)sh gather_stats_everywhere.sh config.sh stop <log_name>
+     It will stop collect stats on each node. If <log_name> is provided,
+     it will create a profile tarball /tmp/<log_name>.tar.gz.
+   
+   c)sh gather_stats_everywhere.sh config.sh analyse log_tarball.tar.gz csv
+     It will analyse the log_tarball and create a csv tarball for this
+     profiling tarball. 
+
+2) The ior-collect-stat.sh will be run as
+        sh ior-collect-stat.sh start <profile> 
+   It will create a ior result csv file. If <profile> is provided, 
+   the detail profile info tarball will be created under /tmp.
+
+Example
+-------
+When you want collect your profile info, you should
+   1)sh gather_stats_everywhere.sh config.sh start 
+        #start the collect profile daemon on each node.
+
+   2)run your test.
+
+   3)sh gather_stats_everywhere.sh config.sh stop log_tarball
+     #stop the collect profile daemon on each node, cleanup
+      the tmp file and create a profiling tarball.
+
+   4)sh gather_stats_everywhere.sh config.sh analyse log_tarball.tar.gz csv
+     #create a csv file according to the profile.
+
+TBD
+------
+Add liblustre profiling support and add more options for analyse.  
+
+
+
+   
+
+