b=3668

b=3668
r=adilger
A new lighter-weight CDEBUG infrastructure based on Zach's tracefile
patches:

- Each CPU gets its own list of pages, so a line can be inserted
  without locking
- we do less printf'ing in the kernel, and send a packed structure of
  integers down for the leading bits of each line
- lctl unpacks them, sorts the results (each page is sorted, but not
  globally), and prints
- we are no longer limited to 5MB, because we don't vmalloc a single
  huge buffer (/proc/sys/portals/debug_size, in pages)
- result: on a single system with client, MDS, OST, running iozone
  with -1 debug is no longer CPU-bound

There is also a new debug daemon, naturally, which works in basically
the same way as the old one. There are a couple of points, though:

- you need to use lctl on the results, to unpack the struct and
  sort. It does not sort the whole n gigabytes as one piece, but it
  does sort each "chunk" that was dumped at a time. Those chunks are
  guaranteed to be chronologically ordered.
- lctl debug_daemon [start|stop] are implemented in terms of /proc
- you can "debug_daemon start <file>", or echo to proc, multiple
  times. If it's already running, it will atomically change to a new
  file, so that you can build a set of rotating debug daemon logs,
  without filling up your disk, and without losing any messages during
  the transition from one file to another.