--- /dev/null
+Patch management scripts
+Andrew Morton <akpm@digeo.com>
+18 October 2002
+
+This is a description of a bunch of shell scripts which I use for
+managing kernel patches. They are quite powerful. They can be used on
+projects other than the linux kernel. They are easy to use, and fast.
+
+You end up doing a ton of recompiling with these scripts, because
+you're pushing and popping all the time. ccache takes away the pain of
+all that. http://ccache.samba.org/ - be sure to put the cache
+directory on the same fs as where you're working so that ccache can use
+hardlinks.
+
+The key philosophical concept is that your primary output is patches.
+Not ".c" files, not ".h" files. But patches. So patches are the
+first-class object here.
+
+Installation
+============
+
+You place all the scripts somewhere in your path, or in
+/usr/lib/patch-scripts.
+
+Terminology
+===========
+
+The patch scripts require three special directories called "pc",
+"patches" and "txt".
+
+If the environment variable PATCHSCRIPTS is set, it is taken to to be
+the directory in which those three directories reside. Typically, it
+would be a relative pathname. So
+
+ setenv PATCHSCRIPTS ./i-put-them-here
+
+would tell the patch scripts to look in ./i-put-them-here/pc, etc.
+
+If PATCHSCRIPTS is not set, and the directory ./patch-scripts is
+present then the patch scripts will us ./patch-scripts/pc/,
+./patch-scripts/patches/ and ./patch-scripts/txt/.
+
+Otherwise, the patch scripts use ./pc, ./patches and ./txt.
+
+In this document, the symbol $P is used to describe the directory which
+holds the pc/, patches/ and txt/ directories, as determined by the
+above search.
+
+It is expected that $P will always expand to a relative path.
+
+Concepts
+========
+
+All work occurs with a single directory tree. All commands are invoked
+within the root of that tree. The scripts manage a "stack" of patches.
+
+Each patch is a changeset against the base tree plus the preceding patches.
+
+All patches are listed, in order, in the file ./series. You manage the
+series file.
+
+Any currently-applied patches are described in the file
+./applied-patches. The patch scripts manage this file.
+
+Each patch affects a number of files in the tree. These files are
+listed in a "patch control" file. These .pc files live in the
+directory $P/pc/
+
+Patches are placed in the directory $P/patches/
+
+Documentation for the patches is placed in $P/txt/
+
+So for a particular patch "my-first-patch" the following will exist:
+
+- An entry "my-first-patch.patch" in ./series
+
+- An entry "my-first-patch" in ./applied-patches (if it's currently applied)
+
+- A file $P/pc/my-first-patch.pc which contains the names of the
+ files which my-first-patch modifies, adds or removes
+
+- A file $P/txt/my-first-patch.txt which contains the patch's
+ changelog.
+
+- A file $P/patches/my-first-patch.patch, which is the output of the
+ patch scripts.
+
+Operation
+=========
+
+When a patch "my-patch" is applied with apatch, or with pushpatch
+(which calls apatch), all the affected files (from $P/pc/my-patch.pc)
+are copied to files with ~my-patch appended. So if $P/pc/my-patch.pc
+contained
+
+ kernel/sched.c
+ fs/inode.c
+
+then apatch will copy those files into kernel/sched.c~my-patch and
+fs/inode.c~my-patch. It will then apply the patch to kernel/sched.c
+and fs/inode.c
+
+When a diff is regenerated by refpatch (which calls mpatch), the diff
+is made between kernel/sched.c and kernel/sched.c~my-patch. How do the
+scripts know to use "~my-patch"? Because my-patch is the current
+topmost patch. It's the last line in ./applied-patches.
+
+In this way, the whole thing is stackable. If you have four patches
+applied, say "patch-1", "patch-2", "patch-3" and "patch-4", and if
+patch-2 and patch-4 both touch kernel/sched.c then you will have:
+
+ kernel/sched.c~patch-2 Original copy, before patch-2
+ kernel/sched.c~patch-4 Copy before patch-4. Contains changes
+ from patch-2
+ kernel/sched.c Current working copy. Contains changes
+ from patch-4.
+
+This means that your diff headers contain "~patch-name" in them, which
+is convenient documentation.
+
+Walkthrough
+===========
+
+Let's start.
+
+Go into /usr/src/linux (or wherever)
+
+ mkdir pc patches txt
+
+Now let's generate a patch
+
+ fpatch my-patch kernel/sched.c
+
+OK, we've copied kernel/sched.c to kernel/sched.c~my-patch. We've
+appended "my-patch" to ./applied-patches and we've put "kernel/sched.c"
+into the patch control file, pc/my-patch.pc.
+
+ Now edit kernel/sched.c a bit.
+
+Now we're ready to document the patch
+
+ Now write txt/my-patch.txt
+
+Now generate the patch
+
+ refpatch
+
+This will generate patches/my-patch.patch. Take a look.
+
+Now remove the patch
+
+ poppatch
+
+applied-patches is now empty, and the patch is removed.
+
+Now let's add a file to my-patch and then generate my-second-patch:
+
+ Add "my-patch.patch" to ./series (no blank lines in that file please)
+
+ pushpatch
+
+OK, the patch is applied again. Let's add another file
+
+ fpatch kernel/printk.c
+
+Note that here we gave fpatch a single argument. So rather than
+opening a new patch, it adds kernel/printk.c to the existing topmost
+patch. That's my-patch.
+
+ Edit kernel/printk.c
+
+Refresh my-patch (you end up running refpatch a lot)
+
+ refpatch
+
+Now start a second patch:
+
+ fpatch my-second-patch kernel/sched.c
+
+Now take a look at applied-patches. Also do an `ls kernel/sched*'.
+
+ Edit kernel/sched.c, to make some changes for my-second-patch
+
+Generate my-second-patch:
+
+ refpatch
+
+Take a look in patches/my-second-patch.patch
+
+Don't forget to add "my-second-patch.patch" to the series file.
+
+And remove both patches:
+
+ poppatch
+ poppatch
+
+
+That's pretty much it, really.
+
+
+Command reference
+=================
+
+Generally, where any of these commands take a "patch-name", that can be
+of the form txt/patch-name.txt, patch-name.pc, just patch-name or
+whatever. The scripts will strip off a leading "txt/", "patches/" or
+"pc/" and any trailing extension. This is so you can do
+
+ apatch patches/a<tab>
+
+to conveniently use shell tabbing to select patch names.
+
+
+
+added-by-patch
+
+ Some internal thing.
+
+apatch [-f] patch-name
+
+ This is the low-level function which adds patches. It does the
+ copying into ~-files and updates the applied-patches file. It
+ applies the actual patch.
+
+ apatch will do a patch --dry-run first and will refuse to apply the
+ patch if the dryrun fails.
+
+ So when you are getting rejects you do this:
+
+ pushpatch # This fails, due to rejects. Drat.
+ apatch -f patch-name # Force the patch
+ (or) pushpatch -f # Force the patch
+
+ OK, you've now applied patch-name, but you have rejects. Go fix
+ those up and do
+
+ refpatch
+
+ And you're ready to move on.
+
+combine-series output-file
+
+ It incrementally combinediffs all the patches in series to make a
+ complete patch for the series. Requires combinediff frmo patchutils.
+
+ See http://cyberelk.net/tim/patchutils/ (Don't download the
+ "experimental" patchutils - it seems to only have half of the
+ commands in it. Go for "stable")
+
+cvs-take-patch
+
+ I forget.
+
+export_patch
+
+ export the patches listed in ./series to a set of files which
+ are named in such a way that the sort order is the same as the
+ order of the series file.
+
+ Usage: export_patch directory [prefix]
+
+ Example:
+
+ Suppose ./series contains
+
+ mango.patch
+ orange.patch
+ banana.patch
+ apple.patch
+ pear.patch
+
+ export_patch ../mypatches fruit
+
+ The patches would be copied to
+
+ ../mypatches/p00001_fruit_mango.patch
+ ../mypatches/p00002_fruit_orange.patch
+ ../mypatches/p00003_fruit_banana.patch
+ ../mypatches/p00003_fruit_banana.patch
+ ../mypatches/p00003_fruit_banana.patch
+
+ Named in this way, someone may easily apply them:
+
+ cat mypatches/p*fruit* | patch -p1
+
+ If prefix is omitted, the patchnames will be transformed
+ such that "original.patch" becomes "pXXXXX_original.patch".
+
+fpatch [patch-name] foo.c
+
+ If patch-name is given, fpatch will start a new patch which
+ modifies (or adds, or removes) the single file foo.c. It updates
+ ./applied-patches and creates pc/patch-name.pc. fpatch will copy
+ foo.c to foo.c~patch-name in preparation for edits of foo.c.
+
+ If patch-name is not given then fpatch will add foo.c to the
+ current topmost patch. It will add "foo.c" to $P/pc/$(toppatch).pc.
+ It will copy foo.c to foo.c~$(toppatch).
+
+import_patch
+
+ Imports a set of patch files, creating $P/pc, $P/txt, $P/patches and
+ ./series as necessary. It also creates $P/txt/*.txt by stripping
+ off the top of the patches (and removes any diffstat output it finds,
+ so that it can eat refpatch output and export_patch output.) The
+ imported patch names are appended to the series file.
+
+ In creating the $P/txt/*.txt files, mail headers are stripped with
+ formail, preserving the "From:" and "Subject:" lines. "DESC" and
+ "EDESC" markers are added if they are not already present, using the
+ "From:" and "Subject:" lines for the DESC portion, if they are present.
+ (See "patchdesc" command, below, for more on these markers.)
+
+ Also, it can rename the patch file as it is imported by stripping out
+ a pattern. This is useful if, as often is the case, you have patch
+ sets with filenames designed to help sort the patches into the correct
+ order, such as "p001_xxx_funky_stuff.patch" you can have it automatically
+ renamed to funky_stuff.patch on import, and let the series file manage
+ the ordering.
+
+ Import_patch will uncompress patches (*.Z, *.bz2, *.gz) as necessary.
+
+ Usage:
+
+ import_patch [-p pattern] patchfile ...
+
+ Example:
+
+ % ls ../fruit/p*patch
+ ../fruit/p00001_northern_apple.patch
+ ../fruit/p00001_tropical_mango.patch
+ ../fruit/p00002_northern_pear.patch
+ ../fruit/p00002_tropical_orange.patch
+ ../fruit/p00003_tropical_banana.patch
+ % import_patch -p 'p[0-9]*_tropical_' ../fruit/p*tropical*
+ Recreated pc/mango.pc
+ Recreated pc/orange.pc
+ Recreated pc/banana.pc
+ % import_patch -p 'p[0-9]*_northern_' ../fruit/p*northern*
+ Recreated pc/apple.pc
+ Recreated pc/pear.pc
+
+ Then you can "pushpatch; refpatch" 5 times.
+
+inpatch
+
+ List the names of ths files which are affected by the current
+ topmost patch.
+
+ This is basically
+
+ cat pc/$(toppatch).pc
+
+mpatch
+
+ A low-level thing to generate patches
+
+new-kernel
+
+ Some thing I use for importing a new kernel from kernel.org
+
+p0-2-p1
+
+ Internal thing to convert patch -p0 form into patch -p1
+
+patchdesc
+
+ Generates a single-line description of a patch.
+
+ The txt/my-patch.txt files have the following format:
+
+ <start of file>
+ DESC
+ some short description
+ EDESC
+
+ The long description
+ <end of file>
+
+ I use
+
+ patchdesc $(cat series)
+
+ to generate short-form summaries of the patch series.
+
+patchfns
+
+ Internal utilities
+
+pcpatch
+
+ Standalone tool to generate a .pc file from a patch.
+
+ Say someone sends you "his-patch.diff". What you do is:
+
+ cp ~/his-patch.diff patches/his-patch.patch
+ pcpatch his-patch
+
+ This generates $P/pc/his-patch.pc and you're all set. Add
+ "his-patch.patch" to ./series in the right place and start pushing.
+
+p_diff
+
+ I forget
+
+poppatch
+
+ Remove one or more patches from the current stack. This command
+ does *not* use the series file. It works purely against
+ applied-patches.
+
+ Usage:
+
+ poppatch
+ Remove the topmost patch
+ poppatch 10
+ Remove ten patches
+ poppatch some-patch-name[.patch]
+ Remove patches until "some-patch-name" is top patch
+
+pstatus
+
+ Shows status of patches
+
+ Usage:
+ pstatus [patchfile ...]
+
+ One line per patch is output showing:
+ 1: Patch number in the series file
+ 2: Whether the patch is currently applied
+ 3: Name of patch
+ 4: Status of the patch (needs pcpatch, changelog, refpatch)
+
+ If no patchfiles are specified, $P/patches/*.patch
+ are assumed.
+
+ Caveats:
+ A patch set which contains separate patches to add a file
+ and modify that same file may give spurious "Needs refpatch"
+ status for the patch which adds the file or the topmost patch.
+
+ptkdiff
+
+ Two modes:
+
+ ptkdiff -
+
+ Run tkdiff against all the file affected
+ by $(toppatch). The diff is only for the changes made
+ by the top patch! ie: it's between "filename" and
+ "filename~toppatch-name".
+
+ ptkdiff filename
+
+ Just run tkdiff against that file,
+ showing the changes which are due to toppatch.
+
+pushpatch [-f]
+
+ Apply the next patch, from the series file.
+
+ This consults ./applied-patches to find out the top patch, then
+ consults ./series to find the next patch. And pushes it.
+
+ pushpatch
+
+ Apply the next patch
+
+ pushpatch 10
+
+ Apply the next ten patches
+
+ pushpatch some-patch-name
+
+ Keep pushing patches until "some-patch-name" is toppatch
+
+ pushpatch -f
+
+ Push the next patch, ignoring rejects.
+
+refpatch
+
+ regnerates the topmost patch. Reads all the affected files
+ from pc/$(toppatch).pc and diffs them against their tilde-files.
+
+ Also pastes into the patch your patch documentation and
+ generates a diffstat summary.
+
+removed-by-patch
+
+ Some thing.
+
+rename-patch
+
+ CVS rename for patches.
+
+rolled-up-patch
+
+ Bit of a hack. Is designed to generate a rolled-up diff of all
+ currently-applied patches. But it requires a ../linux-2.x.y tree to
+ diff against. Needs to be redone.
+
+rpatch
+
+ Internal command
+
+split-patch
+
+ Some thing someone write to split patches up. I don't use it.
+
+tag-series
+
+ Assuming you keep pc/*, patches/* and txt/* under CVS revision
+ control, tag-series allows you to tag a patchset's individual
+ components. I use
+
+ tag-series s2_5_44-mm3 pc/2.5.44-mm3-series
+
+ which will attach the cvs tag "s2_5_44-mm3" to every .pc, .patch
+ and .txt file which is mentioned in the series file
+ "pc/2.5.44-mm3-series".
+
+ It will also tag pc/2.5.44-mm3-series, which is a bit redundant
+ given that I use a different series file for each patchset release..
+
+
+toppatch
+
+ Print the name of the topmost patch. From ./applied-patches
+
+touched-by-patch patch-filename
+
+ List the names of files which are affected by a diff.
+
+unitdiff.py
+
+ Rasmus Andersen's script to convert a diff into minimum-context
+ form. This form has a better chance of applying if you're getting
+ nasty rejects. But patch can and will make mistakes when fed
+ small-context input.
+
+
+Work Practices
+==============
+
+I keep the kernel tree, the $P/pc/, $P/patches/ and $P/txt/ contents under
+CVS control. This is important...
+
+I have several "series" files. I keep these in $P/pc/foo-series and use
+
+ ln -s pc/foo-series series
+
+when I'm working on foo.
+
+If someone sends me a patch I'll do:
+
+ cp ~/whatever patches/his-patch.patch
+ pcpatch his-patch
+ apatch his-patch
+
+ If apatch fails then run `apatch -f his-patch' and fix the rejects.
+
+ refpatch
+
+ to clean up any fuzz.
+
+ poppatch
+ cvs add pc/his-patch.pc patches/his-patch.patch
+ cvs commit pc patches
+
+ Now edit ./series and place "his-patch.patch" in the appropriate place.
+
+
+If you're working on a particular patch (say, "dud-patch") and you
+balls something up, just run:
+
+ refpatch # Generate the crap patch
+ poppatch # Remove it all
+ rm patches/dud-patch.patch
+ cvs up patches/dud-patch.patch
+
+and all is well.
+
+
+Getting updates from Linus
+==========================
+
+What I do is to grab the latest -bk diff from
+http://www.kernel.org/pub/linux/kernel/people/dwmw2/bk-2.5/
+and do:
+
+ gzip -d < cs<tab> > patches/linus.patch
+ pcpatch linus
+ apatch linus | grep diff
+
+ Now fix up all the files which got deleted,
+ because there's something wrong with bitkeeper diffs:
+
+ cvs up -ko <missing files from the above diff>
+
+ apatch linus
+ $EDITOR linus/linus.txt
+
+ Add the changeset number to txt/linus.txt
+
+ refpatch
+ poppatch
+
+ Now add "linus.patch" as the first entry in your ./series file and
+ start pushing your other patches on top of that.
+
+BUGS
+====
+
+Tons and tons. The scripts are fragile, the error handling is ungraceful and
+if you do something silly you can end up in a pickle.
+
+Generally the scripts are very careful to not wreck your files or your
+patches. But they can get the ./applied-patches and ~-files into an
+awkward state.
+
+Usually you can sort it out by copying the ~-files back onto the originals
+and removing the last line from ./applied-patches. Or do a "refpatch ;
+poppatch ; rm patches/troublesome-patch.patch ; cvs up patches".
+
+If it's really bad, just blow away the entire tree and do a new CVS checkout.
+
+
+Working on non-kernel projects
+==============================
+
+Well it's the same thing. Say you've downloaded a copy of util-linux
+and you want to make a change:
+
+ cd /usr/src
+ tar xvfz ~/util-linux.tar.gz
+ cd util-linux
+ mkdir pc patches txt
+ fpatch my-patch sys-utils/rdev.c
+ fpatch sys-utils/ipcs.8
+ <edit, edit>
+ refpatch
+ <ship patches/my-patch.patch>
+
+How to balls things up
+======================
+
+Well here's one way. Suppose you have 20 patches applied, and three of
+them (say, "p1", "p6" and "p11") all modify "foo.c".
+
+Now you go and change foo.c.
+
+Well, to which patch does that change belong? You need to decide.
+Let's say you decide "p6".
+
+If you run `refpatch' when "p11" is toppatch then you lose. The diff
+went into p11.
+
+What you can do is:
+
+1:
+ poppatch p6
+ <edit>
+ refpatch
+ pushpatch p11
+ <test>
+
+ (See why ccache is looking good?)
+
+or
+
+2:
+ <edit>
+ <test>
+ poppatch p6 <hope like hell that the other patches remove cleanly>
+ refpatch
+
+
+Another good way of ballsing up is to cheat. Say "oh I just want to make
+this one-line change". And "oh, and this one".
+
+Now you're getting in a mess. It's much, much better to just use the system:
+
+ fpatch junk file1
+ fpatch file2
+ <edit>
+ <play>
+ refpatch
+ poppatch
+ rm pc/junk.pc patches/junk.patch
+
+Merging with -mm kernels
+========================
+
+Haven't tried this, but it should work:
+
+- Grab all the patches from broken-out/, place them in your $P/patches/
+
+- Copy my series file into ./series (or $P/pc/akpm-series and symlink it)
+
+- pushpatch 99
+
+And you're off and running. The nice thing about this is that you can
+send me incremental diffs to diffs which I already have.
+
+Or whatever. I'm fairly handy with diffs nowadays. Rejects are
+expected. I just prefer to have "one concept per diff".
+
git://git.whamcloud.com / fs / lustre-release.git / blobdiff