Skip to content

Development Workflow

Jump to bottom
nicholasjhorton edited this page Apr 21, 2013 · 33 revisions

This is an outline of the workflow for development and maintenance of the mosaic package.

Dependencies

Maintainers will need the following available to follow this workflow completely.

  • scripting languages: R, bash, perl,
  • R packages for development: devtools, testthat, optparse, roxygen2 (s4 branch from github)
  • R packages for the package itself: lattice, grid, methods, Hmisc, utils, MASS, reshape2, [manipulate]
  • R packages to compile ancillary materials (e.g. books): knitr

Currently, it is assumed that all scripts are run from the parent directory of bin. For example, ./bin/doitall

Github workflow

This outline is still in progress. The git/github workflow is based primarily on the "Private Small Team" model from the ProGit document with a bit of "Private Managed Team" influence.

Seeing the status quo

If you type

gitk &

at the shell prompt, you can see a graphical representation of the commit tree and browse file changes. This can be a handy way to see what the status quo is and to figure out what needs merging where.

Creating new functionality

  1. Create an issue at github describing the new feature to be added and assign the issue to the person doing the coding.

  2. Make sure what you have locally is up to date

git fetch    # safer than git pull in that it allows/requires you to do the merging manually

or

git pull     # this will do any necessary merging that can happen without your intervention

  1. Create a branch named after the issue number (e.g., issue001). Pad to 3 digits.

    Typically this branch will come off of the beta branch or one of its sub-branches. Do not work directly in master or beta.

git branch issue001 origin/beta   # create the new topic branch based on origin/master
git checkout issue001             # switch to the new branch (can also be done inside RStudio)

The last two commands can be combined into

git checkout -b issue001 origin/beta
At this point, **the new topic branch is only on your local machine**. 

(You could build and checkout the new branch using

git checkout -b issue001 beta

but this links your new work to other work you have done since your local beta was in sync with origin/beta. Sometimes this might be just what you want. But if you can work from origin/beta, that is probably the better way to go -- especially if you abandon work currently in the local beta but not in origin/beta.)

  1. Create, document, and test code, committing frequently as you work.
git commit              # or click on commit button in RStudio
# do some stuff
bin/checkpackage.sh     # check that everything is still OK
git commit              # or click on commit button in RStudio
  • Whenever possible, new code should result in new examples, new unit tests, or both.
  • use bin/checkpackage.sh to make sure documentation is up to snuff, examples work, tests pass.
  • First line of commit message should be a short description of committed work. Second line should be blank. The next few lines can add additional details.
  • Avoid committing too many things all at once. Commits are cheap.
  • It is possible to get back to the state you were in at any commit.
  1. Need consultation? Push your topic branch so the rest of us can see it.
git checkout issue001   # or select branch in RStudio
git commit              # or select commit in RStudio
git branch -a           # list all the branches to make sure that there isn't already an origin/issue001 
git push -u origin issue001  # -u will set things up so local issue001 track origin/issue001 and push/pull work
git will report back that the new branch has been made and tracking established. Now others can inspect origin/issue001 or use git pull to pull the branch to their local machine and work with you to fix problems.  Or they can grab a local copy with

git checkout --track origin/issue001

  1. merge topic branch into beta

    When you are to the point where the code in your topic branch is ready to be included in the beta branch,

git checkout beta
git pull                 # or click pull in RStudio (or git fetch plus git merge)
git merge issue001       # incorporate new stuff into local beta
bin/checkpackage.sh      # make sure everything is still good

At this point, you may be done with this branch (see how to delete it below), or you may decide to continue working in this branch. In the latter case, you should first merge the other way around

git checkout issue001
git merge beta

This should be a fast-forward merge because after the previous merge, beta will be a descendent of issue001.

  1. merge local beta into origin/beta

    Since we set up the local beta branch to track origin/beta, this can be done with

git checkout beta        # or select the beta track in RStudio
git push                 # or click push in RStudio

  1. Tagging and removing branches

    Once work on a branch is completed and has been merged into one of the other branches (beta, for example), we don't really need the branch any more. If new issues arise and we reopen the issue, we'll want to begin work somewhere downstream. But we might like a record of where we were when we completed work on the issue. So let's tag it first and then delete.

git checkout issue001
git tag -a 'issue001-1'    # phase 1 of issue001 completed
git checkout beta
git branch -d issue001     # remove the local branch
git push origin :issue001  # remove remote branch too
Now issue001 will not show up in lists of branches, and you won't be tempted to do more work there, but you can still see the state of the project when that issue was completed.  As shown above, the tag is pre-merge.  It would also be possible to tag the beta branch post-merge.  It's not clear to me which is preferable.  Perhaps pre-merge in case we find a problem with the merge and want to go back to just before the merge happened.  If there were issues with the merge that are cleaned up later, another tag can be created if we like.
  1. When ready for public release, beta can be merged into master.
git checkout beta
git merge master
bin/checkpackage.sh
If everything seems clean in beta we can update master

git checkout master
git merge beta             # should be a fast-forward if done after the above
bin/checkpackage.sh        # make sure we're good to go and build package for CRAN release
git tag -a "CRAN x.y-nn"   # tag the release with the version number
git checkout beta          # get off of master so you don't forget

Now submit the tar.gz to to CRAN and get back to work on beta (via issue tracks).
  1. Prior to merging beta into master, it may be useful to have an alpha branch that will become the next beta so that we can continue working on new stuff without delaying the release of more mature code. If we create an alpha branch, it will work just like the beta branch.

Fixing a bug, updating documentation, etc.

But fixes are similar except that those for which immediate deployment is desired could be done in master rather than beta. In brief:

git checkout master
git checkout -b issue002
git commit
git checkout master
git merge issue002
git push

See above for tagging and deleting the branch.

CRAN submission

When master is rebuilt, the resulting package should have a new R version number and be submitted to CRAN.

Building and Checking the Package

origin/beta and especially origin/master should ideally always compile cleanly. Use our scripts to check for this. If you write code to create interactive tools, install the package and test them out. Don't merge in topic branches until everything checks out.

When inevitable bugs are discovered, they should be fixed as quickly as possible.

Miscellaneous Git Stuff

Renaming a Branch

git branch -m oldname newname   # to rename branch named oldname
git branch -m newname  # to rename the current branch

Recovering from "wrong branch syndrome"

If you have accidentally begun working on the wrong branch and catch this early, there is a fairly easy fix. But don't do this if you have already pushed changes that you are trying to revert!

  1. commit your work
git commit -m "work in progress"
  1. Create a new branch.
git branch newbranch
  1. Revert the changes by backing up one commit
git reset --hard HEAD^
  1. Switch to the new branch and continue from there.
git checkout newbranch

The result of this will leave your original branch (beta, master, whatever) back where it was before you accidentally started working there. The new branch will have the "work in progress" stuff that you can now clean up and reintegrate when the time is right. If you need to back up more than one commit, the reverting step will need to repeated or you have to refer to the commit you need directly.

For more information, see [http://effectif.com/git/move-commit-from-one-branch-to-another]

Clone this wiki locally