Github plan

Plan for moving Clawpack to GitHub

See Home for more wiki pages.

This is at least partly out of date!

Reasons for moving

• Loads on kingkong have increased dramatically and we want to move hosting elsewhere.
• Subversion is not adequate for our needs any more -- too difficult to merge for example.
• Most open source projects are moving to centrally-hosted distributed version control, and GitHub seems best.
• We want to make it easier for sub-projects to be managed independently.
• Major changes and reorganization are planned for Clawpack 5.0 and this is the best time to make the switch.

More about Git and GitHub

Git appears to have a steeper learning curve than Mercurial, for example, but is also more powerful for large projects. Many other open source projects have recently moved to GitHub, including several Clawpack uses:

• SciPy: https://github.com/scipy/scipy
• NumPy: https://github.com/numpy/numpy
• matplotlib: https://github.com/matplotlib/matplotlib
• IPython: https://github.com/ipython/ipython

Fernando Perez's blog http://fperez.org/py4science/git.html has many useful links to get started.

See also GitInfo.

Planned structure

There is now a GitHub account named clawpack: https://github.com/clawpack/

This account will host several repositories corresponding to different sub-projects.

Typically a user would have to clone several of these repositories to get everything needed to run a code (but many users would download a tarfile instead, which can be constructed from multiple repositories easily enough).

Splitting into multiple repositories has several advantages:

• The repositories can be managed by different people who take control of those projects.
• Developers would fork a new branch of the project they are working on and when satisfied send pull requests to the manager of that project.
• Developers could also be given push permission to a project without having permission to other projects.
• Users and developers not interested in GeoClaw, say, would not have to worry about what's going on in that repository.
• Ultimately other large projects might be added, such as !ChomboClaw, and separating into repositories will keep other projects from growing huge.
• Official releases of Clawpack will be done by creating tar files that combine together useful projects, e.g. there might be a tarfile for the full set of repositories, a GeoClaw tarfile useful for those doing geophysical flows, etc.

Initial sub-project repositories

The proposed initial repositories are:

• classic: single-grid fortran codes

• amrclaw: AMR codes

• geoclaw: GeoClaw codes

• pyclaw: Pure Python version and Python tools, including PetClaw, the parallel version using PetSC

• sharpclaw: High-order methods (or will this be in pyclaw?)

• riemann: Riemann solvers that can be used with any of the above. This includes both Fortran and Python Riemann solvers. The directory structure could be:

  • /riemann
    • riemann/riemann (for python Riemann solvers)
    • riemann/src (for fortran Riemann solvers)

Within each directory, there could be both Fortran and Python solvers, distinguished by long, explicit names like
rp1_burgers_exact.f and rpt2_euler_roe-with-efix.f. There could also be subdirectories for pointwise Riemann solvers.

• visclaw: visualization tools (what's currently in python/pyclaw/plotters plus matlab tools, others?)

• doc: Master top-level documentation. Sphinx rst files can point to rst files in doc directory of other repositories. The wiki for the doc repository can also serve as the top-level wiki for Clawpack (e.g. this page).

• clawutil: Utility scripts needed by multiple projects, e.g. Makefile.common and Python scripts.

Structure of each repository

We will maintain a uniform structure within most of these repositories. In particular, each should have the following set of subdirectories:

• src: source code. For the Python-based repositories (pyclaw, petclaw, visclaw) this directory is instead named after the package. For the riemann package, there are both src/ and riemann/ subdirectories, to contain the Fortran and Python Riemann solvers, respectively.

• doc: documentation, using Sphinx. Links among the documentation for different packages can be made using the intersphinx extension.

• apps: applications -- examples of how this project can be used. Note that, e.g. classic/apps/acoustics/2d might contain same examples as amrclaw/apps/acoustics/2d (rather than the current system of a app directory having an amr subdirectory). This seems a better way to split it up though and there shouldn't be too much replication of code since, for example, the Riemann solvers will be in the riemann repository, needed by both.

• tests: regression tests. These will be separate from the apps in the future since the needs are different.