-
Notifications
You must be signed in to change notification settings - Fork 47
Buildbot plans
Work in progress
The plan is for the buildbot master hardware to be installed in the week beginning 23/04/2012. I have little info about the exact configuration of this machine - more details may follow next week.
- The buildbot master can also function as a build slave. Most of the shorter tests will probably be run on this machine.
- CX1 - it is possible to set up a slave on CX1 that submits jobs using qsub. This should be used for all the short tests (to make sure they compile/work properly on CX1) and also for longer and performance-sensitive tests.
At present a buildbot that has been used for testing is triggered by Github - Github posts a JSON payload describing each commit to a public-facing webpage that then records the payload and posts it to the buildbot web interface - this is done since it is not possible to expose the Buildbot's web admin port externally.
This setup is probably undesirable for the new buildbot because it creates security and potentially reliability issues. The Fluidity buildbot presently polls an email mailbox for messages the describe each commit - it is similarly possible to set Github to send emails to an email box that the new buildbot may poll in order to trigger builds
Tests will generally be run each time someone pushes commits to the master branch of any OP2 repository. If this causes a test failure, it will be the responsibility of the person who pushed the offending commits to fix it immediately.
It should also be possible to set up temporary queues on the buildbot for testing branches - this will facilitate people being able to test their branches before they are merged into master. It will be sensible to mandate that branches cannot be merged into master unless they have already passed all the tests - this will minimise test failures on the master branch.
In order to actually set these up as tests, we need to establish a list of commands that can be run to invoke each test, and to determine whether the test has passed or failed. The buildbot will determine if a test has passed or failed by examining the return code of each of the steps in the build process - if any of these are non-zero, the test will be considered to have failed. For example, in order to test the building of the most current revision of the OP2 sequential library and the C apps, we might expect to have a list of steps like:
git clone [email protected]:OP2/OP2-Common.git
cd OP2-Common/op2/c
./cmake.local
cd ../../apps/c
./cmake.local
There are a few environment variables that need to be set for most of the OP2 build processes - this is not a problem since we can specify exactly the values of the environment variables on a test-by-test basis.
This is a vague outline of the tests that could be run - please feel free to add more suggestions:
- Successful building of the OP2-Common sequential library in C and the fortran interface.
- Successful building of the apps.
- Successful building of the ROSE translator.
- Successful building of the code generated by the ROSE translator.
- Successful building of the code generated by the MATLAB/Octave translator.
- Successful execution of the generated code.
Mike has also expressed an interest in testing for performance regressions of the translated GPU code. These tests should be run on CX1 in order to have exclusive access to a node to obtain reliable performance results.
If you know the set of steps for a test and would like it added to the Buildbot, please open an issue with a title like "Add test for xxx", describing in the issue what the test is testing, and the steps for executing the test.