Documentation updates. (#203)

daisy/README.md

@@ -36,12 +36,12 @@ Other use-case examples:
       * [CopyGCSObjects](#type-copygcsobjects)
       * [DeleteResources](#type-deleteresources)
       * [IncludeWorkflow](#type-includeworkflow)
-      * [RunTests](#type-runtests)
       * [SubWorkflow](#type-subworkflow)
       * [WaitForInstancesSignal](#type-waitforinstancessignal)
     * [Dependencies](#dependencies)
     * [Vars](#vars)
       * [Autovars](#autovars)
+  * [Testing](#testing)
   * [Glossary of Terms](#glossary-of-terms)
     * [GCE](#glossary-gce)
     * [GCP](#glossary-gcp)
@@ -452,9 +452,6 @@ This IncludeWorkflow step example uses a local workflow file and passes a var,
 }
 ```
 
-#### Type: RunTests
-Not implemented yet.
-
 #### Type: SubWorkflow
 Runs a Daisy workflow as a step. The subworkflow will have some fields
 overwritten. For example, the subworkflow may specify a GCP Project "foo",
@@ -623,6 +620,26 @@ out of convenience. Here is the exhaustive list of autovars:
 | OUTSPATH | Equivalent to ${SCRATCHPATH}/outs. |
 | USERNAME | Username of the user running the workflow. |
 
+## Testing
+Infrastructure has been set up to perform presubmit testing on PRs and
+periodic continuous integration tests against HEAD.
+
+Presubmit checks unit tests, `golint`, `go fmt`, and `go vet` against PRs
+with changes to Daisy source code. Unit test coverage is reported to
+codecov.io, which posts coverage reports on the PR. Presubmit tests are
+gated by repo owners. Repo owners have the following commands available on
+a PR:
+* `/go test`: runs unit tests and reports coverage.
+* `/golint`: runs `golint`.
+* `/go fmt`: runs `go fmt`.
+* `/go vet`: runs `go vet`.
+* `/ok-to-test`: gives Prow the go-ahead to run the entire suite automatically.
+* `/retest`: reruns failed tests, only available after Prow reports failures.
+
+Periodic tests run every 6 hours. Currently, periodic tests include the e2e
+tests here: [../daisy_workflows/e2e_tests](../daisy_workflows/e2e_tests). You
+can see the test results for the e2e tests in testgrid: [https://k8s-testgrid.appspot.com/google-gce-compute-image-tools#daisy-e2e].
+
 ## Glossary of Terms
 Definitions:
 * <a id="glossary-gce"></a>GCE: Google Compute Engine

daisy/prowjob/run/ci/__main__.py

@@ -65,7 +65,7 @@ def build_subsuites(wfs):
             continue
         suite = match.group('suite')
         test_num = match.group('test_num')
-        test_num = int(test_num) if test_num else 0
+        test_num = int(test_num) if test_num else -1
         suites[suite] = suites.get(suite, []) + [(test_num, wf)]
 
     for suite in suites:

daisy_workflows/e2e_tests/README.md

@@ -0,0 +1,31 @@
+# Daisy E2E Tests
+This directory contains end to end tests for Daisy. These tests are run 
+periodically against HEAD.
+
+* Files matching `*.wf.json` are individual test cases.
+  * If you have a sub wf that you do not want to run as a test case, you need
+    to use a different file extension, e.g. `thing.subwf.json`, or put it in a
+    subdirectory, e.g. `subdir/thing.wf.json`.
+* Test cases run in parallel except in cases of ordered test cases.
+* Ordered test cases:
+  * Share a prefix.
+  * Use non-negative numeric suffixes (before the `.wf.json`). If there
+    is a test case without a suffix, it runs first.
+  * e.g. `foo.wf.json`, `foo0.wf.json`, `foo1.wf.json`, and `foo014.wf.json`:
+   `foo.wf.json`  will run, then `foo0.wf.json`, then `foo1.wf.json`, then
+   finally `foo014.wf.json`.
+
+## Test Environment Details
+* Tests run in the GCP project `gce-daisy-test` and have permissions:
+  * GCE read/write
+  * GCS read on the `gce-daisy-test-resources` bucket.
+  * GCS read/write on the `gce-daisy-test-sandbox` bucket.
+* Test logs are written to the `gce-daisy-test` GCS bucket for Gubenator and 
+  testgrid to pick up.
+* Defaults are provided for workflow `Project` and `Zone` fields.
+  * `Project` is `gce-daisy-test`
+  * `Zone` is variable.
+* The following args are passed to the test workflows:
+  * `test-id`: The ID of this test run. Useful for sharing resources between
+    test cases (would probably need to be ordered test cases, since parallel)
+    test cases would cause race conditions).