Merge pull request #14 from kjsanger/feature/ont-email

Add documentation

README.md

@@ -20,10 +20,12 @@ is sent to `porch`.
 
 ## Scope
 
-The current version implements notifications for PacBio sequencing platform
+The current version implements notifications for PacBio and ONT sequencing platform
 customers.
 
-## Running the scripts
+## PacBio
+
+### Running the scripts
 
 To register recently QC-ed entities as tasks with `porch`
 
@@ -41,3 +43,80 @@ Processing includes claiming one task, sending per-study emails and updating the
 status of the `porch` task to `DONE`.
 
 The test data directory has an example of a [configuration file](tests/data/qc_state_app_config.ini).
+
+### ONT
+
+#### Configuration
+
+Configuration requires an INI-format file with the following sections:
+
+```ini
+[MySQL MLWH]
+dbhost = <MySQL host>
+dbport = <MySQL port>
+dbuser = <MySQL user>
+dbpassword = <MySQL password>
+dbschema = <MySQL schema>
+
+[PORCH]
+url = <Porch server http URL>
+admin_token = <Porch server admin authentication token>
+pipeline_token = <Porch server pipeline authentication token>
+
+[MAIL]
+domain = <email FQDN>
+```
+
+The first time a new pipeline is created on a particular `porch` service, the following two
+steps are requiredL
+
+- Register the pipeline with `porch`:
+
+```bash
+ npg_ont_event_notification --verbose --colour --conf-file path/to/config.ini register
+```
+
+This creates a new record of the pipeline and its version in the `porch` database. This
+step requires an admin token for the `porch` service to be in the configuration file. The
+`--verbose` and `--colour` flags are optional, but help to see the progress of the script.
+
+- Obtain a new pipelne authentication token:
+
+```bash
+ npg_ont_event_notification --verbose --colour --conf-file path/to/config.ini token
+```
+
+This creates a new token and prints it to STDOUT. Once the token is obtained, it should be
+added to the configuration file. The token cannot be retrieved again. This token is used to
+authenticate with the `porch` service when submitting and running tasks.
+
+#### Running the script
+
+There are two parts to the notification process:
+
+1. Finding new ONT runs of interest and adding them to the notification pipeline as tasks.
+2. Processing the tasks and sending the notifications.
+
+Each of these steps is typically run as a separate cron job. Adding the same task multiple
+times is safe, as `porch` will process is only once.
+
+- Find new ONT runs and add tasks:
+
+```bash
+<run disocvery script> | npg_ont_event_notification --verbose --colour --conf-file path/to/config.ini --log-config path/to/logging.ini add
+```
+
+The discovery script can be anything that writes [baton](http://wtsi-npg.github.io/baton)
+format JSON to STDOUT, one JSON object per line e.g. the `locate-data-objects` script from
+https://github.com/wtsi-npg/npg-irods-python . The JSON must include the metadata associated
+with the path it represents.
+
+- Process tasks:
+
+```bash
+npg_ont_event_notification --conf-file path/to/config.ini --log-config path/to/logging.ini run
+```
+
+This will claim up to 100 tasks from the `porch` service, process them and mark them as done
+if they are successful, or retry them if they fail. The claim size is fixed and should be
+sufficient to clear all tasks each time the script is run.

src/npg_notify/ont/event.py

@@ -70,16 +70,17 @@
 authentication token, and the MySQL MLWH database connection details. The configuration
 file should be in INI format and have the following sections:
 
+    [MySQL MLWH]
+    dbhost = <MySQL host>
+    dbport = <MySQL port>
+    dbuser = <MySQL user>
+    dbpassword = <MySQL password>
+    dbschema = <MySQL schema>
+
     [PORCH]
     url = <Porch server URL>
-    token = <Porch server authentication token>
-
-    [MySQL MLWH]
-    host = <MySQL host>
-    port = <MySQL port>
-    user = <MySQL user>
-    password = <MySQL password>
-    db = <MySQL database>
+    admin_token = <Porch server admin authentication token>
+    pipeline_token = <Porch server pipeline authentication token>
 
     [MAIL]
     domain = <email FQDN>

tests/data/ont_event_app_config.ini

@@ -1,18 +1,16 @@
-# Configuration for the npg_notify package.
-
-[MLWH]
+[MySQL MLWH]
 
 dbuser = test
 dbpassword = test
-dbhost = 127.0.0.1
+dbhost = mysql-server
 dbport = 3306
 dbschema = study_notify
 
 [PORCH]
 
-url = http://127.0.0.1:8081
+url = http://porch-server:8081
 admin_token = 00000000000000000000000000000000
 
 [MAIL]
 
-domain = some.com
+domain = not-a-real-domain.sanger.ac.uk

tests/ont/test_porch.py

@@ -80,7 +80,7 @@ def test_configure(self):
         assert config is not None
         assert config.admin_token == "0" * 32
         assert config.pipeline_token is None
-        assert config.url == "http://127.0.0.1:8081"
+        assert config.url == "http://porch-server:8081"
 
     @m.context("When configured from both an INI file and environment variables")
     @m.it("Loads the configuration and falls back to environment variables")
@@ -98,7 +98,7 @@ def test_configure_env(self):
         assert config is not None
         assert config.admin_token == "0" * 32
         assert config.pipeline_token == pipeline_token
-        assert config.url == "http://127.0.0.1:8081"
+        assert config.url == "http://porch-server:8081"
 
     @m.context("When a pipeline has been defined")
     @m.it("Can be registered")