diff --git a/.gitignore b/.gitignore index 886d2bb3f..20bf19c96 100644 --- a/.gitignore +++ b/.gitignore @@ -18,6 +18,7 @@ develop-eggs lib lib64 __pycache__ +venv # Installer logs pip-log.txt diff --git a/docs/README.md b/docs/README.md index b23bbede4..02d80aa65 100644 --- a/docs/README.md +++ b/docs/README.md @@ -6,7 +6,7 @@ format. [Jupyter](https://jupyter.org) notebooks are also used for in-depth cod ## Setup To update the documentation, ensure that Sphinx, nbsphinx and Jupyter are installed on your system (hint: -run `pip install -r requirements-dev.txt` to install the dependencies). From here, `make html` will +run `pip install -r requirements.txt` to install the dependencies). From here, `make html` (or `./make.bat html` on Windows) will build the documentation, which you can serve using any standard web server. ## Jupyter diff --git a/docs/source/_static/cap/add_cap_alert.png b/docs/source/_static/cap/add_cap_alert.png new file mode 100644 index 000000000..f766c9174 Binary files /dev/null and b/docs/source/_static/cap/add_cap_alert.png differ diff --git a/docs/source/_static/cap/add_mqtt_broker.png b/docs/source/_static/cap/add_mqtt_broker.png new file mode 100644 index 000000000..dc061cced Binary files /dev/null and b/docs/source/_static/cap/add_mqtt_broker.png differ diff --git a/docs/source/_static/cap/alert_preview.png b/docs/source/_static/cap/alert_preview.png new file mode 100644 index 000000000..8e43d385b Binary files /dev/null and b/docs/source/_static/cap/alert_preview.png differ diff --git a/docs/source/_static/cap/alerts_menu.png b/docs/source/_static/cap/alerts_menu.png new file mode 100644 index 000000000..c8bbeec2d Binary files /dev/null and b/docs/source/_static/cap/alerts_menu.png differ diff --git a/docs/source/_static/cap/broker_configuration.png b/docs/source/_static/cap/broker_configuration.png new file mode 100644 index 000000000..a3020b36d Binary files /dev/null and b/docs/source/_static/cap/broker_configuration.png differ diff --git a/docs/source/_static/cap/dataset_confirmation.png b/docs/source/_static/cap/dataset_confirmation.png new file mode 100644 index 000000000..d74b641d6 Binary files /dev/null and b/docs/source/_static/cap/dataset_confirmation.png differ diff --git a/docs/source/_static/cap/download_view_alert.png b/docs/source/_static/cap/download_view_alert.png new file mode 100644 index 000000000..e3cb962df Binary files /dev/null and b/docs/source/_static/cap/download_view_alert.png differ diff --git a/docs/source/_static/cap/monitoring_dashboard.png b/docs/source/_static/cap/monitoring_dashboard.png new file mode 100644 index 000000000..326c3dac8 Binary files /dev/null and b/docs/source/_static/cap/monitoring_dashboard.png differ diff --git a/docs/source/_static/cap/mqtt_broker_list.png b/docs/source/_static/cap/mqtt_broker_list.png new file mode 100644 index 000000000..66e8cb528 Binary files /dev/null and b/docs/source/_static/cap/mqtt_broker_list.png differ diff --git a/docs/source/_static/cap/mqtt_brokers_menu.png b/docs/source/_static/cap/mqtt_brokers_menu.png new file mode 100644 index 000000000..dbbe39e64 Binary files /dev/null and b/docs/source/_static/cap/mqtt_brokers_menu.png differ diff --git a/docs/source/_static/cap/mqtt_event_list.png b/docs/source/_static/cap/mqtt_event_list.png new file mode 100644 index 000000000..2824a936d Binary files /dev/null and b/docs/source/_static/cap/mqtt_event_list.png differ diff --git a/docs/source/_static/cap/notification_bar.png b/docs/source/_static/cap/notification_bar.png new file mode 100644 index 000000000..818ebcfba Binary files /dev/null and b/docs/source/_static/cap/notification_bar.png differ diff --git a/docs/source/_static/cap/publish_cap_alert.png b/docs/source/_static/cap/publish_cap_alert.png new file mode 100644 index 000000000..d9ab4dd85 Binary files /dev/null and b/docs/source/_static/cap/publish_cap_alert.png differ diff --git a/docs/source/_static/cap/submit_dataset.png b/docs/source/_static/cap/submit_dataset.png new file mode 100644 index 000000000..914670a27 Binary files /dev/null and b/docs/source/_static/cap/submit_dataset.png differ diff --git a/docs/source/_static/cap/template_selection.png b/docs/source/_static/cap/template_selection.png new file mode 100644 index 000000000..40c79fba4 Binary files /dev/null and b/docs/source/_static/cap/template_selection.png differ diff --git a/docs/source/_static/cap/test.png b/docs/source/_static/cap/test.png new file mode 100644 index 000000000..28a8d6a0f Binary files /dev/null and b/docs/source/_static/cap/test.png differ diff --git a/docs/source/index.rst b/docs/source/index.rst index 742c22dde..340ba1db3 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -36,6 +36,7 @@ The user guide helps you setup your own wis2box instance. user/gts-headers-in-wis2 user/public-services-setup user/downloading-data + user/cap-alerts Reference guide =============== diff --git a/docs/source/reference/running/data-pipeline-plugins.rst b/docs/source/reference/running/data-pipeline-plugins.rst index 89adeef6d..00f2e849e 100644 --- a/docs/source/reference/running/data-pipeline-plugins.rst +++ b/docs/source/reference/running/data-pipeline-plugins.rst @@ -98,6 +98,8 @@ A typical BUFR4 plugin workflow definition would be defined as follows: notify: true # trigger GeoJSON publishing for API and UI file-pattern: '^.*\.bin$' +.. _cap-message-data-plugin: + ``wis2box.data.cap_message.CAPMessageData`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/docs/source/user/cap-alerts.rst b/docs/source/user/cap-alerts.rst new file mode 100644 index 000000000..3ad356098 --- /dev/null +++ b/docs/source/user/cap-alerts.rst @@ -0,0 +1,141 @@ +.. _cap-alerts: + +Connecting the CAP Composer with a wis2box +============================================ + +Overview +-------- +The Common Alerting Protocol (CAP) is a simple but general format for exchanging all-hazard emergency alerts and public warnings over all kinds of networks, see the full specification `here `_. + +This section provides guidance one how one can use the `CAP Composer `_ to automate the publishing of CAP alerts to a wis2box. + +Requirements +------------ +In addition to a running wis2box, you will need to install, configure, and run the CAP Composer. + +.. note:: + For information on how to do this, please consult the `CAP Composer documentation `_. + +Process Outline +--------------- +For automated publishing of CAP alerts created by the CAP Composer to a wis2box, we will need to perform the following steps: + +1. *Dataset Creation:* Configure the dataset in wis2box to store the CAP alerts. +2. *MQTT Configuration:* Configure your wis2box broker details in the CAP Composer. + +Dataset Creation +---------------- +Firstly, there must be a dataset in wis2box for the CAP alerts to be stored. To create a dataset, simply navigate to the 'Dataset Editor' page in wis2box-webapp, available on your host at `http:///wis2box-webapp`. + +.. note:: + For more information on how to create a dataset, please see the :ref:`adding-datasets` section of the wis2box setup guide. + +When creating a new dataset for CAP alerts, ensure that the **weather/advisories-warnings** template is selected: + +.. image:: ../_static/cap/template_selection.png + :alt: Select Template + :width: 400 + +This template will use the 'CAPMessageData' data plugin, see more information on this plugin in the :ref:`cap-message-data-plugin` section of the data pipeline plugins documentation. + +Enter the relevant missing information and create the dataset using your processes/wis2box token. + +.. image:: ../_static/cap/submit_dataset.png + :alt: Submit Dataset + :width: 800 + +Now the dataset is created, note down the dataset ID, as it will be required in the next step. + +MQTT Configuration +------------------ +Begin by logging in to the CAP Composer. + +.. note:: + We will assume that you have the necessary admin rights to configure the CAP Composer. If you do not, please contact your CAP focal point. + +Navigate to 'CAP Alerts', then 'MQTT Brokers' in the left-hand menu. + +.. image:: ../_static/cap/mqtt_brokers_menu.png + :alt: MQTT Brokers Menu + :width: 300 + +Click on the 'Add MQTT Broker' button in the top-right corner to add a new broker. You should see the following form: + +.. image:: ../_static/cap/broker_configuration.png + :alt: MQTT Broker Configuration + :width: 600 + +Here, we should fill the following fields: + +- **Name**: A name for the broker. +- **Host**: The ``WIS2BOX_BROKER_HOST`` environment variable from the wis2box configuration. +- **Port**: The ``WIS2BOX_BROKER_PORT`` environment variable from the wis2box configuration. +- **Username**: The ``WIS2BOX_BROKER_USERNAME`` environment variable from the wis2box configuration. +- **Password**: The ``WIS2BOX_BROKER_PASSWORD`` environment variable from the wis2box configuration. +- **WIS2 Node**: Confirmation that the MQTT broker is a WIS2 node (defaults to True). +- **Dataset ID**: The dataset ID of the dataset created in the previous step. + +Once you have filled in the form, click the *Save* button to save the broker configuration. + +You will be redirected back to the 'MQTT Brokers' page, where you should see the newly added broker. + +.. image:: ../_static/cap/mqtt_broker_list.png + :alt: MQTT Broker List + :width: 600 + +Publishing an Alert +------------------- +Let's begin by creating a CAP alert. This can be done by navigating to 'CAP Alerts', then 'Alerts' in the left-hand menu, and clicking the *Add Alert* button in the top-right corner. + +.. image:: ../_static/cap/alerts_menu.png + :alt: Alerts Menu + :width: 300 + +.. note:: + For more information on how to create a CAP alert, please see the `CAP Composer documentation on creating alerts `_. + +Once the alert is finished, if you are a CAP approver you should see a *Publish* button at the bottom: + +.. image:: ../_static/cap/publish_cap_alert.png + :alt: Publish Alert + :width: 300 + +On clicking the *Publish* button, the alert XML file will automatically be created, signed, and published to your wis2box. + +.. note:: + If you are a CAP composer, you will only be able to submit the alert for moderation. It is then the responsibility of the CAP approver to approve and publish the alert. + +You can view the status of the published alert in the 'CAP Alerts', then 'MQTT Broker Events' section of the CAP Composer. + +.. image:: ../_static/cap/mqtt_event_list.png + :alt: MQTT Event List + :width: 600 + +Additional diagnostic information can be found by clicking the *Inspect* button. + +Verifying Receipt of a Published Alert and Viewing the Alert +------------------------------------------------------------ +We can verify that the alert has been successfully published to wis2box by monitoring the dataset in wis2box-webapp. + +Navigate to the 'Monitoring' page in wis2box-webapp, and select the dataset that you created in the previous steps. + +.. note:: + If you do not see the dataset, ensure that the datetime range selected includes the time of the alert publication. + +.. image:: ../_static/cap/monitoring_dashboard.png + :alt: Monitoring Dashboard + :width: 800 + +Provided the publication was successful, you will see a bar in the 'Notifications' section. If you scroll down to the 'Published Data' section, the signed and verified CAP alert should appear in the table to download and view. + +.. image:: ../_static/cap/download_view_alert.png + :alt: Published Data + :width: 800 + +On clicking the *View Alert* button, you should see a visualization of the CAP alert you created earlier. + +.. image:: ../_static/cap/alert_preview.png + :alt: View Alert + :width: 800 + +Congratulations! You have successfully published a CAP alert to a wis2box using the CAP Composer. diff --git a/docs/source/user/setup.rst b/docs/source/user/setup.rst index 13e93701c..1fa41cef5 100644 --- a/docs/source/user/setup.rst +++ b/docs/source/user/setup.rst @@ -3,7 +3,7 @@ Installation and configuration ============================== -This section summarizes the steps required to install a wis2box instance and setup your own datasets using initial configuration files +This section summarizes the steps required to install a wis2box instance and setup your own datasets using initial configuration files provided by using the ``wis2box-create-config.py`` script. Ensure you have Docker, Docker Compose and Python installed on your host, as detailed in :ref:`getting-started`. @@ -32,11 +32,11 @@ Run the following command to create the initial configuration files for your wis .. note:: - The ``wis2box-create-config.py`` script will ask for a directory to store the configuration files. + The ``wis2box-create-config.py`` script will ask for a directory to store the configuration files. Please provide the **absolute** path to the directory where you want to store the configuration files, for example ``/home/wis2box-user/wis2box-data``. This directory will be mapped to ``/data/wis2box`` **inside** the wis2box-management container. - The script will also ask for the URL of your wis2box. Please provide the public URL of your wis2box, for example ``http://mywis2box.example.com``. + The script will also ask for the URL of your wis2box. Please provide the public URL of your wis2box, for example ``http://mywis2box.example.com``. For testing purpose you can also provide the internal IP address you use to access the host, for example ``http://192.168.0.3`` and you change the URL in configuration files at a later point in time. The script will propose to automatically create a password for ``WIS2BOX_WEBAPP_PASSWORD``. This password is used to access the wis2box-webapp interface. @@ -68,7 +68,7 @@ This might take a while the first time, as Docker images will be downloaded. The ``wis2box-ctl.py`` program is used as a convenience utility around a set of Docker Compose commands. You can customize the ports exposed on your host by editing ``docker-compose.override.yml``. - + .. note:: In case you get errors from the Docker daemon stating 'Permission denied', such as: @@ -76,9 +76,9 @@ This might take a while the first time, as Docker images will be downloaded. ``docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))`` Please ensure your username is added to the Docker group using the command: - + ``sudo usermod -aG docker ``. - + Logout and log back in so that your group membership is re-evaluated. @@ -109,7 +109,7 @@ Check that all services are Up and not unhealthy: wis2box-webapp sh /wis2box-webapp/ ... Up (healthy) 4173/tcp -Refer to the :ref:`troubleshooting` section if this is not the case. +Refer to the :ref:`troubleshooting` section if this is not the case. Runtime configuration @@ -162,6 +162,8 @@ The values of ``WIS2BOX_WEBAPP_USERNAME`` and ``WIS2BOX_WEBAPP_PASSWORD`` can be cat wis2box.env | grep WIS2BOX_WEBAPP +.. _adding-datasets: + Adding datasets --------------- @@ -190,8 +192,8 @@ A popup will appear where you can define your "centre-id" and the type of datase The centre-id has to be lowercase and use alphanumeric characters only. The dropdown list shows all currently registered centre-ids on WIS2 as well as any centre-id you have already created in wis2box. -There are 2 pre-defined dataset types for "weather/surface-based-observations/synop" and "weather/surface-based-observations/temp". -We recommend using these pre-defined dataset types to publish your "synop" and "temp" data, respectively. +There are multiple predefined datasets, such as "weather/surface-based-observations/synop", "weather/surface-based-observations/temp", and "weather/advisories-warnings". +We recommend using these particular predefined dataset types to publish your "synop", "temp", and CAP alert data, respectively. The predefined dataset will predefine the topic and data mappings for you. If you want to create a dataset for a different topic, you can select "other" and define the topic and data mappings yourself. @@ -207,7 +209,7 @@ Before publishing the new dataset make to click "Validate form" to check if all :alt: wis2box webapp dataset editor page, validate form Each dataset is associated with data-mappings plugins that transform the data from the input source format before the data is published. -If you are using the pre-defined dataset types for "synop" and "temp" data, the data mappings plugins will be pre-defined for you. +If you are using the predefined dataset types for "synop", "temp", or CAP alert data, the data mappings plugins will be predefined for you. Otherwise, you will need to define the data mappings plugins for your dataset. Finally, click "submit" to publish the dataset: