From 97113ae03631544ea97fae85dee65a4a44610402 Mon Sep 17 00:00:00 2001 From: Marcos Bento Date: Tue, 18 Jun 2024 12:42:30 +0100 Subject: [PATCH] Add Aviso+Mirror how-to instructions Re ECFLOW-1931 --- .../how_to_mirror_a_remote_task_status.rst | 70 +++++++++++++++++++ ...ger_a_task_based_on_aviso_notification.rst | 65 +++++++++++++++++ docs/ug/cookbook/index.rst | 5 +- 3 files changed, 137 insertions(+), 3 deletions(-) create mode 100644 docs/ug/cookbook/how_to_mirror_a_remote_task_status.rst create mode 100644 docs/ug/cookbook/how_to_trigger_a_task_based_on_aviso_notification.rst diff --git a/docs/ug/cookbook/how_to_mirror_a_remote_task_status.rst b/docs/ug/cookbook/how_to_mirror_a_remote_task_status.rst new file mode 100644 index 000000000..b389be5c2 --- /dev/null +++ b/docs/ug/cookbook/how_to_mirror_a_remote_task_status.rst @@ -0,0 +1,70 @@ +.. _how_to_mirror_a_remote_task_status: + +How to mirror a remote Task status? +----------------------------------- + +The following instructions describe the necessary steps to use the ability to +synchronise the status of a :term:`task` between a `remote` ecFlow server +and a `local` ecFlow server. The `remote` server is responsible for actually +executing the task, while the `local` server synchronizes the status of a +particular task, in order to trigger the execution of dependent tasks. + +The deployment of this feature has the following requirements: + - `local` ecFlow 5.13+, to host the dependent task(s) + - `remote` ecFlow 5+, to host the remote task + - Authentication credentials, stored as JSON file + +Setup the ecFlow Server +^^^^^^^^^^^^^^^^^^^^^^^ + +Deploy the `remote` ecFlow server, as per the :ref:`regular instructions` +and specifying username/password access configuration. + +Deploy the Authentication credentials so that the JSON file is accessible +to the `local` ecFlow server. Launch the server. + +Consider using SSL encription for the communication between ecFlow server and +client. + + +Define the Suite with a `mirrored` Task +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +On the :term:`suite` definition file, create a :term:`task` (which will mirror +the remote task status) and assign it a :term:`mirror` attribute. The mirror +attribute must specify the path of the remote node, and the remote server +connection details. Using the mirrored task, other tasks can use triggers to +execute when the remote task becomes complete. + +Follow the recommended practice of defining ecFlow :term:`variables` +at :term:`suite` level: + + - ECF_MIRROR_REMOTE_HOST + - ECF_MIRROR_REMOTE_PORT + - ECF_MIRROR_REMOTE_POLLING + - ECF_MIRROR_REMOTE_AUTH + +**Example 1**: Example suite definition with mirror attributes + + .. code-block:: shell + + suite s + edit ECF_MIRROR_REMOTE_HOST 'remotehost' + edit ECF_MIRROR_REMOTE_PORT '3456' + edit ECF_MIRROR_REMOTE_POLLING '120' + edit ECF_MIRROR_REMOTE_AUTH '/path/to/mirror.auth' + family f + task Task + mirror --name A --remote_path /s1/f1/t1 --remote_host %ECF_MIRROR_REMOTE_HOST% --remote_port %ECF_MIRROR_REMOTE_PORT% --polling %ECF_MIRROR_REMOTE_POLLING% --ssl + task Dependent + trigger Task == complete + endfamily + endsuite + +Deploy the Suite with a `mirrored` Task +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Load the suite definition containing the :term:`mirror` attribute. + +Whenever the ecFlow server is in `RUNNING` state, the synchronisation between +`remote` and `local` servers is enabled automatically. diff --git a/docs/ug/cookbook/how_to_trigger_a_task_based_on_aviso_notification.rst b/docs/ug/cookbook/how_to_trigger_a_task_based_on_aviso_notification.rst new file mode 100644 index 000000000..b8628284e --- /dev/null +++ b/docs/ug/cookbook/how_to_trigger_a_task_based_on_aviso_notification.rst @@ -0,0 +1,65 @@ +.. _how_to_trigger_a_task_based_on_aviso_notification: + +How to trigger a Task based on Aviso notification? +-------------------------------------------------- + +The following instructions describe the necessary steps to use the ability to +trigger a :term:`task` based on an Aviso notification. The intended task must +be assigned an :term:`aviso` attribute, which provides the configuration to +periodically poll the Aviso server, and execution only happens when the +a matching Aviso notification is processed. + +The deployment of this feature has the following requirements: + - ecFlow 5.13+, to host the Aviso dependent task(s) + - Aviso event listener schema + - Authentication credentials, stored as JSON file + +Setup the ecFlow Server +^^^^^^^^^^^^^^^^^^^^^^^ + +Deploy the Authentication credentials and Aviso event listener schema files so +that these files are accessible to the ecFlow server. Launch the server, as per +the :ref:`regular instructions`. + +Define a Suite with an `Aviso` dependent Task +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +On the :term:`suite` definition file, create a :term:`task` and assign it an +:term:`aviso` attribute. + +The :term:`aviso` attribute must specify the base URL to contact the Aviso server, +and the remote server connection details. Using the mirrored task, other tasks can use triggers to execute +when the remote task becomes complete. + +Follow the recommended practice of defining ecFlow :term:`variables` at :term:`suite` level: + + - ECF_AVISO_URL + - ECF_AVISO_SCHEMA + - ECF_AVISO_POLLING + - ECF_AVISO_AUTH + +**Example 1**: Example suite definition with mirror attributes + + .. code-block:: shell + + suite s + edit ECF_AVISO_URL 'http://aviso:2379' + edit ECF_AVISO_SCHEMA '/path/to/event_listener_schema.json' + edit ECF_AVISO_POLLING '120' + edit ECF_AVISO_AUTH '/path/to/aviso.auth' + family f + task Task + aviso --name A --listener '{ "event": "mars", "request": { "step": 0 } }' --url %ECF_AVISO_URL% --schema %ECF_AVISO_SCHEMA% --auth %ECF_AVISO_AUTH% --polling %ECF_AVISO_POLLING% + endfamily + endsuite + +Define a Suite with an `Aviso` dependent Task +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Load the suite definition containing the :term:`aviso` attribute. + +Whenever a task assigned with an :term:`aviso` attribute is (re)queued, +the Aviso will start to be periodically poll. The task is held from execution +until a notification matching the provided listener is found. +Once the task is allowed to execute the periodic polling is stopped, until the +task is requeued again. diff --git a/docs/ug/cookbook/index.rst b/docs/ug/cookbook/index.rst index d5d9d45b9..62eb2b6f3 100644 --- a/docs/ug/cookbook/index.rst +++ b/docs/ug/cookbook/index.rst @@ -7,7 +7,6 @@ Cookbook ============ - This cookbook will provide real examples of using the ecFlow :ref:`python_api`. **Contents** @@ -38,5 +37,5 @@ This cookbook will provide real examples of using the ecFlow :ref:`python_api`. native_python_task one_script_do_it_all python_client_for_fault_tolerance - - + how_to_mirror_a_remote_task_status + how_to_trigger_a_task_based_on_aviso_notification