Skip to content

Commit

Permalink
Website publish
Browse files Browse the repository at this point in the history
  • Loading branch information
arcalot-bot committed Nov 15, 2023
1 parent 50cfa9f commit 93c1955
Show file tree
Hide file tree
Showing 6 changed files with 1,795 additions and 40 deletions.
6 changes: 5 additions & 1 deletion arcaflow/workflows/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -1645,7 +1645,11 @@


<h1 id="creating-arcaflow-workflows">Creating Arcaflow workflows</h1>
<p>Arcaflow workflows consist of three parts:</p>
<p>Arcaflow workflows consist of four parts:</p>
<h2>Version</h2>

<p>The schema version must be at the root of your workflow file. It indicates the semantic version of the workflow file structure being used.</p>
<p><a class="md-button" href="versioning/">Learn more about versioning &raquo;</a></p>
<h2>Inputs</h2>

<p>The input section of a workflow is much like a plugin schema: it describes the data model of the workflow itself. This is useful because the input can be validated ahead of time. Any input data can then be referenced by the individual plugin steps.</p>
Expand Down
12 changes: 11 additions & 1 deletion arcaflow/workflows/output/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -1698,7 +1698,17 @@ <h2 id="writing-multiple-outputs">Writing multiple outputs</h2>
<p>Arcaflow can produce multiple output groups for a workflow. These output groups are mutually exclusive to each other.</p>
<p>A common example of two mutually exclusive events could be the availability of your data storage service. Let&rsquo;s assume the service is either available, or unavailable (the unavailble state also includes any states where an error is thrown during data insertion). Multiple workflow outputs allows you to plan for these two events.</p>
<p>In this example taken from the <a href="https://github.com/arcalot/arcaflow-workflows/blob/main/example-workflow/workflow.yaml">Arcaflow Workflows</a> project, the <code>success</code> output collects the data from the specified steps and inserts it into data storage. The <code>no-indexing</code> output collects the data, the error logs, and does not store the data.</p>
<div class="highlight"><pre><span></span><code>
<div class="highlight"><pre><span></span><code><span class="nt">outputs</span><span class="p">:</span>
<span class="w"> </span><span class="nt">success</span><span class="p">:</span>
<span class="w"> </span><span class="nt">pcp</span><span class="p">:</span><span class="w"> </span><span class="kt">!expr</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">$.steps.pcp.outputs.success</span>
<span class="w"> </span><span class="nt">sysbench</span><span class="p">:</span><span class="w"> </span><span class="kt">!expr</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">$.steps.sysbench.outputs.success</span>
<span class="w"> </span><span class="nt">metadata</span><span class="p">:</span><span class="w"> </span><span class="kt">!expr</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">$.steps.metadata.outputs.success</span>
<span class="w"> </span><span class="nt">opensearch</span><span class="p">:</span><span class="w"> </span><span class="kt">!expr</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">$.steps.opensearch.outputs.success</span>
<span class="w"> </span><span class="nt">no-indexing</span><span class="p">:</span>
<span class="w"> </span><span class="nt">pcp</span><span class="p">:</span><span class="w"> </span><span class="kt">!expr</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">$.steps.pcp.outputs.success</span>
<span class="w"> </span><span class="nt">sysbench</span><span class="p">:</span><span class="w"> </span><span class="kt">!expr</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">$.steps.sysbench.outputs.success</span>
<span class="w"> </span><span class="nt">metadata</span><span class="p">:</span><span class="w"> </span><span class="kt">!expr</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">$.steps.metadata.outputs.success</span>
<span class="w"> </span><span class="nt">no-index</span><span class="p">:</span><span class="w"> </span><span class="kt">!expr</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">$.steps.opensearch.outputs.error</span>
</code></pre></div>


Expand Down
Loading

0 comments on commit 93c1955

Please sign in to comment.