Skip to content

Commit

Permalink
Website publish
Browse files Browse the repository at this point in the history
  • Loading branch information
arcalot-bot committed Apr 30, 2024
1 parent 9edb6bc commit e8401c9
Show file tree
Hide file tree
Showing 37 changed files with 58 additions and 58 deletions.
4 changes: 2 additions & 2 deletions arcaflow/concepts/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down Expand Up @@ -1588,7 +1588,7 @@ <h2>Typing</h2>
<p><a class="md-button" href="typing/">Read more about typing &raquo;</a></p>
<h2>Plugins</h2>

<p>Arcaflow is interoperable between several programming languages. Currently we provide SDKs for Python and Go.</p>
<p>Arcaflow is interoperable between several programming languages. Currently, we provide SDKs for Python and Go.</p>
<p><a class="md-button" href="plugins/">Read more about plugins &raquo;</a></p>
<h2>Workflows</h2>

Expand Down
2 changes: 1 addition & 1 deletion arcaflow/concepts/plugins/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down
2 changes: 1 addition & 1 deletion arcaflow/concepts/typing/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down
2 changes: 1 addition & 1 deletion arcaflow/concepts/workflows/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down
2 changes: 1 addition & 1 deletion arcaflow/contributing/deployers/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down
2 changes: 1 addition & 1 deletion arcaflow/contributing/engine/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@


<link rel="icon" href="../../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down
2 changes: 1 addition & 1 deletion arcaflow/contributing/expressions/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down
4 changes: 2 additions & 2 deletions arcaflow/contributing/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down Expand Up @@ -1633,7 +1633,7 @@ <h1 id="contributing-to-arcaflow">Contributing to Arcaflow</h1>
<h2 id="license">License</h2>
<p>All code in Arcaflow is licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache 2.0 license</a>. The documentation is licensed under <a href="https://creativecommons.org/licenses/by/4.0/">CC-BY-4.0</a>. Please make sure you read and understand these licenses before contributing. If you are contributing on behalf of your employer, please make sure you have permission to do so.</p>
<h2 id="principles">Principles</h2>
<p>While we don&rsquo;t deal in absolutes (only a sith would do that) we hold ourselves to a few key principles. There are plenty of things where we could do better in these areas, so if you find something, please open an issue. It&rsquo;s important!</p>
<p>While we don&rsquo;t deal in absolutes (only a Sith would do that) we hold ourselves to a few key principles. There are plenty of things where we could do better in these areas, so if you find something, please open an issue. It&rsquo;s important!</p>
<h3 id="the-principle-of-the-least-surprise">The principle of the least surprise</h3>
<p>Sometimes, things are just hard to make user-friendly. If presented with two choices, we will always pick the one that doesn&rsquo;t break expectations. What would an average user expect to happen without reading the documentation? If something surprised you, <strong>please open a bug</strong>.</p>
<h3 id="the-principle-of-nice-error-messages">The principle of nice error messages</h3>
Expand Down
2 changes: 1 addition & 1 deletion arcaflow/contributing/plugin-protocol/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down
6 changes: 3 additions & 3 deletions arcaflow/contributing/typing/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down Expand Up @@ -1748,7 +1748,7 @@
<h1 id="the-arcaflow-type-system">The Arcaflow type system</h1>
<p>Arcaflow takes a departure from the classic run-and-pray approach of running workloads and validates workflows before executing them. To do this, Arcaflow starts the plugins as needed before the workflow is run and queries them for their <strong>schema</strong>. This schema will contain information about what kind of input a plugin requests and what kind of outputs it can produce.</p>
<p>A plugin can support multiple <strong>workflow steps</strong> and must provide information about the data types in its <strong>input and output</strong> for each step. A step can have exactly one input format, but may declare more than one output.</p>
<p>The typesystem is inspired by <a href="https://json-schema.org/">JSON schema</a> and <a href="https://swagger.io/specification/">OpenAPI</a>, but it is more restrictive due to the need to efficiently serialize workloads over various formats.</p>
<p>The type system is inspired by <a href="https://json-schema.org/">JSON schema</a> and <a href="https://swagger.io/specification/">OpenAPI</a>, but it is more restrictive due to the need to efficiently serialize workloads over various formats.</p>
<h2 id="types">Types</h2>
<p>The typing system supports the following data types.</p>
<ul>
Expand All @@ -1774,7 +1774,7 @@ <h3 id="ints-floats">Ints, floats</h3>
<h2 id="booleans">Booleans</h2>
<p>Boolean types can take a value of either <code>true</code> or <code>false</code>, but when unserializing from YAML or JSON formats, strings or int values of <code>true</code>, <code>yes</code>, <code>on</code>, <code>enable</code>, <code>enabled</code>, <code>1</code>, <code>false</code>, <code>no</code>, <code>off</code>, <code>disable</code>, <code>disabled</code> or <code>0</code> are also accepted.</p>
<h3 id="lists-maps">Lists, maps</h3>
<p>Lists a7nd maps can have constraints on the minimum or maximum number of items in them (inclusive).</p>
<p>Lists and maps can have constraints on the minimum or maximum number of items in them (inclusive).</p>
<h3 id="objects">Objects</h3>
<p>Object fields can have several constraints:</p>
<ul>
Expand Down
6 changes: 3 additions & 3 deletions arcaflow/getting-started/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down Expand Up @@ -1779,7 +1779,7 @@ <h2 id="step-4-build-the-plugin">Step 4: Build the plugin</h2>
</div>
<div class="admonition tip">
<p class="admonition-title">Did you know?</p>
<p>While Arcaflow is a workflow engine, plugins can be run independentily via the command line. Try running your containerized helo-world plugin directly.</p>
<p>While Arcaflow is a workflow engine, plugins can be run independently via the command line. Try running your containerized hello-world plugin directly.</p>
<div class="tabbed-set tabbed-alternate" data-tabs="2:2"><input checked="checked" id="__tabbed_2_1" name="__tabbed_2" type="radio" /><input id="__tabbed_2_2" name="__tabbed_2" type="radio" /><div class="tabbed-labels"><label for="__tabbed_2_1">Docker</label><label for="__tabbed_2_2">Podman</label></div>
<div class="tabbed-content">
<div class="tabbed-block">
Expand Down Expand Up @@ -1924,7 +1924,7 @@ <h2 id="step-7-run-the-workflow">Step 7: Run the workflow</h2>
<h2 id="next-steps">Next steps</h2>
<p>Congratulations, you are now an Arcaflow user! Here are some things you can do next to start working with plugins and workflows:</p>
<ul>
<li><a href="https://github.com/orgs/arcalot/repositories?q=arcaflow-plugin&amp;type=all&amp;language=&amp;sort=">See our repostories of community-supported plugins &raquo;</a></li>
<li><a href="https://github.com/orgs/arcalot/repositories?q=arcaflow-plugin&amp;type=all&amp;language=&amp;sort=">See our repositories of community-supported plugins &raquo;</a></li>
<li><a href="https://quay.io/arcalot">Get our latest plugin container builds from quay.io &raquo;</a></li>
<li><a href="https://github.com/arcalot/arcaflow-workflows">Experiment with more advanced example workflows &raquo;</a></li>
</ul>
Expand Down
6 changes: 3 additions & 3 deletions arcaflow/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down Expand Up @@ -1602,15 +1602,15 @@ <h2>Shipping expertise</h2>
<p>Arcaflow&rsquo;s design can drastically simplify much of the workflow creation process, and it allows the workflow author to ensure the workflow is <em>locked in end-to-end</em>. <strong>A complete workflow can be version-controlled as a simple YAML file and in most cases can be expected to run in exactly the same way in any compatible environment.</strong></p>
<h2>Not a CI system</h2>

<p>Arcaflow is <strong>not designed to run as a persistent service nor to record workflow histories</strong>, and in most cases it is <strong>probably not the best tool to setup or manage infrastructure</strong>. For end-to-end CI needs, you should leverage a system that provdes these and other features (possibly something from the Alternatives list below).</p>
<p>Arcaflow is <strong>not designed to run as a persistent service nor to record workflow histories</strong>, and in most cases it is <strong>probably not the best tool to set up or manage infrastructure</strong>. For end-to-end CI needs, you should leverage a system that provides these and other features (possibly something from the Alternatives list below).</p>
<p>Arcaflow is, however, <strong>an excellent companion to a CI system</strong>. In many cases, building complex workflows completely within a CI environment can effectively lock you into that system because the workflow may not be easily portable outside of it or run independently by a user. An Arcaflow workflow <strong>can be easily integrated into most CI systems</strong>, so a workflow that you define once may be moved in most cases without modification to different environments or run directly by users.</p>
<h2>Alternatives</h2>

<p>It&rsquo;s important that you pick the right tool for the job. Sometimes, you need something simple. Sometimes, you want something persistent that keeps track of the workflows you run over time. We have collected some common and well-known open source workflow and workflow-like engines into this list and have provided some comparisons to help you find the right tool for your needs.</p>
<p><strong>Here are some of the features that make Arcaflow a unique solution to the below alternatives:</strong></p>
<ul>
<li>Designed for complex branching-action workflows and parallelization</li>
<li>Prioritizes data passing and management via strong typing and schemas to ensure machine readabilitly, workflow validation, and data integrity</li>
<li>Prioritizes data passing and management via strong typing and schemas to ensure machine readability, workflow validation, and data integrity</li>
<li>Runs actions as plugins via container orchestrator APIs</li>
<li>Engine is deployed as a single Golang binary, and plugins are run as containers, minimizing dependencies and maximizing portability</li>
<li>Workflows are designed to be explicitly version controlled to ensure portability to other environments without code or feature drift</li>
Expand Down
6 changes: 3 additions & 3 deletions arcaflow/plugins/go/first/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down Expand Up @@ -1673,7 +1673,7 @@


<h1 id="writing-your-first-go-plugin">Writing your first Go plugin</h1>
<p>In order to create a Go plugin, you will need to create a Go module project (<code>go mod init</code>) and install the Arcaflow SDK usin <code>go get go.flow.arcalot.io/pluginsdk</code>.</p>
<p>In order to create a Go plugin, you will need to create a Go module project (<code>go mod init</code>) and install the Arcaflow SDK using <code>go get go.flow.arcalot.io/pluginsdk</code>.</p>
<p>Writing a Go plugin consists of the following 4 parts:</p>
<ol>
<li>The input data model</li>
Expand All @@ -1692,7 +1692,7 @@ <h2 id="the-input-data-model">The input data model</h2>
<p>The Arcaflow serialization does not use the built-in Go JSON marshaling, so any additional tags like <code>omitempty</code>, or <code>yaml</code> tags are ignored.</p>
</div>
<p>In addition to the struct above, we must also define a schema for the input data structure:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// We define a separate scope, so we can add subobjects later.</span>
<div class="highlight"><pre><span></span><code><span class="c1">// We define a separate scope, so we can add sub-objects later.</span>
<span class="kd">var</span><span class="w"> </span><span class="nx">inputSchema</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="nx">schema</span><span class="p">.</span><span class="nx">NewScopeSchema</span><span class="p">(</span>
<span class="w"> </span><span class="c1">// Struct-mapped object schemas are object definitions that are mapped to a specific struct (Input)</span>
<span class="w"> </span><span class="nx">schema</span><span class="p">.</span><span class="nx">NewStructMappedObjectSchema</span><span class="p">[</span><span class="nx">Input</span><span class="p">](</span>
Expand Down
2 changes: 1 addition & 1 deletion arcaflow/plugins/go/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down
2 changes: 1 addition & 1 deletion arcaflow/plugins/go/schema/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down
2 changes: 1 addition & 1 deletion arcaflow/plugins/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down
2 changes: 1 addition & 1 deletion arcaflow/plugins/packaging/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down
6 changes: 3 additions & 3 deletions arcaflow/plugins/python/data-model/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down Expand Up @@ -1792,7 +1792,7 @@ <h1 id="creating-a-python-data-model">Creating a Python data model</h1>
</ul>
<p>You can read more about the individual types in the <a href="#data-types">data types</a> section</p>
<h2 id="optional-parameters">Optional parameters</h2>
<p>You can also declare any parameter as optional like this:</p>
<p>You can also declare any parameter as optional, like this:</p>
<div class="highlight"><span class="filename">plugin.py</span><pre><span></span><code><span class="nd">@dataclasses</span><span class="o">.</span><span class="n">dataclass</span>
<span class="k">class</span> <span class="nc">MyClass</span><span class="p">:</span>
<span class="n">param</span><span class="p">:</span> <span class="n">typing</span><span class="o">.</span><span class="n">Optional</span><span class="p">[</span><span class="nb">int</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span>
Expand Down Expand Up @@ -1903,7 +1903,7 @@ <h3 id="enums">Enums</h3>
In the above example, the Value1 member of MyEnum is accessed and assigned to the variable value.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Enumeration members are &ldquo;singleton&rdquo; objects, which have a single instance. In Python you should compare enumeration members using <code>is</code> rather than <code>==</code> (for example, <code>variable is MyEnum.Value1</code>). The values of an Enum used in an Arcaflow schema must have values of string or integer data type.</p>
<p>Enumeration members are &ldquo;singleton&rdquo; objects which have a single instance. In Python, you should compare enumeration members using <code>is</code> rather than <code>==</code> (for example, <code>variable is MyEnum.Value1</code>). The values of an Enum used in an Arcaflow schema must have values of string or integer data type.</p>
</div>
<div class="admonition tip">
<p class="admonition-title">Tip</p>
Expand Down
2 changes: 1 addition & 1 deletion arcaflow/plugins/python/embedding/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down
2 changes: 1 addition & 1 deletion arcaflow/plugins/python/faq/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down
6 changes: 3 additions & 3 deletions arcaflow/plugins/python/first/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@


<link rel="icon" href="../../../../arcalot.svg">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.19">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.20">



Expand Down Expand Up @@ -1756,7 +1756,7 @@ <h2 id="step-2-creating-an-input-and-output-data-model">Step 2: Creating an inpu
<div class="highlight"><span class="filename">plugin.py</span><pre><span></span><code><span class="k">def</span> <span class="nf">hello_world</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="k">return</span> <span class="sa">f</span><span class="s2">&quot;Hello, </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">&quot;</span>
</code></pre></div>
<p>However, <strong>that&rsquo;s not how the Arcaflow SDK works</strong>. You must always specify the <em>data type</em> of any variable. Additionally, every function can only have <strong>one input</strong> and it must be a <strong>dataclass</strong>.</p>
<p>However, <strong>that&rsquo;s not how the Arcaflow SDK works</strong>. You must always specify the <em>data type</em> of any variable. Additionally, every function can only have <strong>one input</strong>, and it must be a <strong>dataclass</strong>.</p>
<p>So, let&rsquo;s change the code a little:</p>
<div class="highlight"><span class="filename">plugin.py</span><pre><span></span><code><span class="kn">import</span> <span class="nn">dataclasses</span>

Expand All @@ -1768,7 +1768,7 @@ <h2 id="step-2-creating-an-input-and-output-data-model">Step 2: Creating an inpu
<span class="k">def</span> <span class="nf">hello_world</span><span class="p">(</span><span class="n">params</span><span class="p">:</span> <span class="n">InputParams</span><span class="p">):</span>
<span class="c1"># ...</span>
</code></pre></div>
<p>So far so good, but we are not done yet. The output also has special rules. One plugin function can have more than one possible output, so you need to say which output it is and you need to also return a dataclass.</p>
<p>So far so good, but we are not done yet. The output also has special rules. One plugin function can have more than one possible output, so you need to say which output it is, and you need to also return a dataclass.</p>
<p>For example:</p>
<div class="highlight"><span class="filename">plugin.py</span><pre><span></span><code><span class="kn">import</span> <span class="nn">dataclasses</span>

Expand Down
Loading

0 comments on commit e8401c9

Please sign in to comment.