1. Getting Started

In this section, we will discuss the jarvis command line interface. We will discuss the basic steps of how to create a deployment pipeline.

We will create a pipeline which does the following:

1. Launch OrangeFS
2. Launch Hermes
3. Intercept POSIX I/O with Hermes
4. Launch the IOR benchmark

1.01. Build Resource Graph

First we have to collect information about the system.

jarvis resource-graph create \
  --hostfile=hostfile.txt

This information will be stored under config/resource_graph.yaml. The command uses SSH to connect to all nodes.

1.02. Create an empty pipeline

The following command will create an empty pipeline "my_pipeline":

jarvis create my_pipeline

After creating a pipeline, jarvis will focus on that pipeline.

1.03. Load Modules + Build Environment

Next, we must make jarvis aware of all environment variables needed to execute applications in the pipeline.

module load hermes
module load orangefs
jarvis build-env

NOTE: this is machine-dependent. You would have to have these modules installed on your machine before-hand.

NOTE: LD_PRELOAD should NOT be one of the environment variables set here. Jarvis has specific "Interceptor" nodes for handling LD_PRELOAD.

1.04. Add nodes to the pipeline

To add a node to the pipeline:

jarvis append orangefs
jarvis append hermes
jarvis append hermes-posix
jarvis append ior

Nodes can be configured while appended. For example,

jarvis append orangefs \
  server.pvfs2_port=33436 \
  sleep=10

This will make the OrangeFS node use port 33436 and sleep for 10 seconds to give it some time to launch.

1.05. Start a pipeline

To start the service for the configured pipeline, do:

jarvis start

1.06. Get the status of the pipeline

To check whether the pipeline is functioning, do:

jarvis status

1.07. Stop a pipeline

To stop the pipeline's execution on all nodes, do:

jarvis stop

1.08. Clean a pipeline

Pipelines can create a bunch of data. For example, OrangeFS may contain data leftover from a benchmark. To destroy this data, run:

jarvis clean

1.09. Re-configuring an existing node

There may be cases where you need to reconfigure a node in the pipeline. To do this, run configure:

jarvis configure orangefs \
  server.pvfs2_port=33436

1.10. Unlinking and Removing Nodes

• Unlinking a node means Jarvis will remove the node from the pipeline, without destroying the node's metadata. This will allow it to be re-appended in the future.
• Removing a node means Jarvis will remove the node from the pipeline AND destroy its metadata.

jarvis unlink [node_id]
jarvis remove [node_id]

1.11. Changing to a different pipeline

To make jarvis start, end, etc. apply to a different pipeline, use jarvis cd

jarvis cd [pipeline_name]

1.12. Destroying a pipeline

To destroy a pipeline, its nodes, and all metadata associated with it:

jarvis destroy [pipeline_name]

1.13. Listing Existing Pipelines

To list all existing pipelines, run:

jarvis list