Skip to content

Commit

Permalink
Merge pull request #981 from buildtesters/facility_page_improvement
Browse files Browse the repository at this point in the history
Update to facility page documentation
  • Loading branch information
shahzebsiddiqui authored Jan 14, 2022
2 parents 989c1f6 + b3ef6a5 commit 5e2e501
Show file tree
Hide file tree
Showing 2 changed files with 578 additions and 78 deletions.
76 changes: 1 addition & 75 deletions docs/batch_support.rst
Original file line number Diff line number Diff line change
Expand Up @@ -636,78 +636,4 @@ buildtest will terminate after run stage.
└─────────┴──────────┴───────┴──────────┴─────────┘
Cancelled Jobs: [pbs_hold_job/7cbe643c]
Unable to run any tests
.. _cray_burstbuffer_datawarp:

Cray Burst Buffer & Data Warp
-------------------------------

For Cray systems, you may want to stage-in or stage-out into your burst buffer this
can be configured using the ``#DW`` directive. For a list of data warp examples see
section on `DataWarp Job Script Commands <https://pubs.cray.com/bundle/XC_Series_DataWarp_User_Guide_CLE60UP01_S-2558_include_only_UP01/page/DataWarp_Job_Script_Commands.html>`_

In buildtest we support properties ``BB`` and ``DW`` which is a list of job directives
that get inserted as **#BW** and **#DW** into the test script. To demonstrate let's start
off with an example where we create a persistent burst buffer named ``databuffer`` of size
10GB striped. We access the burst buffer using the `DW` directive. Finally we
cd into the databuffer and write a 5GB random file.

.. Note:: BB and DW directives are generated after scheduler directives. The ``#BB``
comes before ``#DW``. buildtest will automatically add the directive **#BB**
and **#DW** when using properties BB and DW

.. code-block:: yaml
:emphasize-lines: 13-16
:linenos:
version: "1.0"
buildspecs:
create_burst_buffer:
type: script
executor: cori.slurm.debug
batch:
nodecount: "1"
timelimit: "5"
cpucount: "1"
sbatch: ["-C knl"]
description: Create a burst buffer
tags: [jobs]
BB:
- create_persistent name=databuffer capacity=10GB access_mode=striped type=scratch
DW:
- persistentdw name=databuffer
run: |
cd $DW_PERSISTENT_STRIPED_databuffer
pwd
dd if=/dev/urandom of=random.txt bs=1G count=5 iflags=fullblock
ls -lh $DW_PERSISTENT_STRIPED_databuffer/
Next we run this test and inspect the generated test we will see that ``#BB`` and ``#DW`` directives
are inserted after the scheduler directives

.. code-block:: shell
:emphasize-lines: 8-9
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --time=5
#SBATCH --ntasks=1
#SBATCH --job-name=create_burst_buffer
#SBATCH --output=create_burst_buffer.out
#SBATCH --error=create_burst_buffer.err
#BB create_persistent name=databuffer capacity=10GB access_mode=striped type=scratch
#DW persistentdw name=databuffer
cd $DW_PERSISTENT_STRIPED_databuffer
pwd
dd if=/dev/urandom of=random.txt bs=1G count=5 iflag=fullblock
ls -lh $DW_PERSISTENT_STRIPED_databuffer
We can confirm their is an active burst buffer by running the following

.. code-block:: console
$ scontrol show burst | grep databuffer
Name=databuffer CreateTime=2020-10-29T13:06:21 Pool=wlm_pool Size=20624MiB State=allocated UserID=siddiq90(92503)
Unable to run any tests
Loading

0 comments on commit 5e2e501

Please sign in to comment.