Skip to content

Commit

Permalink
fix readme
Browse files Browse the repository at this point in the history
  • Loading branch information
chrisjsewell committed Feb 27, 2019
1 parent 6d43edd commit bc32bb2
Show file tree
Hide file tree
Showing 2 changed files with 39 additions and 31 deletions.
57 changes: 31 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@ atomic-hpc

[![travis](https://travis-ci.org/chrisjsewell/atomic-hpc.svg?branch=master)](https://travis-ci.org/chrisjsewell/atomic-hpc)
[![coveralls](https://coveralls.io/repos/github/chrisjsewell/atomic-hpc/badge.svg?branch=master)](https://coveralls.io/github/chrisjsewell/atomic-hpc?branch=master)
[![codacy](https://api.codacy.com/project/badge/Grade/e0b541be3f834f12b77c712433ee64c9)](https://www.codacy.com/app/chrisj_sewell/atomic-hpc?utm_source=github.com&utm_medium=referral&utm_content=chrisjsewell/atomic-hpc&utm_campaign=Badge_Grade)
[![PyPI](https://img.shields.io/pypi/v/atomic-hpc.svg)](https://pypi.python.org/pypi/atomic-hpc/)

<!-- [![codacy](https://api.codacy.com/project/badge/Grade/e0b541be3f834f12b77c712433ee64c9)](https://www.codacy.com/app/chrisj_sewell/atomic-hpc?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=chrisjsewell/atomic-hpc&amp;utm_campaign=Badge_Grade) -->

**Project**: <https://github.com/chrisjsewell/atomic-hpc>

A package for running multiple executable scripts on both local and remote hosts,
Expand Down Expand Up @@ -33,38 +34,39 @@ Minimal Example
1. Write a yaml configuration file; each run must have a name and a unique id,
then attributes can be set in the (global) `defaults` section or per run (run attributes will overwrite defaults):

**config.yaml**:
```yaml
defaults:
environment: unix

process:
unix:
run:
- echo "hallo world" > hallo.txt
output:
path: output
runs:
- id: 1
name: test_local
- id: 2
name: test_other

```
**config.yaml**:

```yaml
defaults:
environment: unix

process:
unix:
run:
- echo "hallo world" > hallo.txt

output:
path: output
runs:
- id: 1
name: test_local
- id: 2
name: test_other

```

2. Submit it with the command line app (use -h to see all options):

>> run_config config.yaml

3. The results will be available in the output path, with one folder per run:

>> ls -R output
output/1_test_local:
config_1.yaml hallo.txt
output/2_test_other:
config_2.yaml hallo.txt


Minimal Example (Remote and PBS)
--------------------------------
Expand All @@ -73,6 +75,7 @@ Jobs can be submitted to remote hosts and, optionally,
[PBS](https://en.wikipedia.org/wiki/Portable_Batch_System) type systems.

**config_remote.yaml**

```yaml
runs:
- id: 1
Expand Down Expand Up @@ -119,6 +122,7 @@ This is file 1
```

**config.yaml**:

```yaml
defaults:
description: quantum-espresso run
Expand All @@ -135,7 +139,7 @@ defaults:
file1: path/to/input.txt
scripts:
- path/to/script1.in
process:
unix:
run:
Expand All @@ -156,7 +160,8 @@ runs:
```

**Run**:
```

```console
>> run_config config.yaml
>> ls -R output
output/1_run1:
Expand All @@ -182,7 +187,7 @@ process:
start_in_temp: true
run:
- my_program > @{wrkpath}/output.log
```
```

Outputs
-------
Expand Down Expand Up @@ -321,7 +326,7 @@ If you open 'id_rsa.pub' it should contain one line of, similar to:
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwRDgM+iQg7OaX/CFq1sZ9jl206nYIhW9SMBqsOIRvGM68/6o6uxZo/D4IlmQI9sAcU5FVNEt9dvDanRqUlC7ZtcOGOCqZsj1HTGD3LcOiPNHYPvi1auEwrXv1hDh4pmJwdgZCRnpewNl+I6RNBiZUyzLzp0/2eIyf4TqG1rpHRNjmtS9turANIv1GK1ONIO7RfVmmIk/jjTQJU9iJqje9ZSXTSm7rUG4W8q+mWcnACReVChc+9mVZDOb3gUZV1Vs8e7G36nj6XfHw51y1B1lrlnPQJ7U3JdqPz6AG3Je39cR1vnfALxBSpF5QbTHTJOX5ke+sNKo//kDyWWlfzz3rQ== [email protected]

Now log in to the HPC and open (or create) the file '~/.ssh/authorized_keys'.
In a new line at the end of this file, you should add a comment (starting with #) about where that keypair comes from
In a new line at the end of this file, you should add a comment (starting with #) about where that keypair comes from
and then in a second line you should copy and paste the complete contents of your 'id_rsa.pub' file.

#MAC in the office
Expand Down
13 changes: 8 additions & 5 deletions atomic_hpc/context_folder/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,8 @@ def __init__(self, path='.', remote=False, hostname='', **kwargs):
os.makedirs(path)
#raise IOError("the path does not exist: {}".format(path))
if not os.path.isdir(path):
raise IOError("the path is not a directory: {}".format(path))
raise IOError(
"the path is not a directory: {}".format(path))
abspath = os.path.expanduser(os.path.abspath(path))
self._path = pathlib.Path(abspath)
else:
Expand All @@ -73,11 +74,12 @@ def __init__(self, path='.', remote=False, hostname='', **kwargs):
try:
self._ssh.connect(self._hostname, **self._kwargs)
_ = self._ssh.open_sftp()
#sftp.chdir(self._path)
# sftp.chdir(self._path)
self._ssh.close()
except Exception as err:
raise RuntimeError("failed connecting to {0} with args: {1}\n{2}".format(
self._hostname, self._kwargs, err))
raise RuntimeError(
"failed connecting to {0} with args: {1}\n{2}".format(
self._hostname, self._kwargs, err))

def __enter__(self):

Expand All @@ -86,7 +88,8 @@ def __enter__(self):
return LocalPath(self._path)
else:
logger.debug("entering remote path")
return RemotePath(self._ssh, self._hostname, self._path, **self._kwargs)
return RemotePath(
self._ssh, self._hostname, self._path, **self._kwargs)

def __exit__(self, etype, value, traceback):
logger.debug("exiting path")
Expand Down

0 comments on commit bc32bb2

Please sign in to comment.