-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
15 changed files
with
172 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
/.quarto/ | ||
/_site/ | ||
/.vscode/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
project: | ||
type: website | ||
|
||
website: | ||
title: "Peter Mac Nextflow Workshop" | ||
navbar: | ||
left: | ||
- href: index.qmd | ||
text: Home | ||
- href: workshops/00_setup.qmd | ||
text: Setup | ||
- text: Workshop | ||
menu: | ||
- text: "Introduction to Nextflow" | ||
href: workshops/1.1_intro_nextflow.html | ||
- text: "Introduction to nf-core" | ||
href: workshops/1.2_intro_nf_core.html | ||
- text: "Customising & running nf-core pipelines" | ||
href: workshops/2.1_customise_and_run.html | ||
- text: "Troubleshooting Nextflow run" | ||
href: workshops/2.2_troubleshooting.html | ||
- text: "Best practise, tips and tricks" | ||
href: workshops/2.3_tips_and_tricks.html | ||
|
||
format: | ||
html: | ||
theme: cosmo | ||
css: styles.css | ||
toc: true | ||
|
||
|
||
|
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
--- | ||
title: "Introduction to Nextflow and running nf-core workflows" | ||
--- | ||
|
||
This workshop is designed to provide participants with a foundational understanding of Nextflow and nf-core pipelines, with a focus on running existing pipelines efficiently. Participants are expected to have prior experience with the command-line interface and working with cluster systems like Slurm. The primary goal of the workshop is to equip researchers with the skills needed to use nf-core pipelines for their research data. | ||
|
||
*It is not intended for those looking to develop new Nextflow workflows* | ||
|
||
### Trainers | ||
|
||
- Richard Lupat, Bioinformatics Core Facility | ||
- Miriam Yeung, Cancer Genomics Translational Research Centre | ||
- Sanduni Rajapaksa, Research Computing Facility | ||
- Henrietta Holze, Dawson Lab | ||
|
||
### Prerequisites | ||
|
||
- Experience with command line interface and cluster/slurm | ||
- Familiarity with the basic concept of workflows | ||
- Access to Peter Mac Cluster | ||
|
||
### Learning Objectives: | ||
By the end of this workshop, participants should be able to: | ||
|
||
- Gain exposure to key concepts and terminology in Nextflow and nf-core pipelines. | ||
- Understand the foundational knowledge required to navigate and customize the code base of nf-core pipelines. | ||
- Develop basic troubleshooting and customization skills necessary for responsibly applying nf-core pipelines to your own research data. | ||
|
||
### Set up requirements | ||
Please complete the [Setup Instructions](./workshops/00_setup.html) before the course. | ||
|
||
If you have any trouble, please get in contact with us ASAP via Slack/Teams. | ||
|
||
### Workshop schedule | ||
|
||
| Lesson | Overview | Time | | ||
|------------|----------| ---------- | | ||
| [Setup](./workshops/00_setup.html)| Follow these instructions to install VS Code and setup your workspace | Prior to workshop | | ||
| Session kick off| Session kick off: Discuss learning outcomes and finalising workspace setup | 09:30 - 09:40 | | ||
| [Introduction to Nextflow](./workshops/1.1_intro_nextflow.html)| Introduction to Nextflow: Introduce nextflow’s core features and concepts; including CLI and how to run it on Peter Mac cluster | 09:40 - 10:20 | | ||
| [Introduction to nf-core](./workshops/1.2_intro_nf_core.html)| Introduction to nf-core: Introduce nf-core features and concepts, structures, tools, and example nf-core pipelines | 10:20 - 11:00 | | ||
| Break | Break | 11:00 - 11:15 | | ||
| [Customising & running nf-core pipelines](./workshops/2.1_customise_and_run.html)| Customising & running nf-core pipelines: Discuss pipelines’ required inputs, optional inputs, outputs, parameters file and configurations files | 11:15 - 11:45 | | ||
| [Troubleshooting nextflow run](./workshops/2.2_troubleshooting.html)| Troubleshooting nextflow run: Discuss Nextflow logging, caching, task execution directory, dependencies, and manual troubleshooting | 11:45 - 12:15 | | ||
| [Best practise and Q&A](./workshops/2.3_tips_and_tricks.html)| Best practise, tips & tricks for running nextflow pipelines at Peter Mac cluster | 12:15 - 12:45 | | ||
|
||
### Credits and acknowledgement | ||
|
||
^*This workshop is adapted from [Customising Nf-Core Workshop](https://sydney-informatics-hub.github.io/customising-nfcore-workshop) materials from Sydney Informatics Hub*^ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
/* css styles */ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,87 @@ | ||
--- | ||
title: "**Essential Workshop Preparation**" | ||
output: | ||
html_document: | ||
toc: false | ||
toc_float: false | ||
from: markdown+emoji | ||
--- | ||
|
||
In this workshop, we will be using Peter Mac's HPC to run nextflow and nf-core workflows. | ||
|
||
Before joining the workshop, please complete the following checklist: | ||
|
||
- Install Visual Studio Code (recommended) or Terminal application | ||
- Install the Nextflow and Remote-SSH VSC extensions (below) for VSCode user | ||
- Test your remote connection to Peter Mac HPC; and create a directory `/scratch/users/<your-username>/nfWorkshop` in your scratch space. This will be our working directory | ||
|
||
## Option 1: Install and set up Visual Studio Code | ||
|
||
We recommend Visual Studio Code as a source code editor because it is lightweight and has rich support for extensions and syntax highlighting available across various popular operating system. | ||
|
||
Download [Visual Studio Code](https://code.visualstudio.com/download) on your computer and follow the instructions for your specific Operating System as required: | ||
|
||
- [macOS](https://code.visualstudio.com/docs/setup/mac) | ||
- [Windows](https://code.visualstudio.com/docs/setup/windows) | ||
- [Linux](https://code.visualstudio.com/docs/setup/linux) | ||
|
||
Once installed, open VS Code on your computer. | ||
|
||
![](media/00_0_VSCode.png) | ||
|
||
### Install the Nextflow Extension | ||
|
||
The Nextflow extension provides syntax highlighting and quick shortcuts for common code snippets. | ||
|
||
Click on the extensions button (four blocks) on the left side bar. | ||
Search for "Nextflow" in the extensions search bar, then click on the blue "Install" button. | ||
|
||
![](media/00_1_Nextflow_ext.png) | ||
|
||
### Install the Remote-SSH Extension | ||
|
||
Remote-SSH allows us to use any remote machine with a SSH server as your development environment. This lets us work directly on the our cluster's storage. | ||
|
||
Click on the extensions button (four blocks) on the left side bar. | ||
Search for "Remote - SSH" in the extensions search bar, then click on the blue "Install" button. | ||
|
||
![](media/00_2_RemoteSSH_ext.png) | ||
|
||
### Login via Visual Studio Code | ||
|
||
Connect to your instance with VS code by adding the host details to your .ssh config file (if you have not done this previously) | ||
|
||
1. In a new VS code window, type Ctrl+Shift+P if you’re on a Windows machine or Cmd+Shift+P for MacOS to open the command palette | ||
2. Select Remote-SSH: Open SSH configuration file and select your .ssh config file | ||
3. Add a new entry with your details to login to cluster, and save your .ssh config file: | ||
|
||
```default | ||
Host pmac-cluster | ||
HostName kvmpr-res-lnode1.unix.petermac.org.au | ||
User <your-cluster-user-name> | ||
``` | ||
|
||
4. Type Ctrl+Shift+P and select Remote-SSH: Connect to Host; and pmac-cluster (or whatever you name your host above) | ||
5. When prompted, select Linux as the platform of the remote host from the dropdown menu | ||
6. Type in your password and hit enter | ||
|
||
Having successfully logged in, you should see a small blue or green box in the bottom left corner of your screen: | ||
|
||
![](media/00_3_RemoteSSH_Connected.png) | ||
|
||
To set up your VS Code window for the workshop: | ||
|
||
1. Open a new folder in the file explorer panel on the left side of the screen by typing Ctrl + K, Ctrl + O if you’re running Windows or Cmd+K+ Cmd + O for MacOS | ||
2. Select `/scratch/users/<your-username>/nfWorkshop` to open our working directory. If you encountered an error that the directory does not exist, you would need to ssh in to the cluster and create that directory first before attempting this step. | ||
3. When prompted, select the box for Trust the authors of all files in the parent folder ‘home’ then click Yes, I trust the authors | ||
4. You can dismiss the warning message saying our cluster's git version is outdated | ||
5. To open a terminal, type Ctrl+J if you’re on a Windows machine or Cmd+J on MacOS | ||
|
||
## Option 2: Terminal | ||
|
||
No additional setup required. | ||
SSH to cluster as usual, and we assume you are already familiar with command line if you decided to go with this option. :smile: | ||
|
||
|
||
--- | ||
^*This setup instruction is adapted from [Customising Nf-Core Workshop](https://sydney-informatics-hub.github.io/customising-nfcore-workshop/setup.html) materials from Sydney Informatics Hub*^ |
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.