-
Notifications
You must be signed in to change notification settings - Fork 11
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #181 from rynge/past-materials
RESFACIL-434: re-organized past training materials
- Loading branch information
Showing
3 changed files
with
372 additions
and
345 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
369 changes: 369 additions & 0 deletions
369
documentation/support_and_training/training/materials.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,369 @@ | ||
| --- | ||
| ospool: | ||
| path: support_and_training/training/materials.md | ||
| --- | ||
|
|
||
|
|
||
| ## General Overview | ||
|
|
||
| <details> | ||
| <summary>OSPool Basics: Get Running on the OSPool</summary> | ||
| <br> | ||
| <iframe width="100%" height="315" src="https://www.youtube.com/embed/D14eMrkZ2gQ?si=BZ5MtONnRTvmRKxJ" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> | ||
| </br> | ||
| <b>Learning Objectives</b> | ||
| <br> | ||
| Topics covered in this workshop include: | ||
| <ol> | ||
| <li>An introduction to OSG services and the OSPool</li> | ||
| <li>Basics of HTCondor job submission</li> | ||
| <li>Hands-on practice submitting HTCondor jobs</li> | ||
| </ol> | ||
| <b>Prerequisites/Audience</b> | ||
| <br> | ||
| There are no prerequisites for this workshop. This workshop is designed for new HTCondor and OSPool users. | ||
| <br> | ||
| <br> | ||
| <b>Available Materials</b> | ||
| <br> | ||
| <ul> | ||
| <li> <a href="https://docs.google.com/presentation/d/1z-f81xtk_ZXeJcA1kX60JoScXdGfe-xgsB9g5YemrqI/edit#slide=id.g10662d3fe4f_0_0">Presentation Slides</a> </li> | ||
| <li> <a href="https://www.youtube.com/watch?v=D14eMrkZ2gQ">Video Recording</a></li> | ||
| <li> <a href="https://github.com/OSGConnect/tutorial-wordfreq">Wordcount Frequency Tutorial</a></li> | ||
| <li> <a href="https://github.com/OSGConnect/lesson-intro-ospool">Interactive Lesson</a> </li> | ||
| </ul> | ||
| <b>Materials Last Updated</b> | ||
| <br> | ||
| Winter 2023 | ||
| <br> | ||
| </details> | ||
|
|
||
|
|
||
| <details> | ||
| <summary>HTCondor Tips & Tricks: Using condor_q and condor_history to Learn about Your Jobs</summary> | ||
| <br> | ||
| <b>Learning Objectives</b> | ||
| <br> | ||
| This workshop is designed to introduce researchers to helpful HTCondor tools for learning about their HTCondor jobs. | ||
| <br> | ||
| <br> | ||
| <b>Prerequisites/Audience</b> | ||
| <br> | ||
| There are no prerequisites for this workshop, however, a basic understanding of HTCondor job submission and HTCondor submit files will make it easier to understand the content presented. | ||
| <br> | ||
| <br> | ||
| <b>Available Materials</b> | ||
| <br> | ||
| <ul> | ||
| <li> <a href="https://docs.google.com/presentation/d/1AWAOmtwJ68KBSJPYtp6agnB2d-2iTCG5/edit?usp=sharing&ouid=114892809919385436869&rtpof=true&sd=true">Presentation Slides</a></li> | ||
| </ul> | ||
| <b>Materials Last Updated</b> | ||
| <br> | ||
| Spring 2023 | ||
| <br> | ||
| </details> | ||
|
|
||
|
|
||
| <details> | ||
| <summary>Principles of Distributed High Throughput Computing</summary> | ||
| <br> | ||
| <b>Learning Objectives</b> | ||
| <br> | ||
| Have you ever wondered about the “why” of HTCondor? <b>Join us to hear about the “philosophy” of high throughput computing and how HTCondor has evolved to make throughput computing possible.</b> This workshop will be led by a core HTCondor developer, Greg Thain, and is a perfect opportunity for longer-term OSPool users to learn more about our underlying technology. | ||
| <br> | ||
| <br> | ||
| <b>Prerequisites/Audience</b> | ||
| <br> | ||
| There are no prerequisites for this webinar. | ||
| <br> | ||
| <br> | ||
| <b>Available Materials</b> | ||
| <br> | ||
| <ul> | ||
| <li> <a href="https://docs.google.com/presentation/d/1H1rC3aZqAB2GmoAEmwHkkY5ZOWWWpYkbsk5A01mM7GA/edit?usp=drive_link">Presentation Slides</a></li> | ||
| </ul> | ||
| <b>Materials Last Updated</b> | ||
| <br> | ||
| Spring 2024 | ||
| <br> | ||
| </details> | ||
|
|
||
|
|
||
| <details> | ||
| <summary>Access the OSPool via Jupyter Interface</summary> | ||
| <br> | ||
| <b>Learning Objectives</b> | ||
| <br> | ||
| This workshop is designed to introduce researchers to the OSPool's new Jupyter interface feature, including how to access and use Jupyter notebooks. | ||
| <br> | ||
| <br> | ||
| <b>Prerequisites/Audience</b> | ||
| <br> | ||
| There are no prerequisites for this workshop. | ||
| <br> | ||
| <br> | ||
| <b>Available Materials</b> | ||
| <br> | ||
| <ul> | ||
| <li> <a href="https://docs.google.com/presentation/d/1UCpS1iQ3tTaIIdm2OaF1kYbKaAsFz-vN/edit?usp=sharing&ouid=114892809919385436869&rtpof=true&sd=true">Presentation Slides</a></li> | ||
| </ul> | ||
| <b>Materials Last Updated</b> | ||
| <br> | ||
| Fall 2023 | ||
| <br> | ||
| </details> | ||
|
|
||
|
|
||
| <details> | ||
| <summary>Learn About the PATh Facility</summary> | ||
| <br> | ||
| <b>Learning Objectives</b> | ||
| <br> | ||
| The PATh Facility provides dedicated throughput computing capacity to NSF-funded researchers for longer and larger jobs. This training will describe its features and how to get started. <b>If you have found your jobs need more resources (cores, memory, time, data) than is typically available in the OSPool, this resource might be for you!</b> | ||
| <br> | ||
| <br> | ||
| <b>Prerequisites/Audience</b> | ||
| <br> | ||
| There are no prerequisites for this webinar. | ||
| <br> | ||
| <br> | ||
| <b>Available Materials</b> | ||
| <br> | ||
| <ul> | ||
| <li> <a href="https://docs.google.com/presentation/d/10Axe-Y1Mh_-fZrRXOjhvaOVTHybbHdW39PXxydb8-Qs/edit#slide=id.g10c0fd09133_0_7">Presentation Slides</a></li> | ||
| </ul> | ||
| <b>Materials Last Updated</b> | ||
| <br> | ||
| Winter 2023 | ||
| <br> | ||
| </details> | ||
|
|
||
|
|
||
| ## Data | ||
|
|
||
| <details> | ||
| <summary>Move Your Data with Pelican (and the OSDF)</summary> | ||
| <br> | ||
| <iframe width="100%" height="315" src="https://www.youtube.com/embed/KFg5ApsIGN8?si=ohRY8X_0lWmYZ8DJ" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> | ||
| <br> | ||
| <b>Learning Objectives</b> | ||
| <br> | ||
| Pelican is a platform created to enable easier data sharing - within or beyond your institution! This training will cover how Pelican is used to move data within the OSPool and also how you can use Pelican tools to host, upload and download your data. This training is relevant for researchers with large amounts of data, as well as campus representatives, to learn about how Pelican can help with your data movement needs. | ||
| <br> | ||
| <br> | ||
| <b>Prerequisites/Audience</b> | ||
| <br> | ||
| There are no prerequisites for this webinar. | ||
| <br> | ||
| <br> | ||
| <b>Available Materials</b> | ||
| <br> | ||
| <ul> | ||
| <li> <a href="https://docs.google.com/presentation/d/1G9lpiWXErEqsSXxGzKYq9iz-lmoUiNL5K4h9bK9sy8c/edit#slide=id.g10c0fd09133_0_7">Presentation Slides</a></li> | ||
| <li> <a href="https://www.youtube.com/watch?v=KFg5ApsIGN8">Video Recording</a></li> | ||
| </ul> | ||
| <b>Materials Last Updated</b> | ||
| <br> | ||
| Summer 2024 | ||
| <br> | ||
| </details> | ||
|
|
||
|
|
||
| ## Software | ||
|
|
||
| <details> | ||
| <summary>Software Portability on the Open Science Pool</summary> | ||
| <br> | ||
| <b>Learning Objectives</b> | ||
| <br> | ||
| This workshop is designed to introduce concepts pertaining to software portability, including containers, different ways to install software, setting file paths, and other important introductory concepts. | ||
| <br> | ||
| <br> | ||
| <b>Prerequisites/Audience</b> | ||
| <br> | ||
| There are no prerequisites for this workshop, however, a basic understanding of HTCondor job submission and HTCondor submit files will make it easier to understand the content presented. | ||
| <br> | ||
| <br> | ||
| <b>Available Materials</b> | ||
| <br> | ||
| <ul> | ||
| <li> <a href="https://docs.google.com/presentation/d/1Ym5piDoEkDPpInnyYgeqUkV-VWQoBTkI0RXJNG7_LK4/edit#slide=id.g120009588c4_0_0">Presentation Slides</a></li> | ||
| <li> <a href="https://docs.google.com/document/d/1-lfpv7AO7cLZ2yiZQIuiQbb1lyBqh2CB5Ix51cKZ1bk/edit#">List of Commands</a></li> | ||
| <li> Tutorials (used in part) | ||
| <ul> | ||
| <li> <a href="https://github.com/OSGConnect/tutorial-julia">Using Julia on the OSPool</a></li> | ||
| <li> <a href="https://github.com/OSGConnect/tutorial-bwa">High Throughput BWA Read Mapping</a></li> | ||
| </ul> | ||
| </li> | ||
| </ul> | ||
| <b>Materials Last Updated</b> | ||
| <br> | ||
| Summer 2023 | ||
| <br> | ||
| </details> | ||
|
|
||
| <details> | ||
| <summary>Using Containerized Software on the Open Science Pool</summary> | ||
| <br> | ||
| <iframe width="100%" height="315" src="https://www.youtube.com/embed/vu5EZccxgdo?si=AdY6smlO1xiLicCr" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> | ||
| <br> | ||
| <b>Learning Objectives</b> | ||
| <br> | ||
| This workshop is designed to introduce software containers such as Docker, Apptainer, and Singularity. Content covered includes how to create a container, use a container, and techniques for troubleshooting containerized software. | ||
| <br> | ||
| <br> | ||
| <b>Prerequisites/Audience</b> | ||
| <br> | ||
| There are no prerequisites for this workshop, however, a basic understanding of HTCondor job submission and HTCondor submit files will make it easier to understand the content presented. | ||
| <br> | ||
| <br> | ||
| <b>Available Materials</b> | ||
| <br> | ||
| <ul> | ||
| <li> <a href="https://docs.google.com/presentation/d/15jhYzlfzyTaTTEylTW8rD728_GLOSIp5pd6wL3eZBZk/edit?usp=sharing">Presentation Slides</a> </li> | ||
| <li> <a href="https://www.youtube.com/watch?v=vu5EZccxgdo&list=PLIeB7asuU4W5FuqFdyqThCf18vTd_1tfu&index=5&ab_channel=OSG">Video Recording</a></li> | ||
| </ul> | ||
| <b>Materials Last Updated</b> | ||
| <br> | ||
| Fall 2023 | ||
| <br> | ||
| </details> | ||
|
|
||
|
|
||
| <details> | ||
| <summary>Bioinformatics Analyses on the OSPool: A BWA Example</summary> | ||
| <br> | ||
| <b>Learning Objectives</b> | ||
| <br> | ||
| This workshop is designed to show the process of implementing and scaling out a bioinformatics workflow using HTCondor. | ||
| <br> | ||
| <br> | ||
| <b>Prerequisites/Audience</b> | ||
| <br> | ||
| There are no prerequisites for this workshop, however, a basic understanding of HTCondor job submission and HTCondor submit files will make it easier to understand the content presented. | ||
| <br> | ||
| <br> | ||
| <b>Available Materials</b> | ||
| <br> | ||
| <ul> | ||
| <li> <a href="https://docs.google.com/presentation/d/1_6uI7qG2nHHOynal8tinrHy2MCTkTbac/edit?usp=sharing&ouid=114892809919385436869&rtpof=true&sd=true">Presentation Slides</a></li> | ||
| </ul> | ||
| <b>Materials Last Updated</b> | ||
| <br> | ||
| Summer 2023 | ||
| <br> | ||
| </details> | ||
|
|
||
|
|
||
|
|
||
| ## Workflows / Advanced Workloads | ||
|
|
||
| <details> | ||
| <summary>Organizing and Submitting HTC Workloads</summary> | ||
| <br> | ||
| <iframe width="100%" height="315" src="https://www.youtube.com/embed/0JQ8or3lue0?si=mU1VVmUEoEhaz3-4" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> | ||
| <br> | ||
| <b>Learning Objectives</b> | ||
| <br> | ||
| This workshop will present useful HTCondor features to help researchers automatically organize their workspaces on High Throughput Computing systems. | ||
| <br> | ||
| <br> | ||
| <b>Prerequisites/Audience</b> | ||
| <br> | ||
| There are no prerequisites for this workshop, however, a basic understanding of HTCondor job submission and HTCondor submit files will make it easier to understand the content presented. | ||
| <br> | ||
| <br> | ||
| <b>Available Materials</b> | ||
| <br> | ||
| <ul> | ||
| <li> <a href="https://docs.google.com/presentation/d/1auCJfqzc0ZckeTr-QCILaDizsKPsPTli0wTCbluSswY">Presentation Slides</a></li> | ||
| <li> <a href="https://www.youtube.com/watch?v=0JQ8or3lue0&list=PLIeB7asuU4W5FuqFdyqThCf18vTd_1tfu&index=6&ab_channel=OSG">Video Recording</a></li> | ||
| <li> <a href="https://github.com/OSGConnect/tutorial-organizing">Wordcount Frequency Tutorial</a></li> | ||
| </ul> | ||
| <b>Materials Last Updated</b> | ||
| <br> | ||
| Summer 2023 | ||
| <br> | ||
| </details> | ||
|
|
||
|
|
||
| <details> | ||
| <summary>DAGMan: HTCondor's Workflow Manager</summary> | ||
| <br> | ||
| <b>Learning Objectives</b> | ||
| <br> | ||
| Presented by an HTCondor DAGMan developer, this workshop is designed for researchers that would like to learn how to implement DAG workflows and automate workflow management on the OSPool. | ||
| <br> | ||
| <br> | ||
| <b>Prerequisites/Audience</b> | ||
| <br> | ||
| A basic understanding of HTCondor job submission and of an HTCondor submit file is highly recommended for this workshop. | ||
| <br> | ||
| <br> | ||
| <b>Available Materials</b> | ||
| <br> | ||
| <ul> | ||
| <li> <a href="https://docs.google.com/presentation/d/1FbTo_18LL03D41v6DYOfOOooNAy06Vzm/edit?usp=drive_web&ouid=114892809919385436869&rtpof=true">Presentation Slides</a></li> | ||
| <li> <a href="https://github.com/OSGConnect/tutorial-dagman-intro">DAGMan Tutorial</a></li> | ||
| </ul> | ||
| <b>Materials Last Updated</b> | ||
| <br> | ||
| Winter 2023 | ||
| <br> | ||
| </details> | ||
|
|
||
|
|
||
| <details> | ||
| <summary>Pegasus Workflow Management System on the Open Science Pool</summary> | ||
| <br> | ||
| <iframe width="100%" height="315" src="https://www.youtube.com/embed/rCDrPBki7ps?si=9XVzYvfiSBbc8TQV" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> | ||
| <br> | ||
| <b>Learning Objectives</b> | ||
| <br> | ||
| This workshop is designed to introduce Pegasus Workflow Management System, a useful tool for researchers needing to execute a large number of jobs or complex workflows. | ||
| <br> | ||
| <br> | ||
| <b>Prerequisites/Audience</b> | ||
| <br> | ||
| There are no prerequisites for this workshop, however, a basic understanding of HTCondor job submission and HTCondor submit files will make it easier to understand the content presented. | ||
| <br> | ||
| <br> | ||
| <b>Available Materials</b> | ||
| <br> | ||
| <ul> | ||
| <li> <a href="https://docs.google.com/presentation/d/1XQjkq8pxbrXvMYU7zDihkgk2bOy4FNXHINWkhmkNqHQ/edit?usp=sharing">Presentation Slides</a></li> | ||
| <li> <a href="https://www.youtube.com/watch?v=rCDrPBki7ps">Recorded Video</a></li> | ||
| </ul> | ||
| <b>Materials Last Updated</b> | ||
| <br> | ||
| Fall 2023 | ||
| <br> | ||
| </details> | ||
|
|
||
|
|
||
|
|
||
| ## Special Environments | ||
|
|
||
| <details> | ||
| <summary>Special Environments, GPUs</summary> | ||
| <br> | ||
| <b>Learning Objectives</b> | ||
| <br> | ||
| This workshop is designed for researchers interested in learning about using special environments, architectures, or resources such as GPUs. | ||
| <br> | ||
| <br> | ||
| <b>Prerequisites/Audience</b> | ||
| <br> | ||
| There are no prerequisites for this workshop, however, a basic understanding of HTCondor job submission and HTCondor submit files will make it easier to understand the content presented. | ||
| <br> | ||
| <br> | ||
| <b>Available Materials</b> | ||
| <br> | ||
| <ul> | ||
| <li> <a href="https://docs.google.com/presentation/d/1QZnX-VeXHdqWcmgF8EiMac_nl4hHXPvxCvCH-vxBySU/edit?usp=sharing">Presentation Slides</a></li> | ||
| </ul> | ||
| <b>Materials Last Updated</b> | ||
| <br> | ||
| Spring 2023 | ||
| <br> | ||
| </details> |
Oops, something went wrong.