Skip to content

Commit

Permalink
Updated the website for 2025's course
Browse files Browse the repository at this point in the history
  • Loading branch information
JamieMair committed Jan 14, 2025
1 parent 9945a70 commit b7c53f3
Show file tree
Hide file tree
Showing 5 changed files with 57 additions and 38 deletions.
21 changes: 21 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,24 @@
# High Performance Computing in Julia

This is the main site for documentation about he High Performance Computing in Julia MPAGS module. The link to the site can be found [here](https://jamiemair.github.io/mpags-high-performance-computing/).


## Locally running the website

Make sure you have `julia` installed on your system and open up a terminal in this directory:
```bash
julia
```
Then run the following in the REPL:
```julia
using Pkg;
Pkg.activate(".");
Pkg.instantiate();
```
This only needs to be done once. After you can run the project locally with
```julia
using Franklin
serve()
```

This will open up the live server.
17 changes: 6 additions & 11 deletions overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
@def tags = ["syntax", "code"]


# High Performance Computing in Julia, from the ground up.
# High Performance Computing in Julia

This module is designed for teaching programming to new Masters/PhD students, starting from the basics and working up to writing high performance code for any scale. This module will primarily be delivered via the Midlands Physics Alliance Graduate School ([MPAGS](https://warwick.ac.uk/fac/sci/physics/mpags/modules/)). The module is open to anyone else who is interested in the material. All of the resources (including the lecture notes) will be available publicly. Get in touch with me via [email](mailto:[email protected]) if you would like to enroll, and receive updates about the course.

Expand All @@ -16,27 +16,22 @@ Students taking this course should match following description:
No prior knowledge about the Julia programming language is required as this course will provide a crash course to get everyone up to speed at the start of the module. Topics learnt during this course are **not** based solely around Julia, but can be transferred to other languages.

# Topics
This module will focus on teaching students how to write high performance code, taking advantage of modern hardware, to be able to execute code at any scale, from your own machine with multiple cores, to using a GPU, all the way to executing code across an entire High Performance Cluster. We will also touch on Research Software Engineering techniques throughout the module.

This module will focus on teaching students how to write high performance code, taking advantage of modern hardware, to be able to execute code at any scale, from your own machine with multiple cores, to using a GPU, all the way to executing code across an entire High Performance Cluster.

See the [Topics](/topics/) page for a detailed overview of the topics covered.

# Module Convenor

This course is taught by me, **Jamie Mair**, a Research & Teaching Fellow at the University of Nottingham. You can contact me about the course via [email](mailto:[email protected]).
This course is taught by me, **Dr. Jamie Mair**, a Research & Teaching Fellow at the University of Nottingham. You can contact me about the course via [email](mailto:[email protected]).

# Timetable and format

This course will run in the spring term (**January 2023**), starting on Tuesday 17th January. The format of this course will be a 2 1-hour sessions each week on Tuesday and Thursday, being taught at the University of Nottingham in the Physics building, and livestreamed on Teams (link will be sent round via email). Rooms for each session will be posted on the [Timetable](/timetable/) page.

The Tuesday session will be a lecture covering the theoretical material. The Thursday session will be a hybrid lecture/workshop to give you the opportunity to practice what is taught in the lectures through the assignments.

The bulk of the reference material will be provided in the format of a book/lecture notes which is complementary to the course and gives more depth. These notes should also be suitable for students who would like to follow along and do the assignments but cannot attend the timetabled sessions (either online or in-person).
This course will be a self-study module, run in **January 2025**, starting in the w/c Monday 13th January. Each week a series of online videos will be posted, along with accompanying notes online. To be notified of each week's materials, as well as recieving links to the assessment materials, please [email](mailto:[email protected]) me directly to sign up. At the end of the course, once all materials have been posted, links to them will be posted here.

# Assessment

Each week of teaching will have a corresponding assessment delivered via GitHub classroom, with automatic feedback/marking. These problems are usually short and will be started in the workshop and hopefully finished during the workshop on Thursdays. Some assignments may take more time outside the workshop.

More details can be found on the [Resources](/resources/) page.
Each week of teaching will have a corresponding assessment delivered via GitHub classroom, with automatic feedback/marking. For students taking the MPAGS course, links to these assessments will be provided **via email**, so make sure you are on the mailing list. For other students who are not taking the MPAGS module, links to the GitHub templates will be posted here.

# Citation

Expand Down
24 changes: 15 additions & 9 deletions resources.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

## Lecture Notes

The lecture notes are available [here](/assets/book_v1.2.pdf). They are a work in progress and may change.
The original lecture notes from 2023 are available [here](/assets/book_v1.2.pdf). The new course notes will be posted online on a different website, with entries posted weekly to accompany the video material. Links to the book are kept here for reference.

### Changelog
**Current Version: v1.2 (23/01/2023)**
Expand All @@ -27,22 +27,28 @@ For this course, you do **not** need to be an expert with Git, but its use is en

## Assignments

1. Week 1 - Getting Started with Julia: Workflow and Setup - [click here](https://classroom.github.com/a/3bYk2x83)
2. Week 2 - Profiling and Optimisation - [click here](https://classroom.github.com/a/HFbbhcO1)
3. Week 3 - Multithreading - [click here](https://classroom.github.com/a/HqKUZUwc)
4. Week 4 - Multiprocessing - [click here](https://classroom.github.com/a/y9jTWq3J)
5. Week 5 - GPU Programming - [click here](https://classroom.github.com/a/q9ycWkI6)
1. Week 1 - Getting Started with Julia: Workflow and Setup - TBA
2. Week 2 - Profiling and Optimisation - TBA
3. Week 3 - Multithreading - TBA
4. Week 4 - Multiprocessing - TBA
5. Week 5 - GPU Programming - TBA


## Computing Resources

During this course, you should be able to do everything on your own personal machine. I would advise trying to run some code on your local cluster as well if you have access. However, not everybody has access to a GPU, and specifically during part of the course, you will need access to a modern NVIDIA GPU. If you have access to your local institution's HPC already, but do not have GPU access, you can ask your local admin to give you a small number of GPU hours (around 20 should be more than enough).
During this course, you should be able to do everything on your own personal machine. I would advise trying to run some code on your local cluster as well if you have access. You will benefit from being able to access a GPU, but it is not a requirement for this course. Being able to test and run your GPU code locally will make it easier to complete the final assessment, but it can be done without one. If you have access to your local institution's HPC already, but do not have GPU access, you can ask your local admin to give you a small number of GPU hours (around 20 should be more than enough).

Another option is to use Google Colab which has free access to GPU resources for testing. There is also a cheap paid option for better access. The [notebook linked here](https://colab.research.google.com/github/ageron/julia_notebooks/blob/master/Julia_Colab_Notebook_Template.ipynb) has instructions for how to setup Julia and CUDA. This is the easiest way to get started.
Another option is to use Google Colab which has free access to GPU resources for testing. There is also a cheap paid option for better access. The [notebook linked here](https://colab.research.google.com/github/ageron/julia_notebooks/blob/master/Julia_Colab_Notebook_Template.ipynb) has instructions for how to setup Julia and CUDA, however this notebook may not work for everyone.

If you are from UoN and are struggling to access GPUs, contact [myself](mailto:[email protected]) to see if we can arrange temporary access to our in-house Physics cluster. Access to this cluster **cannot** be guaranteed, and we advise students to use this as a last resort.

## Lecture Slides & Recordings (2023)
## Videos

*No videos for 2025 have been posted yet. Sign up to the mailing list to recieve updates, or check back later to find new videos.*

## Lecture Slides & Recordings Archive (2023)

Below is an **archive** of the original lecture videos and accompanying slides. The new videos in 2025 will largely follow the same content, with a few sections expanded upon and others removed. In particular, these recordings do not include the in-person workshops that were ran in 2025.

### (1/10) Hardware & Software Basics
[Lecture Slides](/assets/slides/session_1.pdf)
Expand Down
16 changes: 3 additions & 13 deletions timetable.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,8 @@
@def title = "Timetable"
@def tags = ["syntax", "code"]

# Course Schedule 2022/23
# Course Schedule 2025

The provisional starting date of this course is in January 2023, w/c 16th. Lectures will happen on **Tuesdays 4-5pm** UK time and workshops will happen **4-5pm** on **Thursdays**.
The provisional starting date of this course is in January 2025, w/c Monday 13th. The content for this course will be posted weekly online in the form of one or more videos, with accompanying online notes. Each week, one assessement will be released via GitHub classroom. The due date for all assessments will be at the beginning of March 2025.

**Note**: **All** sessions will be in B17 Physics - University Park Campus, UoN.

| Week | Lecture | Workshop |
|------|----------------------------------------------|----------------------------------------------|
| 1 | **17/01/23** 16:00-17:00 **Physics B17 UoN** | **19/01/23** 16:00-17:00 **Physics B17 UoN** |
| 2 | **24/01/23** 16:00-17:00 **Physics B17 UoN** | **26/01/23** 16:00-17:00 **Physics B17 UoN** |
| 3 | **31/01/23** 16:00-17:00 **Physics B17 UoN** | **02/02/23** 16:00-17:00 **Physics B17 UoN** |
| 4 | **07/02/23** 16:00-17:00 **Physics B17 UoN** | **09/02/23** 16:00-17:00 **Physics B17 UoN** |
| 5 | **14/02/23** 16:00-17:00 **Physics B17 UoN** | **16/02/23** 16:00-17:00 **Physics B17 UoN** |

Sessions will be in person at the University of Nottingham in the Physics building.
Links to the resources will be sent out in an email to the mailing list. Once the course is finished, all materials will be available online.
17 changes: 12 additions & 5 deletions topics.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,26 +5,33 @@

\toc

## Introduction to Software and Hardware
## Computing Foundations

This module will start with a description of modern hardware and software, as many students do not come from a Computer Science background. We aim to teach the student the basic architecture of a modern system, describing in detail parts such as the CPU, memory and co-processors like Graphics Processing Units (GPUs). We do this to ground future discussions of optimisations with a physical model of the computing system one is optimising for. Additionally, we will dive into the basics of software on a computing system, discussing what an operating system is, and how source code is transformed into machine code which can run on the system via the use of compilers. Using the knowledge of this section we will cover the taxonomy of the features of modern programming languages, and their relation to high performance code.

## Julia Crash Course

As we are not expecting any students taking this course to have experience with the Julia programming language, we will cover a quick crash course into the syntax of the language and give students a mental model of how the language functions so that they can understand the code examples throughout the module. This course is grounded in practical examples, given and assessed using Julia. While the concepts in this module are certainly applicable to other languages, Julia has been chosen for this course as it has the lowest barrier to entry for writing high performance, parallel code (including GPU programming).

## Measuring Performance

Optimisation is fairly useless without the ability to measure the impact of your changes. Benchmarking forms a core part of high performance computing, and this section will cover how to efficiently measure the run-time of your code, while we begin to understand what makes code run slowly. We will cover basic benchmarking techniques, such as timing, as well as more advanced techniques, such as profiling, to understand where to focus our time and effort when optimising. Additionally, we will touch on the theory of computational complexity to explain how the performance of an algorithm scales with the problem size.

## Optimising Serial Code
Here, we will focus on trying to get a given algorithm to execute as quickly, or efficiently, as possible on some given hardware. We will touch on lower-level optimisation details such as memory locality, stack vs heap memory and non-branching programming. The aim of this section is to write simple, yet fast, code to run as quickly as possible. We will also talk about Julia specific techniques to increase performance (or rather, avoid bad performance).

## Professional Research Software Engineering
Almost every area of research is increasingly relying on computational techniques to push their fields forward, through large data processing, simulation or machine learning techniques. Often times, techniques yielded from experience with software engineering in industry are often neglected during research as they aren't seen as a high priority. However, software engineering has many best practices and techniques used to produce **high quality** and **error-free** code, without much, if any, loss in productivity. This part of the module aims to convey the advantages of using techniques such as version control, unit testing and documentation to increase the quality of your research, and provide a base to push the field forwards at an accelerated pace. Along with the techniques mentioned, we will also have a large focus on reproducibility and open science, particularly applicable to researchers. These techniques will be emphasised throughout the module and students will get a chance to practice them during the assignments.
Here, we will focus on trying to get a given algorithm to execute as quickly, or efficiently, as possible on some given hardware. We will touch on lower-level optimisation details such as memory locality, stack vs heap memory and non-branching programming. The aim of this section is to write simple, yet fast, code to run as quickly as possible. We will also talk about Julia specific techniques to increase performance (or rather, avoid bad performance).

## Introduction to Parallel Programming
Here, we will introduce the theory of parallel computing and provide a framework to analyse the ability of an algorithm to be accelerated using parallel processing via the use of Directed Acyclical Graphs (DAGs). The theory introduced in this section will help frame the practical applications of parallel computing to any given task, while also giving theoretical devices to aid parallelising algorithms via the use of *maps* and *reductions*. On the practical side of things, we will talk about **race conditions** which are common pitfalls new students make when doing parallel computing.

## Multithreading & Multiprocessing
Multithreading is the easiest and most common paradigm one can use to parallelise one's code. We will start by discussing the advantages and disadvantages of using multithreading, as well as giving many practical examples of how to use multithreading in your code via the discussion of an *embarrassingly-parallel* problem. Next, we will discuss multiprocessing, which can be used to scale up parallel processing across an entire cluster. We will give examples of how multiprocessing can be achieved in Julia and show examples of how to scale up execution to an entire High Performance Cluster.

Multithreading is the easiest and most common paradigm one can use to parallelise one's code. We will start by discussing the advantages and disadvantages of using multithreading, as well as giving many practical examples of how to use multithreading in your code via the discussion of an *embarrassingly-parallel* problem. Next, we will discuss multiprocessing, which can be used to scale up parallel processing across an entire cluster. We will give examples of how multiprocessing can be achieved in Julia and set the scene for running code on a high performance supercomputing cluster.

## Cluster Computing

We will introduce a few ways of deploying code on a cluster, including basic multithreading, multiprocessing and even dive into the basics of MPI (Message Passing Interface) programs. For this, we will give examples of launching code on SLURM backed clusters.

## GPGPU Programming

The end of the module will centre around *General Purpose Graphics Processing Unit* (GPGPU) programming. GPUs are incredible devices with the ability to perform tens of thousands of operations in parallel. Programming for these devices has had a notoriously high barrier to entry until only recently. Julia provides users the ability to write native Julia code that can be executed directly on the GPU, increasing the flexibility and re-useability of the code for execution on many different devices. In this section, we will talk about the theory of GPU computing and give students an idea of what workloads can be accelerated by using a GPU, along with the limitations. Further, we will give students the opportunity to use the GPU via high-level abstractions and even write their own GPU kernels (programs that run on the GPU), all in Julia.

0 comments on commit b7c53f3

Please sign in to comment.