diff --git a/previews/PR53/.documenter-siteinfo.json b/previews/PR53/.documenter-siteinfo.json index 4a530b03..f820e9f6 100644 --- a/previews/PR53/.documenter-siteinfo.json +++ b/previews/PR53/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.5","generation_timestamp":"2024-09-24T01:30:20","documenter_version":"1.7.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.5","generation_timestamp":"2024-09-24T07:00:04","documenter_version":"1.7.0"}} \ No newline at end of file diff --git a/previews/PR53/aws_gpu_setup/index.html b/previews/PR53/aws_gpu_setup/index.html index 79316d7f..8d54c9a2 100644 --- a/previews/PR53/aws_gpu_setup/index.html +++ b/previews/PR53/aws_gpu_setup/index.html @@ -1,4 +1,4 @@ Tutorial 1 · TrixiCUDA.jl
GitHub

Tutorial - Setting Up Cloud GPUs for Running CUDA.jl

Welcome to this tutorial! If you're looking to use GPUs in the cloud for running Julia programs with CUDA.jl, you're in the right place. This tutorial aims to help you set up cloud GPU on Amazon Web Service (AWS) to run CUDA.jl.

Prerequisite

Before the tutorial begins, you should launch P type EC2 instance(s) on your own in AWS. Here are some links for you to quickly get to know key concepts and successfully launch your instance(s).

  • What is Amazon EC2? https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html
  • Which type of EC2 should I use? https://aws.amazon.com/ec2/instance-types/
  • How to launch Amazon EC2? https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html
  • What is spot instance? (optional) https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances.html
  • How to work with spot instance? (optional) https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-requests.html

The last two resources are optional if the cost isn't a concern for you, and you can choose to stick with on-demand instance(s). However, please note that in both cases you will need to manually request a quota for P type instances before launching.

Connect to Instance(s)

Up to this step, we assume that you have already launched your EC2 instance(s). Now, you can connect to your instance(s) via your local terminal or any IDE with an SSH extension.

Move your key pair file (.pem) to the .ssh folder:

$ mv ~/Downloads/<your-key-pair-name>.pem ~/.ssh/<your-key-pair-name>.pem

Change the permission of the key pair file:

$ chmod 600 ~/.ssh/<your-key-pair-name>.pem

Copy the public IPv4 DNS of your instance(s) from AWS and then connect to your instance(s) via SSH:

$ ssh -i ~/.ssh/<your-key-pair-name>.pem ubuntu@<your-ec2-public-ipv4-dns>

When prompted with the question Are you sure you want to continue connecting (yes/no)?, enter yes. At this point, you should be able to successfully connect to your instance(s).

Configure Julia Package

In this step, we assume that you have already connected to your instance(s) and the commands are being executed in your instance(s)'s terminal.

You can find the version of the Julia package you want from https://julialang.org/downloads/. For illustration, we are using Julia version 1.10.0 for Linux on x86 64-bit.

Copy the address of the Julia package and download it to your instance(s):

$ wget https://julialang-s3.julialang.org/bin/linux/x64/1.10/julia-1.10.0-linux-x86_64.tar.gz

Extract your downloaded package:

$ tar -xvf julia-1.10.0-linux-x86_64.tar.gz

Create a symbolic link to the Julia:

$ sudo ln -s ~/julia-1.10.0/bin/julia /usr/local/bin/julia

Remove the original Julia package (optional):

$ rm julia-1.10.0-linux-x86_64.tar.gz

Then you should be able to run julia command successfully in your instance(s)'s root directory.

Configure CUDA Toolkit

Like the former step, we assume that you have connected to your instance(s) and the commands are being executed in your instance(s)'s terminal.

Then you can easily download and configure the desired version of the CUDA Toolkit based on your selected platform through a set of commands available at https://developer.nvidia.com/cuda-downloads. After that, you may verify the version of the CUDA Toolkit you installed by running the nvcc --version command.

Add CUDA Toolkit to Julia

In this step, we are going to add CUDA package to Julia. Again, this is based on the fact that you have already connected to your insrance(s).

Enter into Julia REPL:

$ julia

Enter Jualia package mode and add CUDA to Julia:

pkg> add CUDA

Then you can simply test CUDA in Julia by creating a test.jl file like below:

# This is a test.jl file
 using CUDA
-CUDA.versioninfo()

If the test result displays the version of CUDA without any errors, you have successfully added CUDA to Julia.

Enable SSH with Git Repository (Optional)

This step is optional but recommended for those who are going to operate their git repository on EC2 instance(s).

Create an SSH key in your local terminal:

$ ssh-keygen -t rsa -b 4096 -C <your-email-address>

Save the key to the default directory ~/.ssh and enter a passphrase of your choice.

Change the permission of your SSH key:

$ chmod 600 ~/.ssh/id_rsa

Add your SSH private key to the ssh-agent:

$ ssh-add -k ~/.ssh/id_rsa

Copy the public key and use it to create a new SSH key in your GitHub.

Copy the public key into your EC2 instance(s):

$ cat ~/.ssh/id_rsa.pub | ssh -i ~/.ssh/<your-key-pair-name>.pem ubuntu@<your-ec2-public-ipv4-dns> "cat >> .ssh/authorized_keys"

Copy the private key to your EC2 instance(s):

$ scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/<your-key-pair-name>.pem ~/.ssh/id_rsa ubuntu@<your-ec2-public-ipv4-dns>:~/.ssh/

Then you can directly git clone and git push (or other operations) in your EC2 instance(s).

In The End

After going through the previous steps, you can use CUDA.jl on your cloud GPU through AWS. Please remember to terminate your instance(s) when you no longer need them, as they will continue to incur charges.

+CUDA.versioninfo()

If the test result displays the version of CUDA without any errors, you have successfully added CUDA to Julia.

Enable SSH with Git Repository (Optional)

This step is optional but recommended for those who are going to operate their git repository on EC2 instance(s).

Create an SSH key in your local terminal:

$ ssh-keygen -t rsa -b 4096 -C <your-email-address>

Save the key to the default directory ~/.ssh and enter a passphrase of your choice.

Change the permission of your SSH key:

$ chmod 600 ~/.ssh/id_rsa

Add your SSH private key to the ssh-agent:

$ ssh-add -k ~/.ssh/id_rsa

Copy the public key and use it to create a new SSH key in your GitHub.

Copy the public key into your EC2 instance(s):

$ cat ~/.ssh/id_rsa.pub | ssh -i ~/.ssh/<your-key-pair-name>.pem ubuntu@<your-ec2-public-ipv4-dns> "cat >> .ssh/authorized_keys"

Copy the private key to your EC2 instance(s):

$ scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/<your-key-pair-name>.pem ~/.ssh/id_rsa ubuntu@<your-ec2-public-ipv4-dns>:~/.ssh/

Then you can directly git clone and git push (or other operations) in your EC2 instance(s).

In The End

After going through the previous steps, you can use CUDA.jl on your cloud GPU through AWS. Please remember to terminate your instance(s) when you no longer need them, as they will continue to incur charges.

diff --git a/previews/PR53/dev_env_info/index.html b/previews/PR53/dev_env_info/index.html index 0fa9083b..a7fea5fe 100644 --- a/previews/PR53/dev_env_info/index.html +++ b/previews/PR53/dev_env_info/index.html @@ -212,4 +212,4 @@ | ID ID Usage | |=======================================================================================| | No running processes found | -+---------------------------------------------------------------------------------------+ ++---------------------------------------------------------------------------------------+ diff --git a/previews/PR53/index.html b/previews/PR53/index.html index 9c33b784..692f0a0e 100644 --- a/previews/PR53/index.html +++ b/previews/PR53/index.html @@ -1,2 +1,2 @@ -Home · TrixiCUDA.jl
GitHub
+Home · TrixiCUDA.jl
GitHub
diff --git a/previews/PR53/nsys_profiling/index.html b/previews/PR53/nsys_profiling/index.html index 210cc708..c331882d 100644 --- a/previews/PR53/nsys_profiling/index.html +++ b/previews/PR53/nsys_profiling/index.html @@ -2,4 +2,4 @@ Tutorial 2 · TrixiCUDA.jl
GitHub

Tutorial 2 - Profiling Kernels with Nvidia Night Systems

Welcome again! This tutorial aims to provide instructions on how to profile CUDA kernels using Nvidia Nsight Systems. The instructions are given based on a cloud Unix system and the VS Code platform (optional).

Basically, you can accurately measure execution time by:

  1. CUDA.@time, a user-friendly measurement tool.

  2. BenchmarkTools.@benchmark(often used together with CUDA.@sync or CUDA.synchronize()), a robust measurement tool.

For large applications, simple time measurement is not enough. Here, we introduce Nvidia Nsight Systems for profiling CUDA kernels. This method can provide an overview of how and when the GPU was active, thereby helping identify which kernels need optimization.

Create Profile

Open a new bash terminal and then launch Julia with nsys from Night Systems:

$ nsys launch julia

Enter the package mode and activate the target Julia environment (already configured with the CUDA package):

pkg> activate <path-to-your-project>/Project.toml

Exit the package mode and write your kernels into the Julia command line. Here we use a simple example to show how to profile:

julia> using CUDA
 julia> a = CUDA.rand(1024, 1024, 1024)
 julia> sin.(a) # Run it once to force compilation
-julia> CUDA.@profile sin.(a)

Then a file ending with .nsys-rep (e.g. report1.nsys-rep) will be created in the current directory. This file contains all the profile data.

View Profile

Exit the Julia REPL and retrieve data from the .nsys-rep file.

There are many ways to customize the view of the profile data. Here, we simply mention three methods that can be directly displayed in your terminal.

  1. Display default statistics from a report
$ nsys stats report1.nsys-rep

This way will export an SQLite file named report1.sqlite from report1.nsys-rep (assuming it does not already exist). Print the default reports in column format to the console.

  1. Display specific data from a report
$ nsys stats --report cuda_gpu_trace report1.nsys-rep

This way will export an SQLite file named report1.sqlite from report1.nsys-rep (assuming it does not already exist). Print the report generated by the cudagputrace script to the console in column format.

  1. Generate multiple reports, in multiple formats, output multiple places from a report
$ nsys stats --report cuda_gpu_trace --report cuda_gpu_kern_sum --report cuda_api_sum --format csv,column --output .,- report1.nsys-rep

Export an SQLite file named report1.sqlite from report1.nsys-rep (assuming it does not already exist). Generate three reports. The first, the cudagputrace report, will be output to the file report1_cuda_gpu_trace.csv in CSV format. The other two reports, cudagpukernsum and cudaapi_sum, will be output to the console as columns of data.

View Profile (Optional)

This section is for better visualizing the profile report using the VS Code extension. The VS Code platform is required.

From the previous section, we created .sqlite and .csv files.

For the .sqlite file, you can download the 'SQLite' extension. Open the Command Palette and enter SQLite: Open Database. Then, you can explore and query the SQLite database from the .sqlite file.

For the .csv file, you can download the 'CSV to Table' extension. Open the Command Palette and enter Convert to table from CSV. Then, you can view the CSV file in table format.

References

Profiling GPU code in CUDA.jl. https://cuda.juliagpu.org/stable/development/profiling/

NVIDIA Nsight Systems user guide. https://docs.nvidia.com/nsight-systems/UserGuide/index.html

+julia> CUDA.@profile sin.(a)

Then a file ending with .nsys-rep (e.g. report1.nsys-rep) will be created in the current directory. This file contains all the profile data.

View Profile

Exit the Julia REPL and retrieve data from the .nsys-rep file.

There are many ways to customize the view of the profile data. Here, we simply mention three methods that can be directly displayed in your terminal.

  1. Display default statistics from a report
$ nsys stats report1.nsys-rep

This way will export an SQLite file named report1.sqlite from report1.nsys-rep (assuming it does not already exist). Print the default reports in column format to the console.

  1. Display specific data from a report
$ nsys stats --report cuda_gpu_trace report1.nsys-rep

This way will export an SQLite file named report1.sqlite from report1.nsys-rep (assuming it does not already exist). Print the report generated by the cudagputrace script to the console in column format.

  1. Generate multiple reports, in multiple formats, output multiple places from a report
$ nsys stats --report cuda_gpu_trace --report cuda_gpu_kern_sum --report cuda_api_sum --format csv,column --output .,- report1.nsys-rep

Export an SQLite file named report1.sqlite from report1.nsys-rep (assuming it does not already exist). Generate three reports. The first, the cudagputrace report, will be output to the file report1_cuda_gpu_trace.csv in CSV format. The other two reports, cudagpukernsum and cudaapi_sum, will be output to the console as columns of data.

View Profile (Optional)

This section is for better visualizing the profile report using the VS Code extension. The VS Code platform is required.

From the previous section, we created .sqlite and .csv files.

For the .sqlite file, you can download the 'SQLite' extension. Open the Command Palette and enter SQLite: Open Database. Then, you can explore and query the SQLite database from the .sqlite file.

For the .csv file, you can download the 'CSV to Table' extension. Open the Command Palette and enter Convert to table from CSV. Then, you can view the CSV file in table format.

References

Profiling GPU code in CUDA.jl. https://cuda.juliagpu.org/stable/development/profiling/

NVIDIA Nsight Systems user guide. https://docs.nvidia.com/nsight-systems/UserGuide/index.html