Containerized Development Environment for Amumax, Pyzfn, and Python
Built with the following tools and technologies:
AMU_CONAINER_ENV is a highly portable, containerized development environment dedicated to running simulations and analyses using Amumax, Pyzfn, and Python. It is optimized for use in any environment supporting virtualization and is specifically designed for seamless integration with high-performance computing centers.
- Amumax Integration: Supports efficient simulations of magnetization dynamics with Amumax.
- Python & Pyzfn: Integrated Python environment with Pyzfn for data analysis and numerical experiments.
- Code-Server: Remote VS Code access in a browser, allowing a familiar development environment from anywhere.
- GPU Acceleration: Optional GPU support for improved simulation performance.
- Supercomputing Ready: Configurable for PCSS infrastructure and other HPC centers, allowing optimal binding and resource management.
└── pcss_container_env/
├── code-server
│ ├── config.yaml
│ ├── extensions-list.txt
│ └── settings.json
├── image.def
├── starship.toml
├── .zshrc
└── start.sh- code-server/: Contains configuration files (
config.yaml,extensions-list.txt,settings.json) that are mounted into the container to set up and customize the VS Code environment. This allows for persistent changes to extensions, editor settings, and other customizations, making the development environment adaptable and user-friendly. - starship.toml: Configuration for the terminal prompt (
starship) used inside the container. It is mounted to ensure a consistent and personalized terminal prompt across sessions. - .zshrc: Shell configuration file for Zsh. This is mounted to provide custom aliases, environment variables, and settings, ensuring the user's shell environment is consistently configured every time the container is started.
AMU_CONAINER_ENVTAINER_ENV/
- starship.toml: Configuration for the terminal prompt setup.
- image.def: Singularity definition file to build the container image.
- start.sh: Startup script to initialize the container environment.
- code-server/
- config.yaml: Configuration file for Code-Server settings.
- settings.json: VS Code settings for extensions and environment setup.
- extensions-list.txt: List of VS Code extensions to be installed in the container environment.
Before getting started with amu_container_env, ensure your runtime environment meets the following requirements:
- Singularity: Ensure that Singularity is installed in your environment.
- GPU Acceleration (Optional): NVIDIA drivers installed for GPU usage.
Install amu_container_env using one of the following methods:
Build from source:
- Clone the amu_container_env repository:
git clone https://github.com/kkingstoun/amu_container_env- Navigate to the project directory:
cd amu_container_env- Pull the container image using Singularity:
singularity pull --arch amd64 library://kkingstoun/amuenv/amuenv:latestNote: You might encounter warnings regarding container verification, which can be safely ignored in most cases.
- Run the startup script to set up the container environment:
./start.sh- Start the Code-Server environment to work on your projects using a browser-based VS Code:
code-serverYou will see output similar to:
[2024-11-08T08:06:57.639Z] info HTTP server listening on http://0.0.0.0:8080/
[2024-11-08T08:06:57.639Z] info - Authentication is enabled
[2024-11-08T08:06:57.639Z] info - Using password from /opt/code-server/config/config.yaml
Open your browser and navigate to http://0.0.0.0:8080/ to access the VS Code environment.
To bind specific directories as needed, edit the start.sh script. For example, to bind /mnt/storage_2/ to make it available inside the container:
--bind /mnt/storage_2/:/mnt/storage_2/ \Ensure the required bindings remain unchanged:
singularity run \
--no-home \
--bind /mnt/storage_2/:/mnt/storage_2/ \
--bind "$SINIMAGE_DIR:$SINIMAGE_DIR:rw" \
--bind ./code-server:$SINIMAGE_DIR/.local/etc/code-server:rw \
--bind ./code-server/settings.json:$SINIMAGE_DIR/.local/share/code-server/User/settings.json \
--home "$SINIMAGE_DIR" \
amuenv_latest.sifTo enable GPU access within the container, execute the following commands:
modprobe nvidia_uvm # ENABLE GPUThen, run the container with the necessary GPU bindings:
# Run the Singularity container with GPU support
singularity run \
--nv \ # ENABLE GPU
...The Install-SecureSSHServer.ps1 script sets up a secure SSH server on a Windows client, creating a user with SSH key-based access, and configuring necessary permissions and environment settings. This guide will walk you through the prerequisites and installation steps required to run the script effectively.
This script is provided as experimental software for automated SSH server configuration on Windows. The author assumes no responsibility for any issues that may arise, including potential system instability or security vulnerabilities. Use this script at your own risk and ensure you have proper system backups before proceeding.
- Operating System: Windows 10/11 or Windows Server 2019/2022
- PowerShell: Version 5.1 or newer
- Administrator Privileges: The script must be run as an administrator.
- Open PowerShell as an administrator.
- Navigate to the directory containing the script and run the following commands:
The new SSH user requires a secure password, which will be used in the script. Define this password in PowerShell using the ConvertTo-SecureString command.
Example:
$NewUserPassword = ConvertTo-SecureString "YourStrongPassword@#!" -AsPlainText -ForceChange to the directory containing the Install-SecureSSHServer.ps1 script and execute it with the required parameters.
Example:
cd windows-client/ssh_server/src/
.\Install-SecureSSHServer.ps1 `
-NewUsername "sshuser" `
-NewUserPassword $NewUserPassword `
-Force `
-sshKeyFolderPath "C:\sshuser"-NewUsername: Specifies the username to be created for SSH access. Example:"sshuser"-NewUserPassword: The password for the SSH user, supplied as aSecureString.-Force: (Optional) If provided, removes any existing user with the specified username before creating a new user.-sshKeyFolderPath: (Optional) Directory where SSH keys will be stored. Default is"C:\sshuser".
- The script checks if OpenSSH Client and Server are installed. If they’re missing, they are added to the system.
- It configures the SSH service to start automatically with the system.
- User Creation: A new local user account is created with the provided username and password.
- Home Directory Initialization: The script ensures that the user’s home directory is properly initialized.
- SSH Group Membership: The user is added to a specific SSH group (
SSH Users), which is created if it doesn’t exist.
- Permissions Setup: The script modifies permissions on the SSH configuration directory (
C:\ProgramData\ssh) to ensure only administrators and the system have full access. - Permission Confirmation: If
-NoConfirmis not set, the user is prompted to confirm these changes.
- Backup Configuration: A backup of the current
sshd_configfile is created. - Configuration Changes: Updates are applied to enable key-based authentication, disable root login, and restrict SSH access to members of the
SSH Usersgroup. - Comparison Display: The script displays a side-by-side comparison of the current and new
sshd_configsettings, prompting for user confirmation if necessary.
- Port Configuration: The SSH port is checked, and the script provides an option to change it if it’s set to the default (22).
- Firewall Rule Setup: A new firewall rule is created to allow inbound connections on the selected port.
- Key Generation: If an SSH key pair does not already exist in the specified folder, the script generates one.
- Permissions: Permissions on the generated key files are restricted to ensure they’re only accessible to the system and the SSH user.
- .ssh Directory Setup: The script creates the
.sshfolder in the user's home directory. - Authorized Keys: The public key is appended to the
authorized_keysfile to allow passwordless login. - Permissions: Permissions on
authorized_keysare set to allow only the SSH user and the system to access it.
- The script performs a test SSH connection to
localhostusing the newly created user and key to confirm the setup was successful.
C:\ProgramData\ssh\sshd_config: The script backs up and modifies this file to enforce secure settings.C:\Users\[NewUsername]\.ssh\authorized_keys: This file is updated with the public key to enable SSH access.- Firewall Rules: A new firewall rule may be added if the SSH port is modified from the default.
- Key Storage: SSH keys are generated and stored in the specified folder (
-sshKeyFolderPath) with restricted access permissions. - Logging: The script logs its actions and results to a log file (
$LogPath), with detailed error handling and informational messages. - Administrator Access: Because the script modifies system files and directories, it must be run with elevated privileges.
Due to frequent issues with read/write speeds on the PCSS storage, it is recommended to clone the repository to a local disk on the Proxima nodes:
🚀 Executing command:
srun -n 1 -c 20 --mem=100G -t 24:00:00 --partition=proxima --gres gpu:1 --pty /bin/bashThen, create a local directory to work in:
mkdir -p /mnt/local/$(whoami)/env/After creating the directory, execute the container installation instructions (git clone and Singularity commands) within this folder.
Ensure that only the necessary storage directories with data are mounted to minimize potential bottlenecks.
-
Task 1:Initial setup of Amumax, Pyzfn, and Python environment. -
Task 2: Implement GPU support verification and testing. -
Task 3: Integrate additional Python libraries for enhanced data processing. -
Task 4: Add Boris Computational Spintronics support. -
Task 4: Add Neuralmag support.
- 💬 Join the Discussions: Share your insights, provide feedback, or ask questions.
- 🐛 Report Issues: Submit bugs found or log feature requests for the
amu_container_envproject. - 💡 Submit Pull Requests: Review open PRs, and submit your own PRs.
Contributing Guidelines
- Fork the Repository: Start by forking the project repository to your github account.
- Clone Locally: Clone the forked repository to your local machine using a git client.
git clone https://github.com/kkingstoun/amu_container_env
- Create a New Branch: Always work on a new branch, giving it a descriptive name.
git checkout -b new-feature-x
- Make Your Changes: Develop and test your changes locally.
- Commit Your Changes: Commit with a clear message describing your updates.
git commit -m 'Implemented new feature x.' - Push to github: Push the changes to your forked repository.
git push origin new-feature-x
- Submit a Pull Request: Create a PR against the original project repository. Clearly describe the changes and their motivations.
- Review: Once your PR is reviewed and approved, it will be merged into the main branch. Congratulations on your contribution!
Contributor Graph
This project is protected under the MIT License. For more details, refer to the LICENSE file.
- Special thanks to the PCSS community for their support.
- Built using Amumax, Pyzfn, and Code-Server.