From b12b421b5ef9e48e214b19107397ebf367eb6212 Mon Sep 17 00:00:00 2001 From: Amine Djeghri <32715913+AmineDjeghri@users.noreply.github.com> Date: Wed, 22 Nov 2023 18:16:14 +0100 Subject: [PATCH] Add files via upload --- Linux-MacOS/README.md | 114 +++++++++++- Linux-MacOS/auto_linux_setup.sh | 77 ++++++++ Linux-MacOS/cuda_pytorch_install.md | 174 ++++++++++++++++++ Linux-MacOS/requirements/conda-env-cpu.yaml | 12 ++ Linux-MacOS/requirements/conda-env-gpu.yaml | 12 ++ Linux-MacOS/requirements/requirements-cpu.txt | 5 + .../requirements/requirements-cuda.txt | 20 ++ Linux-MacOS/requirements/requirements.txt | 1 + Linux-MacOS/test.sh | 59 ++++++ 9 files changed, 473 insertions(+), 1 deletion(-) create mode 100644 Linux-MacOS/auto_linux_setup.sh create mode 100644 Linux-MacOS/cuda_pytorch_install.md create mode 100644 Linux-MacOS/requirements/conda-env-cpu.yaml create mode 100644 Linux-MacOS/requirements/conda-env-gpu.yaml create mode 100644 Linux-MacOS/requirements/requirements-cpu.txt create mode 100644 Linux-MacOS/requirements/requirements-cuda.txt create mode 100644 Linux-MacOS/requirements/requirements.txt create mode 100644 Linux-MacOS/test.sh diff --git a/Linux-MacOS/README.md b/Linux-MacOS/README.md index 9daeafb..bcd053a 100644 --- a/Linux-MacOS/README.md +++ b/Linux-MacOS/README.md @@ -1 +1,113 @@ -test +# Auto-Linux-Setup +![WSL terminal](https://github.com/AmineDjeghri/BetterWindows/blob/master/resources/wsl_terminal.jpg) +Welcome to the guide for setting up a powerful terminal & python environment on Linux/WSL. + +The repository contains an automated script to install the main elements I use in Linux/WSL. The repository is currently in developement. +My workflow involves utilizing WSL for development tasks and relying on Windows for all other activities. You can check the parent repository here : https://github.com/AmineDjeghri/Awesome-Windows11-WSL + +**Purpose of this script:** I developed this tool to streamline the setup process when working on various servers, such as AWS instances. Additionally, for those who find manual Linux setup challenging, this script is designed to simplify the process. Forget about the manual setup for each new working machine – just run the script, and you're ready to code. While it's particularly useful for Python developers, feel free to customize it for your specific needs. + +The documentation is divided into two sections: one automatic installation script and the second explaining the manual installation process. +The script incorporates various packages and prompts you for each installation decision. It covers the following aspects: +- Customize the Terminal: font, ZSH, Oh-my-zsh, and the power-level-10k theme. +- Miniconda Installation: The script facilitates the installation of Miniconda and the creation of an isolated environment tailored for PyTorch, designed to be compatible with your GPU/CPU. This step is crucial for users with a GPU who intend to utilize CUDA. Installing CUDA in a separate environment is recommended. Additional details can be found in the provided documentation [here](https://github.com/AmineDjeghri/Auto-Linux-Setup/blob/main/cuda_pytorch_install.md) +- Nvidia Driver installation: To Do + +## Table of contents +1. [Setup Linux automatically ](#1-setup-linux-automatically-) + +2. [Setup Linux manually ](#2-setup-linux-manually--) + * [customize Linux/WSL terminal](#21-customize-linuxwsl-terminal) + * [Install and set up the font](#211-install-and-set-up-the-font) + * [Install ZSH](#212-install-zsh) + * [Install Oh-my-zsh](#213-install-oh-my-zsh) + * [Install the power-level-10k theme](#214-install-the-power-level-10k-theme) + * [install miniconda in WSL:](#22-install-miniconda-in-wsl) + * [Install Cuda using conda or globally in the OS :](#23-install-cuda-using-conda-or-globally-in-the-os--) + +## 1. Setup Linux automatically : +- Install the MesloLGS NF Regular.ttf font on your primary operating system (Windows if you are using WSL). [MesloLGS NF Regular.ttf](https://github.com/romkatv/dotfiles-public/blob/master/.local/share/fonts/NerdFonts/MesloLGS%20NF%20Regular.ttf) +- Configure your terminal to utilize the newly installed font. For Windows users, open the Windows Terminal and navigate to Settings -> Profiles (bottom left) -> Ubuntu terminal (orange logo) -> Additional Parameters -> Appearance -> Change the font to MesloLGS. +- Run the following command inside Linux to automatically install the various elements. Do not run the script with `sudo` otherwise it will install some packages in /root instead of /home. Also, make sure to type `yes` when installing conda. If you are facing some problems, take a look at section 2.2. + +```sh -c "$(wget https://raw.githubusercontent.com/AmineDjeghri/Auto-Linux-Setup/main/linux_terminal_custom.sh -O -)"``` + +- (Optional) If you have custom configurations in your `.bashrc`, consider copying them to the `.zshrc` file. + +## 2. Setup Linux manually : + +## 2.1 customize Linux/WSL terminal + +### 2.1.1 Install and set up the font +- Install the MesloLGS NF Regular.ttf font on your primary operating system (Windows if you are using WSL). [MesloLGS NF Regular.ttf](https://github.com/romkatv/dotfiles-public/blob/master/.local/share/fonts/NerdFonts/MesloLGS%20NF%20Regular.ttf) +- Configure your terminal to utilize the newly installed font. For Windows users, open the Windows Terminal and navigate to Settings -> Profiles (bottom left) -> Ubuntu terminal (orange logo) -> Additional Parameters -> Appearance -> Change the font to MesloLGS. + +### 2.1.2 Install ZSH +4. install Zsh: + + - With the package manager of your choice, _e.g._ `sudo apt install zsh` + +5. Verify installation by running `zsh --version`. Expected result: `zsh 5.8.1` or more recent. +6. Run `zsh` to start configuring it (You can create an empty file with 0, will configure it later). You can delete .zshrc and run `zsh` to configure it again. +7. Log out and log back in again to use your new default shell. +8. Optional:Make it your default shell: `chsh -s $(which zsh)` but doesn't work on every system. If it doesn't work, will do it later with oh-my-zsh. Test that it worked with `echo $SHELL`. Expected result: `/bin/zsh` or similar.Test with `$SHELL --version`. Expected result: 'zsh 5.8' or similar + +source : https://github.com/ohmyzsh/wiki/edit/main/Installing-ZSH.md + +### 2.1.3 Install Oh-my-zsh +#### What is Oh my ZSH ? +- Oh My Zsh is an open source, community-driven framework for managing your zsh configuration. +- Run `sh -c "$(wget https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh -O -)"` and hit yes to make it the default terminal. +- Test if it worked with `echo $SHELL`. Expected result: `/bin/zsh` or similar. Test with `$SHELL --version`: Expected result: 'zsh 5.8' or similar +- If you have conda / cuda or something installed : copy the content from `.bashrc` to `.zshrc` ( `vim .zshrc` or use windows explorer / sublime text . Run again `zsh`, you should see the `(base)` name before the command. + +#### Zsh Plugins : +A Zsh plugin is a set of useful aliases and functions designed to make you more productive. Here are some useful & popular plugins : + +``` +git clone https://github.com/zsh-users/zsh-autosuggestions.git $ZSH_CUSTOM/plugins/zsh-autosuggestions +git clone https://github.com/zsh-users/zsh-syntax-highlighting.git $ZSH_CUSTOM/plugins/zsh-syntax-highlighting +git clone --depth 1 -- https://github.com/marlonrichert/zsh-autocomplete.git $ZSH_CUSTOM/plugins/zsh-autocomplete +``` + +Open your .zshrc file at ~/.zshrc (you can do that through windows explorer and sublime text as you did before) and search for `plugins=(git)`. +If you don't find it, then create it and complete it with the missing plugins as shown in the example bellow : +``` +plugins=(git + dirhistory + history + colored-man-pages + jsontools + zsh-autosuggestions + zsh-syntax-highlighting + zsh-autocomplete +``` +- run - `zsh` to restart the terminal, now you can see the changes when you try to write a command like `cd` +- If there are some problems test the solutions available [here](https://stackoverflow.com/a/37175174/8354747) & [here](https://stackoverflow.com/a/36994356/8354747) +Others : +- Auto update oh my zsh : uncomment this: `zstyle ':omz:update' mode auto` +- Add the following alias to the end of .zshrc file to easily open sublime Text from windows: `alias sublime="subl.exe"`. Try it with : `sublime .zshrc` +- You can visit this :[website](https://www.linkedin.com/pulse/how-install-start-using-oh-my-zsh-boost-your-mantas-levinas/?trk=pulse-article_more-articles_related-content-card) to understand more about the installed plugins. you can skip directly to `7. Enable Zsh Plugins` section and start reading. You will see that you have installed most of the commands there. + +### 2.1.4 Install the power-level-10k theme +#### zsh themes : Powerlevel10k +- Powerlevel10k is a theme for Zsh. It emphasizes speed, flexibility and out-of-the-box experience. +- `git clone --depth=1 https://github.com/romkatv/powerlevel10k.git ${ZSH_CUSTOM:-$HOME/.oh-my-zsh/custom}/themes/powerlevel10k` +- Open .zshrc and set `ZSH_THEME="powerlevel10k/powerlevel10k"`. +- install this font on windows [MesloLGS NF Regular.ttf](https://github.com/romkatv/dotfiles-public/blob/master/.local/share/fonts/NerdFonts/MesloLGS%20NF%20Regular.ttf) +- open windows terminal and go to settings -> profiles (bottom left) -> Ubuntu terminal(orange logo) -> additional parameters-> apparence -> change the font to MesloLGS. +- You can also do the step aboce inside powershell if you use it to call ubuntu with `wsl` or `ubuntu` command . You can also change it's background color +- (optional) add a new color scheme with this background #383B40. Select the new color scheme on your terminal an put a transparency of 85% [example]([here](https://pureinfotech.com/change-color-scheme-windows-terminal/)) +- reload ubuntu terminal with `zsh` and configure your theme. + +### 2.2 install miniconda in WSL: + - run `wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh` (Conda 23.3.1 Python 3.10.10 released April 24, 2023) + - run `chmod +x Miniconda3-latest-Linux-x86_64.sh` + - run `bash Miniconda3-latest-Linux-x86_64.sh` + - restart the shell with `bash` command (or `zsh` if you changed your shell) + - if `(base)` is not showing and `.bachrc` file doesn't contain the init of conda, go to `~/miniconda3/bin` and run `conda init` + - close and reopen ubuntu terminal or run `bash`, you should see `(base)` at the left of the any command. + - run `conda env list` to check the installed environements and their path. + +### 2.3 Install Cuda using conda or globally in the OS : +- Follow this [conda_cuda_installation guide](https://github.com/AmineDjeghri/Auto-Linux-Setup/blob/main/cuda_pytorch_install.md) \ No newline at end of file diff --git a/Linux-MacOS/auto_linux_setup.sh b/Linux-MacOS/auto_linux_setup.sh new file mode 100644 index 0000000..4a39a4a --- /dev/null +++ b/Linux-MacOS/auto_linux_setup.sh @@ -0,0 +1,77 @@ +#!/bin/bash + +# Function to ask yes/no questions +ask_yes_no() { + while true; do + read -p "$1 (yes/no) [default: yes]: " yn + case $yn in + [Yy]* ) return 0;; + [Nn]* ) return 1;; + "" ) return 0;; # Default to yes if Enter is pressed + * ) echo "Please answer yes or no.";; + esac + done +} + +# Ask user if they want to customize the terminal +if ask_yes_no "Do you want to customize the terminal by installing ZSH, Oh My Zsh, and the Powerlevel10k theme?"; then + # Check if the font "MesloLGS NF Regular.ttf" is installed + if ask_yes_no "Is the font 'MesloLGS NF Regular.ttf' installed?"; then + # Install zsh + echo "Installing zsh..." + apt install zsh + + # Check zsh version + echo "Checking zsh version..." + zsh --version + + # Restart the shell script + echo "Restarting shell script..." + source ~/.zshrc + echo "Current shell: $SHELL" + $SHELL --version + + # Install Oh My Zsh + echo "Installing Oh My Zsh..." + sh -c "$(wget https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh -O -)" + + # Install plugins + echo "Installing plugins..." + git clone https://github.com/zsh-users/zsh-autosuggestions.git ${ZSH_CUSTOM:-$HOME/.oh-my-zsh/custom}/plugins/zsh-autosuggestions + git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-$HOME/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting + git clone --depth 1 -- https://github.com/marlonrichert/zsh-autocomplete.git ${ZSH_CUSTOM:-$HOME/.oh-my-zsh/custom}/plugins/zsh-autocomplete + + # Install Powerlevel10k theme + echo "Installing Powerlevel10k theme..." + git clone --depth=1 https://github.com/romkatv/powerlevel10k.git ${ZSH_CUSTOM:-$HOME/.oh-my-zsh/custom}/themes/powerlevel10k + + echo "Updating .zshrc configuration..." + curl -sL "https://raw.githubusercontent.com/AmineDjeghri/Auto-Linux-Setup/main/.zshrc" > ~/.zshrc + source ~/.zshrc + # Print installation message + echo -e "\e[93m(Optional) If you have custom configurations in your .bashrc, consider copying them to the .zshrc file.\e[0m" + + else + echo "\033[0;33mPlease install the font 'MesloLGS NF Regular.ttf' before customizing the terminal." + echo "You can check this link for instructions: https://github.com/AmineDjeghri/Auto-Linux-Setup/tree/main#1-setup-linux-automatically-\e[0m" + fi +else + echo "\e[93mTerminal customization skipped.\e[0m" +fi + +# Ask user if they want to install Miniconda +if ask_yes_no "Do you want to install Miniconda3?"; then + # Download and install Miniconda + wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh + chmod +x Miniconda3-latest-Linux-x86_64.sh + bash Miniconda3-latest-Linux-x86_64.sh + source ~/.zshrc + echo "initializing conda in ZSH'" + cd ~ + ./miniconda3/bin/conda init zsh + source ~/.zshrc + + # Display additional message + echo "\e[93mRestart the Ubuntu terminal, you should see '(base)' at the left of any command." + echo "Run 'conda env list' to check the installed environments and their paths.\e[0m" +fi diff --git a/Linux-MacOS/cuda_pytorch_install.md b/Linux-MacOS/cuda_pytorch_install.md new file mode 100644 index 0000000..2b07f7c --- /dev/null +++ b/Linux-MacOS/cuda_pytorch_install.md @@ -0,0 +1,174 @@ +# Conda, Pytorch & CUDA installation inside Linux/WSL + +Installing CUDA inside a Conda environment instead of globally on your computer has several advantages: + +1. Isolation: When you install CUDA within a Conda environment, it won't interfere with the CUDA installation at the global level or with other Conda environments. + +2. Version Control: Different projects may require different versions of CUDA or GPU-related libraries. By creating separate Conda environments for different projects, you can easily switch between CUDA versions and ensure that each project uses the specific version it needs. + +3. Clean Uninstall: If you later decide to remove a project and its associated dependencies, it's straightforward to delete the Conda environment, which ensures a clean uninstallation without leaving traces on your system. + + +## Summary +1. [Install the Nvidia driver](#1-install-the-nvidia-driver) + +2. [First and Recommended Option: Installing CUDA Inside a Conda Environment](#2-first-and-recommended-option-installing-cuda-inside-a-conda-environment) + * [Automatic installation](#21-automatic-installation-) + * [Manual installation ](#22-manual-installation) +2. [Second Option: Install CUDA Globally (Not Recommended)](#3-second-option-install-cuda-globally-not-recommended) + + +## 1. Install the Nvidia driver +- If you are on Linux and not WSL, download and install the nvidia driver .RUN file inside Linux: [nvidia drivers](https://www.nvidia.fr/Download/index.aspx?lang=fr). Then run `nvidia-smi` on the terminal, it should print you all the information about your GPU +- If you use WSL, install the nvidia driver (.exe) on windows and not WSL from [nvidia drivers](https://www.nvidia.fr/Download/index.aspx?lang=fr). Run `nvidia-smi` on both windows & WSL Terminal, it should print you all the information about your GPU + +## 2. First and Recommended Option: Installing CUDA Inside a Conda Environment + +#### 2.1. Automatic installation : + +#### 2.2 Manual installation +The next commands need to be ran inside WSL or Linux and not Windows : +- Install build-essential `sudo apt-get install build-essential` (required by some packages like llama-cpp-python for example) + +- Create a `conda-env-gpu.yaml` file with the content below (you can modify the name of the env and the packages). I usually have two yaml files : +One for GPU containing Pytorch with CUDA named `conda-env-gpu.yml` another one for CPU (not containing cuda) named `conda-env-cpu.yml`. I specify the version of pytorch in requirements.txt. +If pytorch was installed with cuda, running the requirements.txt won't install again pytorch since the same version is installed. +Running the conda-env-cpu.yaml that doesn't contain pytorch will install pytorch CPU from requirements.txt + + + +*conda-env-gpu.yaml* +``` +name: env-cuda +dependencies: + - python=3.9 + - git + - pip + - conda-forge::ninja + - nvidia/label/cuda-11.8.0::cuda + - conda-forge::ffmpeg + - conda-forge::gxx=11.4 + - pip: + - -r requirements-cuda.txt + # - -e ."[dev]" +``` + +*requirements-cuda.txt* +``` +# torch with CUDA support +--extra-index-url https://download.pytorch.org/whl/cu118 +torch==2.0.1+cu118 + +# requirements +-r requirements.txt +``` + +*requirements.txt* +``` +torch==2.0.1 +``` + +*conda-env-cpu.yaml* +``` +name: env-cpu +dependencies: + - python=3.9 + - git + - pip + - conda-forge::ninja + - conda-forge::ffmpeg + - conda-forge::gxx=11.4 + - pip: + - -r requirements.txt +``` + +*requirements-cpu.txt* +``` +--extra-index-url https://download.pytorch.org/whl/cpu +torch==2.0.1 + +# requirements +-r requirements.txt +``` + +Create an environment with CUDA support by executing the command: `conda env update -n my-env -f conda-env-gpu.yml; conda activate my-env;`. The `-n my-env` option will supersede the environment name specified within the file. Alternatively, for CPU support only, use the command: `conda env update -n my-env -f conda-env-cpu.yml; conda activate my-env;`. + +- Run `nvcc --version ; # should be cuda_11.8.r11.8` +- Verify the successful installation of PyTorch by executing the following Python code: + +```py +import torch +# setting device on GPU if available, else CPU +device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') +print('Using device:', device) +print() + +#Additional Info when using cuda +if device.type == 'cuda': + print(torch.cuda.get_device_name(0)) + print('Memory Usage:') + print('Allocated:', round(torch.cuda.memory_allocated(0)/1024**3,1), 'GB') + print('Cached: ', round(torch.cuda.memory_reserved(0)/1024**3,1), 'GB') +``` + +- When utilizing pre-commit hooks, ensure to exclude the file requirements-cuda.txt to prevent sorting the packages. +- Prioritize the installation of applications using pip; if pip is not applicable, resort to conda; and if conda is not an option, utilize apt-get. For instance, when installing Tesseract through conda, it is preferable to install it within the conda environment rather than globally using apt-get. + +## 3. Second Option: Install CUDA Globally (Not Recommended) + +* Inside linux, run `nvidia-smi`, you should see a table showing information about your GPU. +* If cuda is not installed and you run `nvcc --version`, you will get an error because cuda toolkit is not installed yet. +* If you have cuda toolkit installed on windows (for WSL) or Linux, run `nvcc --version`, you should see the version of cuda which means that you already have cuda set up globally. if there are two different CUDA versions shown by nvcc and NVIDIA-smi, it is normal: [source](https://stackoverflow.com/a/53504578) + +Now, we will install cuda on WSL, the following commands must all be runned inside WSL. [If you want to understand more](https://docs.nvidia.com/cuda/wsl-user-guide/index.html#getting-started-with-cuda-on-wsl): +* Run `sudo apt-key del 7fa2af80` +* The CUDA driver installed on Windows host will be stubbed inside the WSL 2 as libcuda.so, therefore users must not install any NVIDIA GPU Linux driver within WSL 2 +* You can only install a cuda toolkit that is compatible with WSL inside wsl : [link](https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_local) by following these instructions. +Since we we are installing cuda 11.7, follow the commands below. + +``` +wget https://developer.download.nvidia.com/compute/cuda/11.7.0/local_installers/cuda_11.7.0_515.43.04_linux.run +sudo sh cuda_11.7.0_515.43.04_linux.run +``` +* Restart the terminal and run `nvcc --version` to test cuda installation inside wsl. If nvcc is not recognized, and you see something like `Command 'nvcc' not found, but can be installed with: +sudo apt install nvidia-cuda-toolkit`, it may be possible that the path is not inside .bashrc. First, verify that there is `/usr/local/cuda-11.7` inside WSL .Then, open `.bashrc` and add this at the end if it's not there: +``` +export CUDA_HOME=/usr/local/cuda-11.7 +export PATH=${CUDA_HOME}/bin:${PATH} +export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:$LD_LIBRARY_PATH +``` +* Restart the terminal and run again the command `nvcc --version`, you should see `Cuda compilation tools, release 11.7, V11.7.64` +* You may also need to install cudnn from [here](https://developer.nvidia.com/rdp/cudnn-archive) version v8.9.1 (May 5th, 2023) for CUDA 11.x. You need to sign in, download the right version of cdnn for cuda 11.x. directly in wsl using windows explorer and then extract it with windows explorer then run it using `sudo dpkg -i cudnn-local-repo-ubuntu2204-8.9.1.23_1.0-1_amd64.deb` or `sudo apt install ./cudnn-local-repo-ubuntu2204-8.9.1.23_1.0-1_amd64.deb`. if you install it twice you should see `cudnn-local-repo-ubuntu2204-8.9.1.23 is already the newest version (1.0-1).` + +- test pytorch : + ```sh + conda create -n test python=3.10; + conda activate test; + pip install torch; + ``` + + ```py + import torch + # setting device on GPU if available, else CPU + device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') + print('Using device:', device) + print() + + #Additional Info when using cuda + if device.type == 'cuda': + print(torch.cuda.get_device_name(0)) + print('Memory Usage:') + print('Allocated:', round(torch.cuda.memory_allocated(0)/1024**3,1), 'GB') + print('Cached: ', round(torch.cuda.memory_reserved(0)/1024**3,1), 'GB') + ``` + +(Optional) Tutorials and sources : +* [Download cuda 11.7](https://developer.nvidia.com/cuda-11-7-0-download-archive?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_local) +* [Cuda-wsl Nvidia guide](https://docs.nvidia.com/cuda/wsl-user-guide/index.html#getting-started-with-cuda-on-wsl-2](https://docs.nvidia.com/cuda/wsl-user-guide/index.html#getting-started-with-cuda-on-wsl-2)) +* [Cuda-wsl Ubuntu guide](https://ubuntu.com/tutorials/enabling-gpu-acceleration-on-ubuntu-on-wsl2-with-the-nvidia-cuda-platform#3-install-nvidia-cuda-on-ubuntu) + + + + + + diff --git a/Linux-MacOS/requirements/conda-env-cpu.yaml b/Linux-MacOS/requirements/conda-env-cpu.yaml new file mode 100644 index 0000000..435d19f --- /dev/null +++ b/Linux-MacOS/requirements/conda-env-cpu.yaml @@ -0,0 +1,12 @@ +name: env-cpu +dependencies: + - python=3.9 + - git + - pip + - conda-forge::ninja + - conda-forge::ffmpeg + - conda-forge::gxx=11.4 + - pip: + - --extra-index-url https://download.pytorch.org/whl/cpu + - torch==2.0.1 + - -r requirements-cpu.txt diff --git a/Linux-MacOS/requirements/conda-env-gpu.yaml b/Linux-MacOS/requirements/conda-env-gpu.yaml new file mode 100644 index 0000000..79b7bf5 --- /dev/null +++ b/Linux-MacOS/requirements/conda-env-gpu.yaml @@ -0,0 +1,12 @@ +name: cuda-env +dependencies: + - python=3.9 + - git + - pip + - conda-forge::ninja + - nvidia/label/cuda-11.8.0::cuda + - conda-forge::ffmpeg + - conda-forge::gxx=11.4 + - pip: + - -r requirements-cuda.txt + # - -e ."[dev]" diff --git a/Linux-MacOS/requirements/requirements-cpu.txt b/Linux-MacOS/requirements/requirements-cpu.txt new file mode 100644 index 0000000..5cb1e3e --- /dev/null +++ b/Linux-MacOS/requirements/requirements-cpu.txt @@ -0,0 +1,5 @@ +--extra-index-url https://download.pytorch.org/whl/cpu +torch==2.0.1 + +# requirements +-r requirements.txt diff --git a/Linux-MacOS/requirements/requirements-cuda.txt b/Linux-MacOS/requirements/requirements-cuda.txt new file mode 100644 index 0000000..ae24299 --- /dev/null +++ b/Linux-MacOS/requirements/requirements-cuda.txt @@ -0,0 +1,20 @@ +# torch with CUDA support +--extra-index-url https://download.pytorch.org/whl/cu118 +torch==2.0.1+cu118 + +# llama-cpp-python with CUDA support +--extra-index-url https://jllllll.github.io/llama-cpp-python-cuBLAS-wheels/AVX2/cu118 +--prefer-binary +llama-cpp-python + +# Spacy with cuda +spacy[cuda-autodetect]==3.6.1 + +# requirements +-r requirements.txt + +# ExLlama +exllama@ https://github.com/jllllll/exllama/releases/download/0.0.17/exllama-0.0.17+cu118-cp39-cp39-linux_x86_64.whl + +# ExLlamaV2 +exllamav2==0.0.2 diff --git a/Linux-MacOS/requirements/requirements.txt b/Linux-MacOS/requirements/requirements.txt new file mode 100644 index 0000000..65c65ae --- /dev/null +++ b/Linux-MacOS/requirements/requirements.txt @@ -0,0 +1 @@ +torch==2.0.1 diff --git a/Linux-MacOS/test.sh b/Linux-MacOS/test.sh new file mode 100644 index 0000000..87ab9b1 --- /dev/null +++ b/Linux-MacOS/test.sh @@ -0,0 +1,59 @@ +#!/bin/bash + +# Function to ask yes/no questions +ask_yes_no() { + while true; do + read -p "$1 (yes/no): " yn + case $yn in + [Yy]* ) return 0;; + [Nn]* ) return 1;; + * ) echo "Please answer yes or no.";; + esac + done +} + +# Ask user if they want to customize the terminal +if ask_yes_no "Do you want to customize the terminal by installing ZSH, Oh My Zsh, and the Powerlevel10k theme?"; then + # Check if the font "MesloLGS NF Regular.ttf" is installed + if ask_yes_no "Is the font 'MesloLGS NF Regular.ttf' installed?"; then + # Install zsh + echo "Installing zsh..." + sudo apt install zsh + + # Check zsh version + echo "Checking zsh version..." + zsh --version + + # Restart the shell script + echo "Restarting shell script..." + source ~/.zshrc + echo "Current shell: $SHELL" + $SHELL --version + + # Install Oh My Zsh + echo "Installing Oh My Zsh..." + sh -c "$(wget https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh -O -)" + + # Install plugins + echo "Installing plugins..." + git clone https://github.com/zsh-users/zsh-autosuggestions.git $ZSH_CUSTOM/plugins/zsh-autosuggestions + git clone https://github.com/zsh-users/zsh-syntax-highlighting.git $ZSH_CUSTOM/plugins/zsh-syntax-highlighting + git clone --depth 1 -- https://github.com/marlonrichert/zsh-autocomplete.git $ZSH_CUSTOM/plugins/zsh-autocomplete + + # Install Powerlevel10k theme + echo "Installing Powerlevel10k theme..." + git clone --depth=1 https://github.com/romkatv/powerlevel10k.git ${ZSH_CUSTOM:-$HOME/.oh-my-zsh/custom}/themes/powerlevel10k + + echo "Updating .zshrc configuration..." + curl -sL "https://raw.githubusercontent.com/AmineDjeghri/Auto-Linux-Setup/main/.zshrc" > ~/.zshrc + zsh + # Print installation message + echo -e "\e[93mInstallation DONE. Do not forget to .zshrc with the content of .bashrc if required\e[0m" + echo "Script execution completed." + else + echo "Please install the font 'MesloLGS NF Regular.ttf' before customizing the terminal." + echo "You can check this link for instructions: https://github.com/AmineDjeghri/Auto-Linux-Setup/tree/main#1-setup-linux-automatically-" + fi +else + echo "Terminal customization skipped." +fi