Add native support to docker scripts for passing Nvidia Graphics into…

… Docker (StoglRobotics#66) * Add support for creating nvidia supported containers. * Install additonal packages: bach-completion, nala, ros-vcstool, pre-commit * Use nala per default in when setting up docker. * allow docker container to use ros_distro that is not installed Co-authored-by: Manuel M <[email protected]>
Robotgir · Oct 5, 2022 · a3e02d1 · a3e02d1
1 parent 0d78128
commit a3e02d1
Show file tree

Hide file tree

Showing 10 changed files with 292 additions and 41 deletions.
diff --git a/COLCON_IGNORE b/COLCON_IGNORE
diff --git a/docs/docker/general_information_docker/general_information_docker.rst b/docs/docker/general_information_docker/general_information_docker.rst
@@ -13,7 +13,7 @@ You have to install docker which is dependent on the operating system you are us
 
 *   `Windows <https://docs.docker.com/desktop/windows/install/>`_
 *   `Mac <https://docs.docker.com/desktop/mac/install/>`_
-*   `Linux <https://docs.docker.com/engine/install/>`_: it depends.
+*   `Linux <https://docs.docker.com/desktop/install/linux-install/>`_: it depends.
 
 You can verify that you have docker successfully installed by running:
 

diff --git a/docs/docker/nvidia_docker/setup_nvidia_support.rst b/docs/docker/nvidia_docker/setup_nvidia_support.rst
@@ -9,6 +9,8 @@ Prerequisites
 """"""""""""""
 Before we can install the NVIDIA Container Toolkit we have to install the nvidia drivers for our platform and install docker.
 
+**NOTE**: Check the end of the file for some tips about throubleshoting.
+
 Nvidia drivers
 ----------------
 Make sure you have the NVIDIA drivers for your Linux distribution installed. This can either be done by using your package manager or you can download the ``.run`` installers from `NVIDIA Driver Downloads <https://www.nvidia.com/Download/index.aspx?lang=en-us>`_.
@@ -51,8 +53,12 @@ Docker
 
 Make sure docker is installed and it's working correctly. For instructions on how to install docker have a look :ref:`in general info on docker installation <general-info-on-docker-installation>`.
 
-Installation
-""""""""""""""""
+How to setup a container with nvidia support
+""""""""""""""""""""""""""""""""""""""""""""""
+Install nvidia-docker
+----------------------
+**NOTE**: If you already have installed nvidia-docker, you can skip this and go right to the next section.
+
 1. You then have to install the NVIDIA Container Toolkit for docker as described `in official documentation <https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker>`_.
    For Ubuntu this can be done as follows:
 
@@ -113,6 +119,13 @@ Installation
     |=============================================================================|
     +-----------------------------------------------------------------------------+
 
+Change the Dockerfile
+----------------------------------
+(1.) If you haven't done so already:
+   Create a new docker workspace with the :ref:`setup-ros-workspace-docker<uc-setup-docker-workspace>` command.
+
+   **NOTE**: If you set up a nvidia-docker container, you are finished at this point.
+
 2. Replace the the ``FROM ubuntu:<version>`` directive in your Dockerfile with the nvidia container of your needs. The following table gives you a quick overview:
 
    .. list-table:: Examples for nvidia containers
@@ -154,3 +167,46 @@ Installation
 
 
 You now should have a docker container which exposes your nvidia drivers and can switch to your workspace with ``rtw_switch_to_docker``.
+
+Troubleshoting
+""""""""""""""
+
+Be sure that graphics card driver is installed properly
+--------------------------------------------------------
+Check if drivers and libraries are installed properly by executing ``nvidia-smi`` after a fresh system start (Yes, you have to restart your computer!).
+There you should see expected version of Nvidia driver.
+If you get any error follow the next steps to be sure that the expected version of driver is installed properly.
+
+1. Install required version of the driver using GUI for additional drivers in system's settings or ``ubuntu-drivers`` command.
+
+   .. note::
+      Be careful when executing command if multiple devices are using custom drivers you can unintenntionaly install wrong driver for another device (you will know if this is relevant for you - but it is imprtant to note it.)
+
+2. Delete all other nvidia driver's version and corresponding libraries - use ``purge`` command for it.
+3. Restart your computer.
+4. If you have issues with the graphics after restart do the following:
+
+   1. Close your eyes and breathe slowely in and out at least once :)
+   2. You probably didn't install everything properly so the open-source ``noveau`` driver is used which is not adequate for this scenario.
+   3. Use ``<CTRL> + <ALT> + <F2-3-4...>`` keys to switch to a linux terminal.
+   4. Login there and execute ``ubuntu-drivers`` command to install missing drivers.
+   5. Now restart again and everything should work properly.
+
+5. Now check again output of the ``ṅvidia-smi`` command
+
+
+Graphics card should be used in the ``Performance`` mode
+---------------------------------------------------------
+If you have only ``X`` in the output from ``nvidia-smi`` than make sure that the graphics card uses ``Performance Mode`` in ``PRIME profile`` section of "NVIDIA X Server Settings" application.
+
+
+``nvidia-smi`` command inside docker container
+-----------------------------------------------
+Take a note that ``nvidia-smi`` command in the docker container is necessary test to see if docker has access to the graphic card, but it doesn't shows any applications that are using it.
+You can see on your host if a docker application is using graphic card and how much.
+
+
+References
+"""""""""""
+
+- [Docker configuration for rviz using nvidia graphics card (by @ruffsl)](https://gist.github.com/ruffsl/e7ca631a618ece3eb0be8e4bf168accb)
diff --git a/scripts/_RosTeamWs_Defines.bash b/scripts/_RosTeamWs_Defines.bash
@@ -355,6 +355,11 @@ function is_valid_ros_distribution {
 
 function check_ros_distro {
   ros_distro=$1
+  if [ -z "$2" ]; then
+    use_docker="false"
+  else
+    use_docker=$2
+  fi
 
   # check if the given distribution is a distribution supported by rtw
   while ! is_valid_ros_distribution "$ros_distro" rtw_supported_ros_distributions[@];
@@ -363,16 +368,19 @@ function check_ros_distro {
       read ros_distro
   done
 
-  if [ ! -d "/opt/ros/$ros_distro" ]; then
-    local upper_case=$(echo $ros_distro | tr '[:lower:]' '[:upper:]')
-    local alternative_ros_location=ALTERNATIVE_ROS_${upper_case}_LOCATION
-    if [ ! -f "${!alternative_ros_location}/setup.bash" ]; then
-      print_and_exit "FATAL: ROS '$ros_distro' not installed on this computer! Exiting..."
-    else
-      user_decision "Using ${ALTERNATIVE_ROS_LOCATION} for ${ros_distro}." user_answer
-      # check if the chosen ros-distro location is correct.
-      if [[ " ${negative_answers[*]} " =~ " ${user_answer} " ]]; then
-        print_and_exit "Please set your ALTERNATIVE_ROS_LOCATION to the correct location. Exiting..."
+  # inside docker container we don't need to check if ros distro is present on system
+  if [ "${use_docker}" != "true" ]; then
+    if [ ! -d "/opt/ros/$ros_distro" ]; then
+      local upper_case=$(echo $ros_distro | tr '[:lower:]' '[:upper:]')
+      local alternative_ros_location=ALTERNATIVE_ROS_${upper_case}_LOCATION
+      if [ ! -f "${!alternative_ros_location}/setup.bash" ]; then
+        print_and_exit "FATAL: ROS '$ros_distro' not installed on this computer! Exiting..."
+      else
+        user_decision "Using ${ALTERNATIVE_ROS_LOCATION} for ${ros_distro}." user_answer
+        # check if the chosen ros-distro location is correct.
+        if [[ " ${negative_answers[*]} " =~ " ${user_answer} " ]]; then
+          print_and_exit "Please set your ALTERNATIVE_ROS_LOCATION to the correct location. Exiting..."
+        fi
       fi
     fi
   fi
@@ -405,8 +413,13 @@ function set_ros_version_for_distro {
 
 function check_and_set_ros_distro_and_version {
   ros_distro=$1
+  if [ -z "$2" ]; then
+    use_docker="false"
+  else
+    use_docker=$2
+  fi
 
-  check_ros_distro "${ros_distro}"
+  check_ros_distro "${ros_distro}" "${use_docker}"
   set_ros_version_for_distro "${ros_distro}"
 }
 

diff --git a/scripts/configuration/terminal_coloring.bash b/scripts/configuration/terminal_coloring.bash
@@ -1,6 +1,6 @@
 #!/bin/bash
 #
-# Copyright 2022 Manuel Muth (Stogl Robotics Consulting)
+# Copyright 2022 Stogl Robotics Consulting UG (haftungsbeschränkt)
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -13,6 +13,9 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+#
+# Authors: Manuel Muth, Denis Štogl
+#
 
 # BEGIN: Stogl Robotics custom setup for nice colors and showing ROS workspace
 
@@ -85,6 +88,6 @@ function parse_ros_workspace {
   fi
 }
 
-export PS1="\[${TERMINAL_COLOR_LIGHT_GRAY}\]"'[\t]\['"\[${TERMINAL_COLOR_LIGHT_GREEN}\]"'\u\['"\[${TERMINAL_COLOR_LIGHT_GRAY}\]"'@\['"\[${TERMINAL_COLOR_BROWN}\]"'\h\['"\[${TERMINAL_COLOR_YELLOW}\]"'${text}\['"\[${TERMINAL_COLOR_LIGHT_GRAY}\]"':'"\["'$(set_ros_workspace_color)'"\]"'$(parse_ros_workspace)\['"\[${TERMINAL_COLOR_GREEN}\]"'$(parse_git_branch_and_add_brackets)>\['"\[${TERMINAL_COLOR_LIGHT_PURPLE}\]"'\W\['"\[${TERMINAL_COLOR_LIGHT_PURPLE}\]"'$\['"\[${TERMINAL_COLOR_NC}\]"'\[\e[m\] '
+export PS1="\[\e]0;"'$(parse_ros_workspace)'"\a\]\[${TERMINAL_COLOR_LIGHT_GRAY}\]"'[\t]\['"\[${TERMINAL_COLOR_LIGHT_GREEN}\]"'\u\['"\[${TERMINAL_COLOR_LIGHT_GRAY}\]"'@\['"\[${TERMINAL_COLOR_BROWN}\]"'\h\['"\[${TERMINAL_COLOR_YELLOW}\]"'${text}\['"\[${TERMINAL_COLOR_LIGHT_GRAY}\]"':'"\["'$(set_ros_workspace_color)'"\]"'$(parse_ros_workspace)\['"\[${TERMINAL_COLOR_GREEN}\]"'$(parse_git_branch_and_add_brackets)>\['"\[${TERMINAL_COLOR_LIGHT_PURPLE}\]"'\W\['"\[${TERMINAL_COLOR_LIGHT_PURPLE}\]"'$\['"\[${TERMINAL_COLOR_NC}\]"'\[\e[m\] '
 
 # END: Stogl Robotics custom setup for nice colors and showing ROS workspace
diff --git a/scripts/docker/_RosTeamWs_Docker_Defines.bash b/scripts/docker/_RosTeamWs_Docker_Defines.bash
@@ -47,7 +47,8 @@ build_docker_image () {
   --build-arg uid=$UID \
   --build-arg gid=$GROUPS \
   --build-arg home=$HOME \
-  -t "$docker_image_tag" . || { return 1; }
+  -t "$docker_image_tag" . \
+  -f "$docker_file_path" || { return 1; }
 
   cd "$prev_pwd" || { print_and_exit "Build of docker container succeeded but changing back previous working directory failed."; }
 }
@@ -73,17 +74,35 @@ create_docker_container() {
   fi
   local docker_host_name=$4
 
+  local xauth_file_name=/tmp/${docker_host_name}.docker.xauth
+
+  # BEGIN: Needed for Nvidia support
+  if [ ! -f $xauth_file_name ]
+  then
+    touch $xauth_file_name
+    xauth_list=$(xauth nlist :0 | sed -e 's/^..../ffff/')
+    if [ ! -z "$xauth_list" ]
+    then
+      echo $xauth_list | xauth -f $xauth_file_name nmerge -
+    fi
+    chmod a+r $xauth_file_name
+  fi
+  # END: Needed for Nvidia support
+
   echo "Instantiating docker image '$docker_image_tag' and mapping workspace folder to '$ws_folder'."
-  echo "ros_team_ws is located under /opt/RosTeamWS/ros_ws_${RosTeamWS_DISTRO}/src/ros_team_workspace"
   xhost +local:docker
   docker run \
   --net=host \
+  --gpus all \
   -h ${docker_host_name} \
   -e DISPLAY \
+  -e QT_X11_NO_MITSHM=1 \
+  -e XAUTHORITY=$xauth_file_name \
   --tmpfs /tmp \
+  -v "$xauth_file_name:$xauth_file_name" \
   -v /tmp/.X11-unix/:/tmp/.X11-unix:rw \
-  -v "$ws_folder":"$ws_folder":rw \
   -v "$HOME/.ssh":"$HOME/.ssh":ro \
+  -v "$ws_folder":"$ws_folder":rw \
   --name "$docker_image_tag"-instance \
   -it "$docker_image_tag" /bin/bash
 }

diff --git a/scripts/setup-ros-workspace.bash b/scripts/setup-ros-workspace.bash
@@ -5,6 +5,53 @@ setup_ws_script_own_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" > /dev/null &&
 source $setup_ws_script_own_dir/../setup.bash
 source $setup_ws_script_own_dir/docker/_RosTeamWs_Docker_Defines.bash
 
+# All the possible supported ros distributions supported by rtw
+if [ -z "$ros_distributions_20_04" ]; then
+  readonly ros_distributions_20_04=("foxy" "galactic")
+fi
+if [ -z "$ros_distributions_22_04" ]; then
+  readonly ros_distributions_22_04=("humble")
+fi
+if [ -z "$ros_distributions_20_and_22_04" ]; then
+  readonly ros_distributions_20_and_22_04=("rolling")
+fi
+
+ubuntu_20_04_version="ubuntu:20.04"
+ubuntu_20_04_tag="ubuntu_20_04"
+ubuntu_22_04_version="ubuntu:22.04"
+ubuntu_22_04_tag="ubuntu_22_04"
+
+select_normal_or_nvidia_docker() {
+  # setup Dockerfile: set correct docker version
+  echo -e "Do you want to setup a 'standard' or 'nvidia-based' docker container? [1]"
+  echo "(1) standard"
+  echo "(2) nvidia-based"
+  echo -n -e ""
+  read choice
+  choice=${choice:="1"}
+
+  docker_file=""
+  case "$choice" in
+  "1")
+    docker_file="standard.dockerfile"
+    ;;
+  "2")
+    # The version naming differs for nvidia-dockers e.g. instead of ubuntu:22.04->ubuntu22.04
+    if [[ "$ubuntu_version" == "${ubuntu_20_04_version}" ]]; then
+      ubuntu_version="ubuntu20.04"
+    elif [[ "$ubuntu_version" == "${ubuntu_22_04_version}" ]]; then
+      ubuntu_version="ubuntu22.04"
+    else
+      print_and_exit "Something went wrong. The Ubuntu version ${ubuntu_version} should be supported by nvidia but somehow it's not."
+    fi
+    ubuntu_version_tag=${ubuntu_version_tag}_nvidia
+    docker_file="nvidia.dockerfile"
+    notify_user "NOTE: Make sure that you have setup nvidia-drivers to support this!"
+    notify_user "To abort press <CTRL>+<C>, to continue press <ENTER>."
+    read
+  esac
+}
+
 check_user_input () {
   ws_path="$1"
   if [ -z "$ws_path" ] || [[ "$ws_path" == "-" ]]; then
@@ -18,31 +65,47 @@ check_user_input () {
   ws_name=$(basename ${ws_path})
 
   # ros distribution name will be set in ${ros_distro}
-  check_and_set_ros_distro_and_version "$2"
+  check_and_set_ros_distro_and_version "$2" "$use_docker"
 
-  # Todo Manuel make this more generic
   if [ "$use_docker" == true ]; then
-    if [ "$ros_distro" == "rolling" ]; then
+    # according to selected ros distro the ubuntu version is selected.
+    if [[ " ${ros_distributions_20_04[*]} " =~ " ${ros_distro} " ]]; then
+      ubuntu_version=${ubuntu_20_04_version}
+      ubuntu_version_tag=${ubuntu_20_04_tag}
+      user_decision "The ros distribution ${ros_distro} is currently only on ${ubuntu_version} supported! Continue?"
+      if [[ " ${negative_answers[*]} " =~ " ${user_answer} " ]]; then
+        print_and_exit "Aborting creation of new docker workspace. Exiting..."
+      fi
+      select_normal_or_nvidia_docker
+    elif [[ " ${ros_distributions_22_04[*]} " =~ " ${ros_distro} " ]]; then
+      ubuntu_version=${ubuntu_22_04_version}
+      ubuntu_version_tag=${ubuntu_22_04_tag}
+      user_decision "The ros distribution ${ros_distro} is currently only on ${ubuntu_version} supported! Continue?"
+      if [[ " ${negative_answers[*]} " =~ " ${user_answer} " ]]; then
+        print_and_exit "Aborting creation of new docker workspace. Exiting..."
+      fi
+      select_normal_or_nvidia_docker
+    elif [[ " ${ros_distributions_20_and_22_04[*]} " =~ " ${ros_distro} " ]]; then
       echo -e "${TERMINAL_COLOR_USER_INPUT_DECISION}'$ros_distro' is currently supported on multiple versions of Ubuntu. Which version of ubuntu would you like to choose:"
       select ubuntu_version in Ubuntu_20_04 Ubuntu_22_04;
       do
         case "$ubuntu_version" in
               Ubuntu_20_04)
-                  ubuntu_version="ubuntu:20.04"
-                  ubuntu_version_tag="ubuntu_20_04"
+                  ubuntu_version=${ubuntu_20_04_version}
+                  ubuntu_version_tag=${ubuntu_20_04_tag}
                   break
                 ;;
               Ubuntu_22_04)
-                  ubuntu_version="ubuntu:22.04"
-                  ubuntu_version_tag="ubuntu_22_04"
+                  ubuntu_version=${ubuntu_22_04_version}
+                  ubuntu_version_tag=${ubuntu_22_04_tag}
                   break
                 ;;
         esac
       done
       echo -n -e "${TERMINAL_COLOR_NC}"
+      select_normal_or_nvidia_docker
     else
-      ubuntu_version="ubuntu:20.04"
-      ubuntu_version_tag="ubuntu_20_04"
+      print_and_exit "The selected ros distribution ${ros_distro} is not supported for docker workspaces!"
     fi
   fi
 }
@@ -290,8 +353,7 @@ create_workspace_docker () {
   echo "" >> "$ws_docker_folder/bashrc"
   echo "$alias_name" >> "$ws_docker_folder/bashrc"
 
-  # setup Dockerfile: set correct docker version
-  cp "$DOCKER_TEMPLATES/Dockerfile" "$ws_docker_folder/."
+  cp "$DOCKER_TEMPLATES/$docker_file" "$ws_docker_folder/Dockerfile"
   sed -i "s/UBUNTU_DUMMY_VERSION/${ubuntu_version}/g" "$ws_docker_folder/Dockerfile"
   sed -i "s/ROS_DUMMY_VERSION/${docker_ros_distro_name}/g" "$ws_docker_folder/Dockerfile"
   sed -i "s/ROS_TEAM_WS_DUMMY_BRANCH/${rtw_branch_for_ros_distro}/g" "$ws_docker_folder/Dockerfile"