How to run tasks from pre-built docker images

[nikiniki1]

Hello!
I have a pre-built docker image with all dependencies that I need, so I want to run a task in it, without downloading additional dependencies.
So I initialized clearml agent daemon and redefine default_docker_image.
As example, I created a test task:

from clearml import Task
from tqdm import tqdm
import time

task = Task.init('test_proj', 'test_job')

lst = list(range(5))

for i in tqdm(lst):
    time.sleep(1)

Next, in the web, I selected my task and enqueued it.
Аfter that clearml starts running the task and tries to install dependencies into my container, but I don't need that and I want it to just execute the code in it.

docker run -t --gpus all -l clearml-worker-id=DESKTOP-V7BPCPO:0 -l clearml-parent-worker-id=DESKTOP-V7BPCPO:0 -e CLEARML_WORKER_ID=DESKTOP-V7BPCPO:0 -e CLEARML_DOCKER_IMAGE=learning-library -e CLEARML_TASK_ID=5cddd0d7aa744ce48bed9ba867c6fbf1 -v /tmp/.clearml_agent.dgpuml_i.cfg:/tmp/clearml.conf -e CLEARML_CONFIG_FILE=/tmp/clearml.conf -v /home/niki/.clearml/apt-cache:/var/cache/apt/archives -v /home/niki/.clearml/pip-cache:/root/.cache/pip -v /home/niki/.clearml/pip-download-cache:/root/.clearml/pip-download-cache -v /home/niki/.clearml/cache:/clearml_agent_cache -v /home/niki/.clearml/vcs-cache:/root/.clearml/vcs-cache -v /home/niki/.clearml/venvs-cache:/root/.clearml/venvs-cache --rm learning-library bash -c echo \'Binary::apt::APT::Keep-Downloaded-Packages "true";\' > /etc/apt/apt.conf.d/docker-clean ; chown -R root /root/.cache/pip ; export DEBIAN_FRONTEND=noninteractive ; export CLEARML_APT_INSTALL="$CLEARML_APT_INSTALL libsm6 libxext6 libxrender-dev libglib2.0-0" ; [ ! -z $(which git) ] || export CLEARML_APT_INSTALL="$CLEARML_APT_INSTALL git" ; declare LOCAL_PYTHON ; [ ! -z $LOCAL_PYTHON ] || for i in {15..5}; do which python3.$i && python3.$i -m pip --version && export LOCAL_PYTHON=$(which python3.$i) && break ; done ; [ ! -z $LOCAL_PYTHON ] || export CLEARML_APT_INSTALL="$CLEARML_APT_INSTALL python3-pip" ; [ -z "$CLEARML_APT_INSTALL" ] || (apt-get update -y ; apt-get install -y $CLEARML_APT_INSTALL) ; [ ! -z $LOCAL_PYTHON ] || export LOCAL_PYTHON=python3 ; $LOCAL_PYTHON -m pip install -U "pip<20.2 ; python_version < \'3.10\'" "pip<22.3 ; python_version >= \'3.10\' and python_version <= \'3.11\'" "pip>=23,<24.3 ; python_version >= \'3.12\'" ; $LOCAL_PYTHON -m pip install -U clearml-agent ; cp /tmp/clearml.conf ~/default_clearml.conf ; NVIDIA_VISIBLE_DEVICES=all $LOCAL_PYTHON -u -m clearml_agent execute --disable-monitoring  --id 5cddd0d7aa744ce48bed9ba867c6fbf1']

[ainoam]

@nikiniki1 The 'Python Packages' section of the task records the python environment, and tells the agent what to install.
Before enqueuing the task - reset the 'Python Packages' section so the agent will not install anything.

[nikiniki1]

@ainoam Thanks! It slipped my mind)
But docker runs with some updating and installations (e.g. apt-get update, installs libsm6, libxext6, libxrender-dev, libglib2.0-0 etc). Still, clearml, although it does not try to install python dependencies, connects to the server and wants to install system dependencies, which I cannot install in any way (agents are isolated from net), but at the same time my docker image is configured including these system dependencies. In other words, there is no need to check and re-install dependencies.
P.S. It would be easier for me to configure the docker container launch command myself, but I don’t quite understand where in the clearml-agent code I can do this

[jkhenning]

@nikiniki1 to disable installing packages inside a docker you can pass the CLEARML_AGENT_SKIP_PIP_VENV_INSTALL or the CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL environment variables (see here)