Skip to content

Commit

Permalink
Update documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
piotr-grodek-dsai committed Nov 3, 2023
1 parent a9592b9 commit 418d1a0
Show file tree
Hide file tree
Showing 6 changed files with 103 additions and 29 deletions.
3 changes: 2 additions & 1 deletion DESCRIPTION.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
`deepsense.ai <https://deepsense.ai>`_ **ds_pycontain**
-------------------------------------------------------------

**ds_pycontain** is a small python package to help with docker containers and images.
**ds_pycontain** is a small python package to help with docker containers and images and provide Python REPL running in a docker container.

Example use case you might consider is to isolate python code execution generated by untrusted LLM by running it in a docker container.

Expand All @@ -11,3 +11,4 @@ This package makes it a bit easier to:
* Pull docker images from dockerhub (or similar).
* Run docker container to execute a one-off command.
* Run docker container to execute a long-running process and communicate with it.
* Run python commands in a container and get the result.
8 changes: 8 additions & 0 deletions docs/api/ds_pycontain.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,14 @@ ds_pycontain.docker_containers module
:undoc-members:
:show-inheritance:

ds_pycontain.python_dockerized_repl module
----------------------------------------------------

.. automodule:: ds_pycontain.python_dockerized_repl
:members:
:undoc-members:
:show-inheritance:

Module contents
---------------

Expand Down
25 changes: 1 addition & 24 deletions docs/code_documentation.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,6 @@
# Code documentation

**ds_pycontain** is a python package which provides an abstraction over the docker API.

Supported functionality covers:
- Building docker images from Dockerfiles
- Pulling docker images from dockerhub (or similar)
- Running docker containers to execute a one-off command
- Running docker containers to execute a long-running process and communicate with it


```python
from ds_pycontain import DockerContainer, DockerImage, get_docker_client

client = get_docker_client()

# This will fetch the image from dockerhub if it is not already present
# with the "latest" tag. Then container is started and commands are run
with DockerContainer(DockerImage.from_tag("alpine")) as container:
ret_code, output = container.run("touch /animal.txt")
assert ret_code == 0

ret_code, output = container.run("ls /")
assert ret_code == 0
assert cast(bytes, output).find(b"animal.txt") >= 0
```
This is the documentation for the code of the project API.

```{toctree}
---
Expand Down
5 changes: 2 additions & 3 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,15 @@
`deepsense.ai <https://deepsense.ai>`_ - pycontain
====================================================================================================================

Documentation for **ds_pycontain** python package to work with docker containers and images.

Example use case you might consider is to isolate python code execution generated by untrusted LLM by running it in a docker container.
Documentation for **ds_pycontain** python package to work with docker containers and images, as well as providing python REPL running in a container.

This package makes it a bit easier to:

* Build docker images from Dockerfiles or in-memory string.
* Pull docker images from dockerhub (or similar).
* Run docker container to execute a one-off command.
* Run docker container to execute a long-running process and communicate with it.
* Run python code in a docker container and communicate with it.


.. toctree::
Expand Down
1 change: 1 addition & 0 deletions docs/licenses.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,6 @@ Licenses
=====================

List of automatically detected licenses of all detected python packages with `pip-licenses`.
They are all projects that are used in the project development.

.. include:: licenses_table.rst
90 changes: 89 additions & 1 deletion docs/project_overview.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,94 @@
# Project overview

**ds_pycontain** is a python package which provides an abstraction over the docker API and provide Python REPL running in a docker container.

Supported functionality covers:
- Building docker images from Dockerfiles
- Pulling docker images from dockerhub (or similar)
- Running docker containers to execute a one-off command
- Running docker containers to execute a long-running process and communicate with it
- Run python commands in a container and get the result.

## Motivation

Main motivation is to allow to orchestrate running unsafe code or commands in isolated environment.
The docker API is quite complicated and not well documented or typed.
This project aims to provide a higher level abstraction over the docker API.

Main motivation is to allow to orchestrate running unsafe code or commands in isolated environment.
What is also provided is **a python REPL running in a docker container**.

This might be useful to improve security for execution of LLM models/agents generated code, which generally should not be trusted.

## Example code snippets

### Execute commands in container running in the background:

Below is a short snippet showcasing how to run docker container in the background and execute commands in it.

```python
from ds_pycontain import DockerContainer, DockerImage, get_docker_client

client = get_docker_client()

# This will fetch the image from dockerhub if it is not already present
# with the "latest" tag. Then container is started and commands are run
with DockerContainer(DockerImage.from_tag("alpine")) as container:
ret_code, output = container.run("touch /animal.txt")
assert ret_code == 0

ret_code, output = container.run("ls /")
assert ret_code == 0
assert cast(bytes, output).find(b"animal.txt") >= 0
```

### Docker images

Images can be pulled from dockerhub or built from dockerfile.

```python
from ds_pycontain import DockerImage

# pull or use alpine:latest
image = DockerImage.from_tag("alpine")
# use provided tag to pull/use the image
image = DockerImage.from_tag("python", tag="3.9-slim")
# use this dockerfile to build a new local image
image = DockerImage.from_dockerfile("example/Dockerfile")
# you can provide a directory path which contains Dockerfile, set custom image name
image = DockerImage.from_dockerfile("path/to/dir_with_Dockerfile/", name="cow")
```

### Python REPL running in docker container

Running Python code in docker container is rather easy with this package.

```python
from ds_pycontain.python_dockerized_repl import PythonContainerREPL

# To start python REPL in container it is easy,
# just be aware that it will take some time to start the container
# and ports might be allocated by OS, so use different port/retry
# if you get error.
repl = PythonContainerREPL(port=7121)

# You can run python commands in the container
# and it will keep state between commands.
out1 = repl.exec("x = [1, 2, 3]")
assert out1 == ""
# Eval returns string representation of the python command
# as it would be in python REPL:
out2 = repl.eval("len(x)")
assert out2 == "3"

# Exec returns captured standard output (stdout)
# so it won't return anything in this case:
out3 = repl.exec("len(x)")
assert out3 == ""
# but exec with print works:
out4 = repl.exec("print(len(x))")
assert out4 == "3\n"

# You can also get error messages if code is wrong:
err = repl.exec("print(x")
assert "SyntaxError" in err
```

0 comments on commit 418d1a0

Please sign in to comment.