Discover and document why AppArmor and containers are a difficult combination

[timmc-edx]

Acceptance Criteria:

• Answer the following question:
  • Can an AppArmor profile be used on a subprocess in a docker container?
• Document the answer and potentially update plan/tickets (potentially ADR).
• Create follow-up tickets.

Notes:

It is received wisdom that edxapp cannot be containerized due to the presence of AppArmor in codejail. Check if this is still true (maybe the situation has changed) and document the results of our findings, probably in the codejail repo.

Leads:

• Kubernetes Support eduNEXT/tutor-contrib-codejail#24
• ["FEAT"] Support for K8s deployments eduNEXT/codejailservice#15

[timmc-edx]

I've discovered one possible reason I can't get apparmor working: On Ubuntu, /etc/apparmor/logprof.conf is set up to limit how the python executables can be affected by apparmor.

EDIT: Ah! I need to add --copies to the virtualenv setup.

[timmc-edx]

Some things I've learned:

• AppArmor profiles are kernel-level, so they're shared between the host and the container
• Profiles cannot be adjusted from inside the container without the container having elevated privileges
• A Docker container as a whole can have a profile applied, and that profile can restrict access to files in the container's filesystem (confining all processes)

Useful resources:

• Worked example of applying apparmor profile to a container as a whole: https://docs.docker.com/engine/security/apparmor/
• Fairly thorough documentation on AppArmor from SUSE: https://documentation.suse.com/sled/15-SP4/html/SLED-all/part-apparmor.html

Open questions:

• Can a profile loaded on the host confine executables in the container?
  • Profiles match executables by path, and paths are different inside the container. Can in-container executables even be matched? Probably, since in-container file resources can be matched successfully.
  • A container-level profile might be able to hand-off to subprocesses and confine those according to secondary profiles. This is probably the correct approach to try.

[timmc-edx]

The answer turns out to be yes, you can do that! Here's a demo of confining a single executable inside a container:

Demo

In host

sudo emacs -nw /etc/apparmor.d/experiment-docker-block-bin

#include <tunables/global>
profile docker-block-bin flags=(attach_disconnected, mediate_deleted) {
  #include <abstractions/base>

  network inet tcp,
  network inet udp,
  network inet icmp,

  file,

  /edx/app/codejail/venv/bin/python Cx -> sandbox,

  profile sandbox flags=(attach_disconnected, mediate_deleted) {
    file,
    deny /etc/** r,
  }
}

• Run sudo apparmor_parser /etc/apparmor.d/experiment-docker-block-bin to load it.
• Add security_opt: ['apparmor=docker-block-bin'] to the lms service in Devstack's docker-compose.yml
• make lms-up

In lms-shell

git clone https://github.com/openedx/codejail /edx/app/codejail
cd /edx/app/codejail
python3.8 -m venv --copies venv
./venv/bin/pip install -r requirements/sandbox.txt

Probe to test confinement:

• /edx/app/codejail/venv/bin/python
• open('/etc/passwd').read()

---

...and it turns out we had this solution all along:

https://github.com/eduNEXT/tutor-contrib-codejail/blob/main/tutorcodejail/templates/codejail/apps/profiles/docker-edx-sandbox