Cannot use shell form for command specified in Compose file #12403
Comments
|
This is the expected behavior |
|
For completeness - why is this intended behavior? |
|
Basically: preserve backward compatibility |
|
@ijessen-mitll your reproduction has the Dockerfile service build an image with I don't think your comparison was correct either, so I've provided an extended version with heavy commentary for context (in case anyone else lands here). Main issue with your comparison was the Observations
Here's a table of the different variants from the revised
The blank line examples seem to be something like this: docker run --rm alpine /bin/sh -c '/bin/sh -c echo "host is $HOSTNAME"'However I kind of expected
Reproductiondocker compose up --force-recreate --buildservices:
# NOTE: The shell-form variants below would fail YAML parsing with:
# "mapping values are not allowed in this context"
# - Workaround Variant A => single quote wrapping the full value
# - Workaround Variants B + C => Replace `host:` with `host is`
# No shell is used, so no interpolation of the variable HOSTNAME.
# https://docs.docker.com/reference/dockerfile/#variable-substitution
# NOTE: The Alpine image has no default entrypoint set, so the command runs without a shell.
# This differs from the Dockerfile CMD example below, which with shell-form uses SHELL:
# https://docs.docker.com/reference/dockerfile/#shell
# NOTE: `echo` is a built-in command in the shell,
# /bin/echo is also available as a separate command which is called here,
# Docker locates the binary to execute via the PATH environment:
# NOTE: The equivalent via docker CLI would instead output the Docker host hostname:
# `docker run --rm alpine echo "host: $HOSTNAME"`
shell-form-a:
image: alpine:latest
command: 'echo "host: $$HOSTNAME"'
# Explicit use of shell, interpolation is valid:
shell-form-b:
image: alpine:latest
command: /bin/sh -c 'echo "host is $$HOSTNAME"'
# Same as shell-form-b, except an entrypoint configures a shell explicitly.
# NOTE: This actually fails and outputs a blank output for some reason,
# expectation was for the command to be appended to the entrypoint as a string arg for `-c`.
shell-form-c:
image: alpine:latest
# Blank output (exec + shell maybe incompatible?):
# NOTE: Potentially because there is no equivalent CLI command for exec-form with `--entrypoint`?
entrypoint: ["/bin/sh", "-c"]
command: echo "host is $$HOSTNAME"
# Correct output:
# NOTE: Equivalent via CLI: `docker run --rm --entrypoint /bin/sh alpine -c 'echo "host is $HOSTNAME"'`
#entrypoint: ["/bin/sh"]
#command: -c 'echo "host is $$HOSTNAME"'
# Equivalent of shell-form-a output (no shell environment, no interpolation):
exec-form-a:
image: alpine:latest
command: ["echo", "host: $$HOSTNAME"]
# Equivalent of shell-form-b or shell-form-c, uses a shell thus interpolation is valid:
exec-form-b:
image: alpine:latest
command: ["/bin/sh", "-c", "echo host: $$HOSTNAME"]
# Interoplation by shell works as intended with double quotes instead,
# exec-form can also use either single or double quotes for it's YAML string values:
# NOTE: Unlike shell-form-c, this is valid due to entrypoint + command using exec form.
exec-form-c:
image: alpine:latest
entrypoint: ["/bin/sh", "-c"]
command: ['echo "host: $$HOSTNAME"']
# Despite using a shell, single quote wrapping around the variable has
# the shell parse it as a string literal (skips interpolation):
exec-form-d:
image: alpine:latest
entrypoint: ["/bin/sh", "-c"]
command: ["echo 'host: $$HOSTNAME'"]
# This example highlights the implicit `SHELL ["/bin/sh", "-c"]` for shell-form with CMD,
# Docker CLI or Compose can both replace CMD at runtime which ignores SHELL:
# NOTE: Uncomment the SHELL instruction to produce a failure at runtime.
dockerfile-shell-form-a:
build:
dockerfile_inline: |
FROM alpine:latest
#SHELL ["/bin/invalid"]
CMD echo "host: $$HOSTNAME"
# Exits, no output
# NOTE: If ENTRYPOINT used shell-form, then CMD would be ignored (as would a runtime command override):
# https://docs.docker.com/reference/dockerfile/#entrypoint
dockerfile-shell-form-b:
build:
dockerfile_inline: |
FROM alpine:latest
ENTRYPOINT ["/bin/sh", "-c"]
CMD echo "host: $$HOSTNAME"
# Same output as shell-form-a and exec-form-a: needs a shell environment to work.
dockerfile-exec-form-a:
build:
dockerfile_inline: |
FROM alpine:latest
CMD ["echo", "host: $$HOSTNAME"]
# REFERENCE: The exec-form with CMD is not affected by SHELL:
# NOTE: The same exemption applies shell vs exec forms for RUN and ENTRYPOINT too.
# If either ENTRYPOINT or CMD use shell-form, those will still default to SHELL:
# https://docs.docker.com/reference/dockerfile/#understand-how-cmd-and-entrypoint-interact
dockerfile-exec-form-b:
build:
dockerfile_inline: |
FROM alpine:latest
SHELL ["/bin/invalid"]
CMD ["echo", "host: $$HOSTNAME"]
# Same output as shell-form-c, blank.
# EDIT: This has an error, it should have had a CMD like exec-form-c,
# providing a single value for `-c`.
dockerfile-exec-form-c:
build:
dockerfile_inline: |
FROM alpine:latest
ENTRYPOINT ["/bin/sh", "-c"]
CMD ["echo", "host: $$HOSTNAME"] |
|
Follow-up to previous comment addressing the Expected output should be like For Compose the mistake with shell-form syntax is that Compose always deconstructs the input into exec-form style. For shell strings that makes it important when the image or compose config has an entrypoint configured for a shell (or similar) to wrap the command in quotes so it's used as a single string arg. When no shell environment is relied upon, the direct command without an entrypoint can work well with the command split/converted into exec-form. services:
# Good
shell-form-a:
image: alpine:latest
command: uname -m -o
# Good
# Unlike ENTRYPOINT, the shell-form is technically treated as the exec-form here,
# there is no implicit use of SHELL instruction and the command is not ignored.
shell-form-b:
image: alpine:latest
entrypoint: uname
command: -m -o
# Good
# shell-form vs exec-form doesn't really exist in compose equivalents:
shell-form-c:
image: alpine:latest
entrypoint: ["uname"]
command: -m -o
# Bad
# Outputs only 'Linux', mimics `docker run --rm alpine /bin/sh -c uname`
# Cause: The command is converted to exec-form, which is fine for CLI args
# but in this case command is intended as a single value to `/bin/sh -c` arg,
# where it only receives `uname` due to splitting the command string into separate components.
shell-form-d:
image: alpine:latest
entrypoint: ["/bin/sh", "-c"]
command: uname -m -o
# Good
# When the entrypoint sets a shell, shell-form syntax needs to ensure the command is provided to `-c` as a single string,
# In compose and docker CLI, you can ensure this with quote wrapping, this varies for compose due to YAML inference of strings.
shell-form-e:
image: alpine:latest
entrypoint: ["/bin/sh", "-c"]
# Preserve the single quote:
#command: "'uname -m -o'"
# Multi-line string does not require nested quotes
command: |
'uname -m -o'
# Good
# No entrypoint/shell, so the exec-form should have the command properly deconstructed:
exec-form-a:
image: alpine:latest
command: ["uname", "-m", "-o"]
# Good
# Since the entrypoint ends with the option `-c`, a single string value is expected,
# Thus exec-form clearly treats it as such:
exec-form-b:
image: alpine:latest
entrypoint: ["/bin/sh", "-c"]
command: ["uname -m -o"]
# Good
# Alternative exec-form syntax (YAML list), with chaining of other commands in the shell:
exec-form-c:
image: alpine:latest
#command: ["/bin/sh", "-c", "uname -m -o && echo hello && echo world"]
command:
- /bin/sh
- -c
- uname -m -o && echo hello && echo world
# Good
# This style of exec-form with YAML lists can use multi-line strings,
# which for `sh -c` is a nicer way to chain multiple commands vs `&&`
exec-form-d:
image: alpine:latest
command:
- /bin/sh
- -c
- |
uname -m -o
echo hello
echo world
# Good
# Unlike Compose or the Docker CLI, shell-form in Dockerfile provides a string to implicit SHELL:
dockerfile-shell-form-a:
build:
dockerfile_inline: |
FROM alpine:latest
CMD uname -m -o
# Bad
# Fails to run command, mimics: `docker run --rm alpine /bin/sh -c /bin/sh -c uname -m`
# Cause: Effectively runs `/bin/sh -c /bin/sh`, ignoring the rest.
# If using `tty: true` container doesn't immediately exit, the tty for the shell keeps it open instead.
dockerfile-shell-form-b:
build:
dockerfile_inline: |
FROM alpine:latest
ENTRYPOINT ["/bin/sh", "-c"]
CMD uname -m -o
# Bad
# Outputs an arg parsing error from uname, mimics: `docker run --rm alpine uname -m /bin/sh -c -o -p`
# Cause: Implicit SHELL prepended due to using CMD shell-form syntax
dockerfile-shell-form-c:
build:
dockerfile_inline: |
FROM alpine:latest
ENTRYPOINT ["uname", "-m"]
CMD -o -p |
|
@polarathene your analysis is 100% correct, still this is a corner case and nobody complained about this until this issue has been opened, while compose is 10 years old :) |
|
Yeah no worries. I don't think it's a compose specific issue (other than being able to use Both seem to process the value of |
Description
A command override in a Compose service definition will never execute in shell form, only ever exec form. This is contrary to the documentation statement that "the value can also be a list, in a manner similar to Dockerfile". A Dockerfile CMD specified as a list triggers exec form, while a Dockerfile CMD specified as a plain string triggers shell form (see here).
Steps To Reproduce
Example
compose.yaml:Results (notice the difference between the compose shell form output from the other two):
Compose Version
Docker Environment
Anything else?
If the behavior is as-intended, the Compose reference documentation should be updated to clarify that a string command will be executed in exec mode (contrary to the Dockerfile behavior).
The text was updated successfully, but these errors were encountered: