Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add error message when docker isn't running #2942

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

dansola
Copy link
Contributor

@dansola dansola commented Nov 20, 2024

Why are the changes needed?

When a user isn't running docker, it is unclear when they see the logs:

Running Execution on Remote.
Failed to check if the image exists with error:
 Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
Flytekit assumes the image ghcr.io/dansola/test-image:ppjeS9H6wavg5FvXbowKEQ already exists.

Ideally we'd have more clear logs when we can narrow down that the issue is that docker isn't running.

What changes were proposed in this pull request?

Added additional logging.

How was this patch tested?

Did not run docker, ran pyflyte run --remote with an ImangeSpec, observed the log message:

Running Execution on Remote.
Unable to connect to Docker daemon. Please ensure Docker is installed and running.
You can start Docker by:
  - Windows/Mac: Start the Docker Desktop application
  - Linux: Run 'sudo systemctl start docker'
Flytekit assumes the image ghcr.io/dansola/test-image:ORI1BZVtVKVludLUJsq_Rg already exists.

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

@@ -278,6 +278,16 @@ def exist(self) -> Optional[bool]:
f" pip install --upgrade docker"
)

if isinstance(e, DockerException) and "Error while fetching server API version" in str(e):
click.secho(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the image exists in your ghcr.io registry and docker is not running, then that should still work. This message tells a user to install Docker, but it is not strictly required.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup. But I don't think we want to hide the fact that docker should generally be running. If it's not clear, the user will eventually run into images that don't exist in their registry and they will get image pull failures they don't understand. This happened today.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe I should change the "error message" wording in the PR title, but this change doesn't block images being found and used if the docker is off but the image exists

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's write out these details in the message. Something like: "Docker is not running and flytekit will assume that the image exists in your registry. If you want to build the image, please install docker: ..."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, will do!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I was going to change the click.secho(f"Flytekit assumes the image {img_name} already exists.", fg="blue") in should_build(), but that could occur even if docker is running, for example an APIError that returns None.

The current suggestion actually mentions that docker isn't running if it catches the exact error docker throws, then the Flytekit assumes the image ghcr.io/dansola/test-image:ORI1BZVtVKVludLUJsq_Rg already exists. log still comes up after.

Do you think it's not clear that those two are associated? Happy to keep it general and just toss in the mention of docker in should_build(), but I just don't want to sound that alarm if it isn't the case.

Copy link

codecov bot commented Nov 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.00%. Comparing base (faee3da) to head (af94679).
Report is 1 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #2942       +/-   ##
===========================================
+ Coverage   79.32%   94.00%   +14.68%     
===========================================
  Files         199       38      -161     
  Lines       20870     1768    -19102     
  Branches     2684        0     -2684     
===========================================
- Hits        16555     1662    -14893     
+ Misses       3566      106     -3460     
+ Partials      749        0      -749     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

@@ -278,6 +278,16 @@ def exist(self) -> Optional[bool]:
f" pip install --upgrade docker"
)

if isinstance(e, DockerException) and "Error while fetching server API version" in str(e):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I'd like to see the original error still.

Suggested change
if isinstance(e, DockerException) and "Error while fetching server API version" in str(e):
click.secho(f"Failed to check if the image exists with error:\n {e}", fg="red")
if isinstance(e, DockerException) and "Error while fetching server API version" in str(e):

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I'll make sure the original error is still shown.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants