Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the messages for notation verify #620

Open
toddysm opened this issue Apr 6, 2023 · 4 comments
Open

Improve the messages for notation verify #620

toddysm opened this issue Apr 6, 2023 · 4 comments
Assignees
Labels
error message Issues related to error message improvement need discussion UX User experience changes
Milestone

Comments

@toddysm
Copy link
Contributor

toddysm commented Apr 6, 2023

What is the areas you experience the issue in?

Notation CLI

What is not working as expected?

The following two output messages from notation verify look very similar although one is successful and the other is error:

$ notation verify $APPLICATION_IMAGE
Warning: Always verify the artifact using digest(@sha256:...) rather than a tag(:kubeconeu-demo-v1) because resolved digest may not point to the same signed artifact, as tags are mutable.
Successfully verified signature for ghcr.io/toddysm/flasksample@sha256:b4c0ed46d75ff7a1e8bcc166156db356dbb3768181d010a639ad4b4dd2f43db7
$ notation verify $TEST_IMAGE
Warning: Always verify the artifact using digest(@sha256:...) rather than a tag(:kubeconeu-demo-v1) because resolved digest may not point to the same signed artifact, as tags are mutable.
Error: signature verification failed: artifact "ghcr.io/toddysm/net-monitor@sha256:8c7a3bba4f87b1b9dc01e74b0a3b29cfa3fe492c320d536de87ce1ff67072442" has no applicable trust policy

What did you expect to happen?

We should rework those to make it obvious what is the result of the command. Right now they both start with the warning and this is the first thing the user sees. The user is more interested in the result of the command than the warning. An example can be:

SUCCESS: Successfully verified for signature for artifact ghcr.io/toddysm/flasksample@sha256:b4c0ed46d75ff7a1e8bcc166156db356dbb3768181d010a639ad4b4dd2f43db7

Warning: Always verify the artifact using digest(@sha256:...) rather than a tag(:kubeconeu-demo-v1) because resolved digest may not point to the same signed artifact, as tags are mutable.

and

FAILURE: Failed to verify the signature for artifact "ghcr.io/toddysm/net-monitor@sha256:8c7a3bba4f87b1b9dc01e74b0a3b29cfa3fe492c320d536de87ce1ff67072442". There is no applicable trust policy for that scope.

Warning: Always verify the artifact using digest(@sha256:...) rather than a tag(:kubeconeu-demo-v1) because resolved digest may not point to the same signed artifact, as tags are mutable.

Adding white space and moving the warning to secondary provides better usability. Also, the current messages are not consistent in the language and not capitalized and grammatically correct.

How can we reproduce it?

Use the commands above

Describe your environment

N/A

What is the version of your Notation CLI or Notation Library?

Notation: Notary v2, A tool to sign, store, and verify artifacts.

Version: 1.0.0-rc.3
Go version: go1.20.1
Git commit: 233c0ea

@toddysm toddysm added bug Something isn't working UX User experience changes triage Need to triage labels Apr 6, 2023
@yizha1 yizha1 added this to the 1.0.0 milestone Apr 26, 2023
@yizha1 yizha1 removed the triage Need to triage label Apr 26, 2023
@yizha1
Copy link
Contributor

yizha1 commented May 19, 2023

Move to post v1 release, as it is not critical for v1 release cc @vaninrao10

@yizha1 yizha1 modified the milestones: 1.0.0, future May 19, 2023
@FeynmanZhou FeynmanZhou modified the milestones: Future, 1.2.0 Apr 12, 2024
@FeynmanZhou
Copy link
Member

Moved this issue to v1.2.0 milestone and linking #834 as we are improving all error messages in v1.2.0.

@FeynmanZhou
Copy link
Member

FeynmanZhou commented Apr 12, 2024

Move the status message to the first line sounds reasonable because in general users care about the operation result at the first. Warning message is a follow-up reminder that users should be paying attention to.

For the operation status, I think it is not necessary to show SUCCESS or FAILURE at the beginning because when operation succeeded or failed, the error message starts with Successfully xxx or Error:. This is obvious for users.

How about improve the success and error messages as follows?

Successfully verified for signature for artifact ghcr.io/toddysm/flasksample@sha256:b4c0ed46d75ff7a1e8bcc166156db356dbb3768181d010a639ad4b4dd2f43db7
Warning: Always verify the artifact using digest(@sha256:...) rather than a tag(:kubeconeu-demo-v1) because resolved digest may not point to the same signed artifact, as tags are mutable. 
Error: Failed to verify the signature for artifact "ghcr.io/toddysm/net-monitor@sha256:8c7a3bba4f87b1b9dc01e74b0a3b29cfa3fe492c320d536de87ce1ff67072442". There is no applicable trust policy for that scope.
Warning: Always verify the artifact using digest(@sha256:...) rather than a tag(:kubeconeu-demo-v1) because resolved digest may not point to the same signed artifact, as tags are mutable.

@JeyJeyGao
Copy link
Contributor

Warning messages may be generated dynamically during the verification process. Any delay in displaying these warning messages will prevent users from knowing the current status of the verification. Are there any well-known CLIs that also delay showing warnings, so we can follow best practices?
@FeynmanZhou @toddysm

@yizha1 yizha1 modified the milestones: 1.2.0, 1.3.0 Jul 12, 2024
@yizha1 yizha1 added the error message Issues related to error message improvement label Jul 12, 2024
@JeyJeyGao JeyJeyGao added the triage Need to triage label Oct 16, 2024
@yizha1 yizha1 added need discussion and removed triage Need to triage labels Oct 16, 2024
@FeynmanZhou FeynmanZhou removed the bug Something isn't working label Jan 14, 2025
@FeynmanZhou FeynmanZhou modified the milestones: 1.3.0, 2.0.0 Jan 14, 2025
@FeynmanZhou FeynmanZhou modified the milestones: 2.0.0, 2.1.0 Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
error message Issues related to error message improvement need discussion UX User experience changes
Projects
Status: Todo
Development

No branches or pull requests

4 participants