PATCH: read {path}: file already closed error following update of go-containerregistry to v0.15.2

[parvathikp100]

Describe the bug

We recently upgraded our go-containerregistry version from v0.11.0 to v0.15.2 and there has been a sudden surge in the number of the following

Patch "https://xxx/blobs/uploads/xxx?_state=xxx%3D%3D": read /xxx/oci/blobs/sha256/xxx: file already closed

error which seem to be originating from the http.PATCH request in streamBlob function

Also interesting is to note that similar surge in errors have been seen in http requests being made from commitManifest and initiateUpload

The major change that is common to streamBlob, initiateUpload and commitManifest is the addition of retry.Never(ctx) which seems to be affecting the retries somehow?

This is the PR that introduced it - #1502 (comment)

To Reproduce

No clue on reproducing it yet, problem seems to be insufficient retries if request fails due to some transient issue because these requests have passed sometimes for the same blobs

Expected behavior

Blobs are written successfully without errors at any of the http requests

Additional context

Added a comment on the original PR with some more details - #1502 (comment)

• Output of crane version
• Registry used (e.g., GCR, ECR, Quay)

[parvathikp100]

I also added a comment on this PR that introduced the retry predicate - #1502 (comment).

[parvathikp100]

Also see a rise in errors for other http requests originating from uploadOne():

Put "https://xxx/manifests/sha256:xxx": http: ContentLength=2706 with Body length 0

PUT https://xxx/blobs/uploads/xxx?_state=REDACTED&digest=sha256%3Axxx: DIGEST_INVALID: provided digest did not match uploaded content; map[Digest:sha256:xxx Reason:map[]]

[parvathikp100]

@jonjohnsonjr can you help look into this? Do you think our suspicions regarding what could have caused this surge in failures is valid?

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Keep fresh with the 'lifecycle/frozen' label.