Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish images to GHCR alongside DockerHub #940

Merged
merged 1 commit into from
Oct 31, 2021
Merged

Publish images to GHCR alongside DockerHub #940

merged 1 commit into from
Oct 31, 2021

Conversation

ferferga
Copy link
Contributor

Description

Publishes the same image to GHCR alongside DockerHub.

Before merging, you must ensure that GITHUB_TOKEN have permissive permissions (the defaults). See [this article](https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token= for more information.

Motivation and Context

DockerHub is having tighter limits for pulling and GHCR is, honestly, a great alternative where a lot of containers are being published as well. It's also faster to pull from GHCR in my area than DockerHub.

How Has This Been Tested?

There's no proper way for me to test this as it's only a simple CI change and I don't have token permissions. However, I based my changes in my previous work for GHCR publishing in Jellyfin Vue and Jellyfin's packaging pipelines

Additional Info

I have a script meant to be run as a cron to sync tags that are present in DockerHub but not in GHCR here (which is really useful when DockerHub limits are reached while pulling the images for syncing). Could be used by some of the maintainers of this project to make GHCR a 1:1 mirror with DockerHub, including older images.

Also, you might want to consider implementing this another script in a cron GitHub Action that prunes images not associated with any tag (something that DockerHub does automatically, but GHCR keeps all the dangling images by default) to keep the package clean. I can make a PR after this one is merged if this behaviour is desired for you.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.

@PromoFaux
Copy link
Member

You've read my mind... I started looking at doing exactly this during the week, and then I got distracted looking at a way to combine the workflows into one (based on something @crazy-max mentioned about using the docker meta action)

You should be able to test this on your fork by merging into dev over there, but you'll need to set up the docker hub flavoured secrets, too.

Maybe we could eventually employ something similar to what I've done here, so that building and pushing to GHCR on forks is easier:

https://github.com/PromoFaux/Matterhook.NET.MatterhookClient/blob/master/.github/workflows/build.yml#L45-L51

@ferferga
Copy link
Contributor Author

ferferga commented Oct 30, 2021

@PromoFaux I assume it will work straightaway (the one that I have more doubts with is the "Test & Build workflow") as its exactly the same pipeline I used in Jellyfin Vue (refs in description).

However, I will verify as you say 👍🏻. Give me some time and I'll report back.

@ferferga
Copy link
Contributor Author

@crazy-max
Copy link

@PromoFaux When this PR is merged, I will open a new one with a single workflow as discussed.

Copy link
Member

@PromoFaux PromoFaux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

And @crazy-max - you're a superstar, thanks!

@PromoFaux PromoFaux merged commit e874dc1 into pi-hole:dev Oct 31, 2021
@ferferga
Copy link
Contributor Author

ferferga commented Oct 31, 2021

@PromoFaux Are you interested in the PR for removing dangling images on cron?

Also — reminder for the syncing script I made.

Please, can you run it locally to make GHCR a 1:1 mirror (including older images)? I would like to pin some devices to the older versions, and I prefer to have all of them pointing to GHCR instead of DockerHub.

Thank you very much for the merge!

@PromoFaux
Copy link
Member

@PromoFaux Are you interested in the PR for removing dangling images on cron?

Yep, go right ahead with that please.

I will take a look at that other script either later or tomorrow (most likely tomorrow as I'm currently operating on a hangover 😅)

@ferferga
Copy link
Contributor Author

@PromoFaux Are you interested in the PR for removing dangling images on cron?

Yep, go right ahead with that please.

I will take a look at that other script either later or tomorrow (most likely tomorrow as I'm currently operating on a hangover 😅)

Great, thank you very much!

Don't hesitate in asking me any question about it!

@ferferga
Copy link
Contributor Author

@PromoFaux Sorry for posting and annoy you here again, but I forgot to mention in the description that new packages are made private automatically when being published for the first time (at least it was like that when it was beta) 😁. Right now it's not accessible because of this so don't forget about it as well!

@PromoFaux
Copy link
Member

I think there might have been a glitch somewhere along the line... the images appear to be there now?

Regards your other script, I need to go through and tidy up what is on docker hub anyway - there are a lot of images that can be pruned (named containers that are also in multiarch images, for example), so will do that before syncing

@PromoFaux
Copy link
Member

@ferferga I am a forgetful man - I will look at this - I'm just giving docker hub a tidy up first, as there are a lot of images that can disappear (duplicate digests/test builds etc) before I run your script to sync

@crazy-max I have been a big brave boy and worked it out myself (i think!)

@PromoFaux
Copy link
Member

Ah... so running your script, I realise a major limitation - and I'm not sure how best to work around this... we need to sync multiarch. I realised after running this on my desktop that it was only syncing amd64. Any ideas?

@PromoFaux
Copy link
Member

PromoFaux commented Nov 18, 2021

For anyone reading in the future: https://github.com/containers/skopeo

And the command that worked: skopeo copy docker://pihole/pihole:latest docker://ghcr.io/pi-hole/pihole:latest --all

Worked that into your script @ferferga, and it is happily syncing now 🎉

@ferferga
Copy link
Contributor Author

@PromoFaux Thank you very much! For the proposal and very good catch!

Yes, as I used the normal docker pull, it obviously pulled the image for the architecture of the system that's running. That was something I didn't thought of.

Thank you very much again as it will be very useful for us as well at Jellyfin.

@PromoFaux
Copy link
Member

I actually got the idea after reading some of the comments/suggestions on your abovementioned PRs :)

@crazy-max
Copy link

@PromoFaux
Copy link
Member

Thank you, I will look over them when I am next at a real computer. Is there anything super urgent you think needs addressing?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants