Fix "no implicit conversion of Hash into Integer" error when running docker-compose up locally

[ericvennemeyer]

Overview

As a new volunteer developer, I want to be able to run docker-compose up without encountering a "no implicit conversion of Hash into Integer" error, so that I can run the website locally using Docker.

For this issue, we will be investigating and finding a solution to the "no implicit conversion of Hash into Integer" error.

Details

Several new volunteers have experienced a "no implicit conversion of Hash into Integer" error in Docker while trying to run docker-compose up in their local development environments. A temporary fix has been found, but we need to find a permanent/sustainable solution in order to prevent this error from occurring in the future.

Initial research suggests there is a conflict between the most recent version of Ruby (3.x) and Jekyll 3.9 and higher. The reason for this is not currently known, but the temp fix involves modifying the "image" keyword in the docker-compose.yml file (located in the root of the website directory) from:

image: jekyll/jekyll:pages

to:

image: jekyll/jekyll:4.2.0

This will tell Docker to pull a Jekyll image that uses an older version of Ruby (2.7). However, this will cause a conflict when merging a local branch with gh-pages, and may also create security vulnerabilities in the website.

Action Items

• Research changes between Ruby 2.7 and Ruby 3.x. Determine whether any address critical security issues.
• Briefly research changes between Jekyll 3.9 and earlier versions.
• (If needed) dig into current vs. earlier versions of the Jekyll image on Docker Hub, see what changes have been made and how they might affect our website
• Look into whether the inherit bug causing the conflict between Ruby 3.x and Jekyll is being addressed
• Once a potential solution is found, check with team leads to confirm it will not create issues for the website
  • After conferring with the leads, implement the solution in a topic branch in your forked repo
  • Open a pull request

Resources/Instructions

Official Jekyll Docker image page on Docker Hub

Jekyll-Docker repo on GitHub

Links to various posts by other users experiencing same/similar issues:
envygeeks/jekyll-docker#335
https://talk.jekyllrb.com/t/error-no-implicit-conversion-of-hash-into-integer/5890
https://stackoverflow.com/questions/66113639/jekyll-serve-throws-no-implicit-conversion-of-hash-into-integer-error

[ericvennemeyer]

@SAUMILDHANKAR @JessicaLucindaCheng @tamara-snyder @answebdev just created a new issue for the Docker error that new volunteers have been experiencing. As this is my first new issue, please let me know if you have any suggestions as to how it could be improved.

Progress: Determined a temp fix, but still looking into permanent solution.
Availability: Wednesday 4/20: 2hrs, Thursday 4/21: 2hrs
ETA: Hopefully by early next week.

Reminder: I will not be able to attend any hfla meetings this week. The next meeting I can attend will be Tuesday, 4/26. Thanks!

[JessicaLucindaCheng]

@ericvennemeyer Good job on writing the issue! I made the following changes to what you wrote:

• I added the following last Action Items
  • After conferring with the leads, implement the solution in a topic branch in your forked repo
  • Open a pull request
• I moved what you wrote in the Overview into the Details section. Then, I wrote an Overview as a user story by following the Give a Good Overview section in the How to Create Issues wiki page

[JessicaLucindaCheng]

@Providence-o @ExperimentsInHonesty Even though Eric has not completed a good first issue yet, Bonnie (at the Sun, Apr 17 meeting) said it was okay for Eric to start work on this issue because Eric has already done a lot of research and work associated with figuring out a fix to the error.

Also, I know usually we wait until the issue has been given a Milestone by a PM and prioritized in the Prioritized Backlog column before we allow developers to work on it. However, I think an exception can be made for Eric to self-assign and continue working on this issue since Eric has already done a lot of research and work to figure out a solution to the problem in order to be able to use Docker to run the website locally.

[ericvennemeyer]

@JessicaLucindaCheng Thanks for the adjustments to my original issue. I'm going to wrap up work on it for this week. Please find my progress report below. The next meeting I will be able to attend is this coming Tuesday, 4/26, at 7pm.

i. Progress: I have determined that reverting to older issues of Ruby and Jekyll within a Docker container will have no impact on the security or functioning of the live H4LA site itself. However, as @cnk pointed out in a related Slack thread, we should ensure that the local environment the devs use to modify/test the website uses the same versions of Jekyll, Ruby, etc. that GitHub Pages actually uses. This will guarantee the site behaves exactly the same locally as it does when live.

ii. Blockers: As far as I can tell, there are no preexisting Jekyll Docker images that use precisely the same versions that GitHub Pages uses, namely Jekyll 3.9.2 and Ruby 2.7.3. (Worth noting here that GitHub Pages is using fairly old versions of both; the current Jekyll is 4.2.2, and current Ruby is 3.1.2. I have no idea if/when they will update, but at this point they are running a few years behind.)

Here are some potential resolutions to the problem:

1. The Docker image we currently use, jekyll/jekyll:pages, is intended specifically for GitHub Pages. I've reached out to the maintainer of this repo (who goes by the username envygeeks) directly to request that he modify the image to match the versions of Jekyll and Ruby that GH Pages uses, but have not yet heard back. If those changes were made, it would require no changes on our end.
2. We could settle for one of the official images that most closely matches the GH Pages dependencies. None are perfect, but the closest is jekyll/jekyll:3.8.5, which uses Jekyll 3.8.5 and Ruby 2.6.3. By way of comparison, the jekyll/jekyll:pages image that H4LA has been using for the last few years also uses Jekyll 3.8.5. I tested this version out on my laptop and it seemed to work. One potential downside is that this older image is no longer actively maintained, and it may use other dependencies that are out of date as well.
3. We could create our own Docker image for GH Pages, based on a currently existing image but customized to use exactly the dependencies that GH Pages does. The benefit of this would be that H4LA could have more upstream control over the image, rather than relying on open source code maintained by someone else. The downside would be that H4LA would have to keep track of the credentials for Docker Hub, where the image would be hosted, and be responsible for updating the image if GH Pages were ever to change its dependencies. This option would also require further research as I'm not yet 100% sure how to do it.

iii. Availability: Tuesday 4/26: 2-3hrs, Wednesday 4/27: 2hrs

iv. ETA: Depends on how the team leads would like to proceed. If it seems ok to use the jekyll/jekyll:3.8.5 image, the issue could be resolved now. If we want to build our own Docker image, I estimate another week or so of research.

[JessicaLucindaCheng]

@ericvennemeyer Thanks for the update! We can discuss the next steps at the Tue, Apr 26 meeting.

[ericvennemeyer]

@JessicaLucindaCheng @SAUMILDHANKAR Update per the website team meeting on Tuesday, 4/26:

To follow up on last night's meeting, I rechecked the images listed in the official Jekyll Docker image registry. As mentioned, none of them exactly match the versions of Jekyll/Ruby that we're looking for. However:

1. If we're looking to match the Jekyll version that GitHub Pages uses as closely as possible, we should use jekyll/jekyll:3.8.6. That's the latest version that isn't Jekyll 4. BUT, it uses Ruby 2.6 vs. GitHub's Ruby 2.7.

2. Another option is to match the Jekyll version that the jekyll/jekyll:pages image uses, because H4LA devs have apparently been using that one for years without issues. In that case, we could go with jekyll/jekyll:3.8.5. BUT, again, it uses Ruby 2.6.

3. If, on the other hand, we want to match the Ruby version to that used by GH Pages, then any of the images from (in order of newer to older) jekyll/jekyll:4.2.0 through jekyll/jekyll:minimal use Ruby 2.7. BUT, the Jekyll version is 4, which is a major update above v3.9 that GH Pages uses.

I'm assuming that it's more important for the Jekyll version to be closer, in order to ensure the site is served locally in the same way it would be when live, so my best guess for now is to recommend option 1.

[ericvennemeyer]

@JessicaLucindaCheng @SAUMILDHANKAR One last update for the week:

i. Progress: I've reached out to both Ops and Engineering teams on Slack, and have had responses on both channels. @cnk pointed out the H4LA team does not typically have a lot of devops expertise, and thus it would still be preferable to find an existing Docker image that stays pinned to what GH Pages uses. Following her advice, I've opened a new issue on the jekyll-docker repo asking for either jekyll:pages to be updated to match the GH Pages dependency versions, or for a recommendation as to which image would be best to use.

In the meantime, @Aveline-art suggested a possible way to modify jekyll:pages to use the Ruby/Jekyll versions we need; Jason Ebueng in Ops has offered to take a look; and Hasan Diwan in Engineering has offered to work on the issue with me one-on-one. I will follow up on all of this next week.

iii. Availability: I will be out of town and offline from this afternoon (Thursday 4/28) until the evening of Monday 5/2. I will have some time to work on this again on Tuesday 5/3, and will attend the website meeting that evening.

Have a great weekend!

[SAUMILDHANKAR]

@ericvennemeyer Thanks for sharing your update and making great progress on your issue. Just wanted to let you know that there is no website team meeting on Tuesday, 5/3. Our next team meeting would be the Sunday all-team meeting on 5/8. We would still be doing one-off meetings if required and following up on slack channel as well. Keep up the good work. Thank you.

[ericvennemeyer]

@SAUMILDHANKAR Thanks for the heads-up, I had forgotten there are no meetings this week. Unfortunately, I can't make the all-team meeting on Sunday, 5/8. I will continue to work on this issue, however, and will be available for a one-off meeting or Slack discussion if you would like to check in before next Tuesday 5/10. My schedule should get less weird after this week, so I will be able to attend meetings more regularly in the future.

[JessicaLucindaCheng]

@ericvennemeyer I think an update on this issue with progress, blockers, availability, and ETA would suffice for a check in. Also, if there was anything you would like to discuss, I think a Slack discussion will work, unless you think a one-off meeting would be a better way to discuss it depending on the discussion item.

[ericvennemeyer]

@JessicaLucindaCheng Sounds good, I'll post an update on Friday. At the moment, I don't think I have anything more to discuss, but will reach out on Slack if something comes up in the next two days. Thanks.

[ericvennemeyer]

Hello @SAUMILDHANKAR @JessicaLucindaCheng! (Not sure if I need to tag you guys in each update, let me know if not.)

Cool news!

i. Progress: Because it seems unlikely the official Jekyll Docker maintainers will get to my issue anytime soon, I forked the whole repo and modified their Dockerfile to build a custom GH Pages image on my own computer. After some futzing, this eventually worked, and I was able to spin up a Docker container and view the H4LA website locally. As far as I can tell, it has all the same dependency versions as GH Pages uses. It currently lives at my personal Docker Hub repo.

In order to test this out, other volunteers should simply alter the image: key in their docker-compose.yml files to specify ericvennemeyer/hackforla:gh-pages as the image.

If this works, we could ultimately create a Docker Hub repo for Hack For LA and keep the image there, so the organization can control the credentials for the account. The problem, as before, would be that if anything ever changes with GitHub Pages, or there are any problems with the image down the road, someone at H4LA will have to be responsible for maintaining all of this.

Given that, I would like to try creating one more Docker image that uses the official GitHub Pages ruby gem, which is maintained externally and always pinned to the GH Pages dependency versions. If I'm understanding Docker correctly, this gem will be installed fresh whenever someone uses docker-compose up for the first time, and any software updates that happen on their end will automatically be installed. This approach will require a bit more research to figure out, but I think it could solve all our problems.

ii. Blockers: None at the moment!

iii. Availability: (Maybe) Tuesday 5/10: 1-2 hours; Wednesday 5/11: 1-2 hours; Thursday 5/12: 1-2 hours

iv. ETA: To build a new docker image as proposed above, probably all of next week?

As a reminder, I can't make the all team meeting on Sunday 5/8. I will attend the dev meeting on Tuesday 5/10.

[JessicaLucindaCheng]

@ericvennemeyer Thanks for all your hard work on this issue! You don't have to @ mention us every time you give a progress update. You only have to @ mention us if you have a question, want feedback, or want us to respond to something.

[ericvennemeyer]

i. Progress: Looked a little into the possibility of creating an automated system to ensure the GH-Pages dependencies are up to date when a new image is built. Unfortunately, I think this could be complicated from a technical standpoint, and might also make Docker slightly more complicated for new volunteers to use. (In the past, the h4la site repo included its own dockerfile, and the pre-work instructions required volunteers to build the Docker image from scratch when first using the software, rather than just running docker-compose up. We would probably need to revert to that system in order for any automated solution to work.)

I think the best solution is to maintain our own Docker Hub repo with our own GH-Pages-specific image. @cnk suggested in a slack discussion that maintaining this image could be a good starter project for volunteers on the Ops team. I will reach out to Ops and loop in the Website technical leads next week to see about setting something like that up. In the meantime, I would like to spend a few days streamlining the image I created last week.

ii. Blockers: None at the moment.

iii. Availability: Monday 10/16: 2-3 hours, Tuesday 10/17: 2-3 hours

iv. ETA: Hope to have this whole issue cleared by end of next week.

[ericvennemeyer]

i. Progress: Attended Ops team meeting Wednesday night. They were amenable to the idea of opening a Hack For LA account on Docker Hub and hosting a custom Docker image for the website team there, and also potentially to Ops volunteers maintaining the image. Details still need to be ironed out, but as a next step they requested I open an issue in the hackforla/ops repo and ultimately pitch the idea to Bonnie. I will open this issue on Monday.

ii. Blockers: None.

iii. Availability: Sporadically throughout next week, as needed.

iv. ETA: Unclear at this point, but the work going forward is mostly not technical.

[ericvennemeyer]

i. Progress: Opened new issue in Ops team repo to create decision record re: building and hosting a custom Docker image in a HFLA Docker Hub repo, and potentially having Ops volunteers maintain the code going forward. Began researching GitHub Actions as a possible means of automatically building and pushing that image when the code is updated. Also looked a little more into refining the image I've already built. Will attend the Ops meeting this Sunday to discuss with Bonnie.

ii. Blockers: None.

[github-actions]

@ericvennemeyer

Please add update using the below template (even if you have a pull request). Afterwards, remove the 'To Update !' label and add the 'Status: Updated' label.

1. Progress: "What is the current status of your project? What have you completed and what is left to do?"
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures: "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either: 1) place your issue in the developer meeting discussion column and ask for help at your next meeting, 2) put a "Status: Help Wanted" label on your issue and pull request, or 3) put up a request for assistance on the #hfla-site channel.

_{You are receiving this comment because your last comment was before Tuesday, June 7, 2022 at 12:21 AM PST.}

[ericvennemeyer]

i. Progress: Attended Ops team meeting Sunday 6/5. Explained issue and my recommended fix to Bonnie and the Ops team. It was decided that HFLA will begin maintaining/hosting our own GitHub Pages Docker image. To achieve this, I will:

• Create a new Docker Hub account for HFLA
• Push the Docker image I've created to the newly-created HFLA ghpages-docker repo
• Write a README.md with usage instructions for the Dockerfile in hackforla/ghpages-docker
• Create a bot on GitHub to check the version dependencies for GitHub Pages and open a new issue in the Ops repo when something is updated
• Write a new Wiki in the Ops repo to detail the process for maintaining the Docker image

ii. Blockers: None.

iii. Availability: Tuesday 6/14 and Thursday 6/16, as needed to get the above completed.

iv. ETA: I hope to have all of the above done by end of next week, but it may be the following week. My first priority, however, will be to update the docker-compose.yml file to point to the new image in the new HFLA Docker Hub repo and make a pull request to close out this issue.

[DonaldBrower]

Hey @ericvennemeyer, I was also having issues with docker-compose up, but it was a different error than the one you mention at the top of the thread. Here's the issue and my workaround -- just in case it's relevant to the new image.

[ericvennemeyer]

Hey @DonaldBrower, thanks for sharing that! I've seen that webrick error mentioned during my researches, and I had thought it was a problem between Ruby 3.x and Jekyll 4.x, but the jekyll/jekyll:pages image still uses Jekyll 3.8.5. So I'm not sure why that is now the default error that people on the website team are getting. Regardless, it's all somewhat related to the error that kicked off this issue. GitHub Pages uses old versions of both Ruby and Jekyll (2.7.3 and 3.9.2, respectively), and the official Jekyll Docker images have moved beyond that, thus creating the errors we've been seeing. Because it's difficult to find existing, actively-maintained images that still use the same versions that GitHub Pages does, I ended up creating a new image for us to use. Hopefully it will be published this week, so new volunteers can avoid resorting to workarounds.

[DonaldBrower]

@ericvennemeyer -

So I'm not sure why that is now the default error that people on the website team are getting.

The latest versions of jekyll/jekyll:pages (at least the current and previous one) use Jekyll 3.9.2 -- this version doesn't rely on the pathutil gem, which was the cause of the no implicit conversion of Hash into Integer error.

That leaves the webrick error -- Starting with Ruby 3.0.0, webrick is no longer included as a default gem in ruby, so it should be included manually. This is true for all versions of Jeykll. From the docs:

If you are using Ruby version 3.0.0 or higher, step 5 may fail. You may fix it by adding webrick to your dependencies: bundle add webrick.

[ericvennemeyer]

@DonaldBrower Thanks for the explanation. I do remember reading that issue early on, but didn't know how to understand it at the time. If GitHub Pages ever moves to Ruby 3, this will be super helpful information to have when updating the Dockerfile.

[ericvennemeyer]

i. Progress: New image is tested and ready to go. Resolving some permissions/account issues with Docker Hub and our new hackforla/ghpages-docker repo, but should have this resolved shortly.

ii. Blockers: The permissions/account issues mentioned above.

iii. Availability: Provisionally Tuesday 6/21 and Wednesday 6/22 (after noon)

iv. ETA: Next week?