Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Production docker image built without test gems #334

Closed
wants to merge 4 commits into from

Conversation

dragon-dxw
Copy link
Contributor

Backport of https://github.com/citizensadvice/corporate-api/pull/55

The CA Corporate API is headless, so it doesn't use node, which means it's likely that this doesn't yet correctly handle node modules.

Changes in this PR

Before this change our Dockerfile wasn't correctly creating Docker images. It was bundling gems from the test group despite expecting RAILS_ENV to be set as production by default. We don't want to be deploying live artefacts that have test dependencies and tooling installed. They are bigger and increase the surface area for complexity and attacks.

The meat of this change is centered around the Dockerfile. Instead of using RAILS_ENV to try and let the user convey what gems they expected, we switch to creating more explicit docker build stages and select them with the target field instead.

Testing

  1. DOCKER=1 script/test should still run all the tests (and have the test gems to do so)
  2. DOCKER=1 script/server should still start up a local development server and be accesible at localhost:3000

If you want to test this in production too you can temporarily copy this into your app directory, into a file called docker-compose.production.yml. Once there you can run docker-compose -f docker-compose.production.yml up to check it boots in the right environment and watch that the right gems were installed:

version: "3.8"
services:
  web:
    build:
      context: .
      target: production
    command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
    ports:
      - "3000:3000"
    depends_on:
      - db
    env_file:
       - .env.development.local
    environment:
      # DONE: Update rails-template with application name
      DATABASE_URL: postgres://postgres:password@db:5432/corporate-api-production
      AUTOMATICALLY_FIX_LINTING: "true"
    volumes:
      - .:/srv/app
    tty: true
    stdin_open: true
    networks:
      - prod

  db:
    image: postgres
    volumes:
      - db-data:/var/lib/postgresql/data
    environment:
      POSTGRES_PASSWORD: password
      POSTGRES_HOST_AUTH_METHOD: trust
    networks:
      - prod

networks:
  prod:

volumes:
  db-data:

@tahb tahb marked this pull request as ready for review April 1, 2022 17:41
@tahb
Copy link
Contributor

tahb commented Apr 1, 2022

These changes worked well for CAEW and we should get this merged in!

@pezholio
Copy link
Contributor

I'd like to get this in, but the tests are failing at the mo

@tahb
Copy link
Contributor

tahb commented Jun 8, 2022

I've checked with Dragon and I'm going to try and move this along.

@tahb tahb force-pushed the fix/docker-production-test-gems branch 2 times, most recently from 5ab524e to e90cc67 Compare June 8, 2022 15:41
@tahb
Copy link
Contributor

tahb commented Jun 8, 2022

@pezholio back in business with a green test suite 👍

@tahb tahb requested review from erbridge and pezholio June 8, 2022 16:34
Rakefile Outdated Show resolved Hide resolved
Rakefile Outdated Show resolved Hide resolved
Rakefile Outdated Show resolved Hide resolved
Dockerfile Show resolved Hide resolved
Dockerfile Show resolved Hide resolved
Dockerfile Outdated Show resolved Hide resolved
Dockerfile Show resolved Hide resolved
config/environments/production.rb Show resolved Hide resolved
@tahb tahb force-pushed the fix/docker-production-test-gems branch 5 times, most recently from ec897be to 5923dca Compare September 23, 2022 13:57
@tahb
Copy link
Contributor

tahb commented Sep 23, 2022

@erbridge I completely forgot about this! I've actioned your feedback now.

Copy link
Contributor

@erbridge erbridge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks sensible to me. Minor commit message issue, but otherwise 👍.

@tahb tahb force-pushed the fix/docker-production-test-gems branch 2 times, most recently from c453b42 to a10301d Compare September 30, 2022 09:20
@dragon-dxw
Copy link
Contributor Author

LGTM, I can't approve as the creator, but @tabh feel free to approve on my behalf if you're happy with it :)

Copy link
Contributor

@tahb tahb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving on behalf of Dragon who's the original author and therefore can't :)

tahb added 4 commits December 21, 2023 10:02
`bundle config set with $BUNDLE_GEM_GROUPS` wasn't doing what we thought in the production context.

It'd work out to be `bundle config set with production` which would bundle everything, rather than only production groups.
I'm not sure where this assumption came from. 'default' also doesn't work.

As the first step of a bit of a refactoring of the Dockerfile, we cement the assurance that by default the dockerfile _only_ bundles production gems.
We remove the way the build_arg `RAILS_ENV` was used to accomodate being built for production, development and test. This hopefully makes it much clearer.

(I've also set NODE_ENV to follow the same path as RAILS_ENV -- Dragon)
Instead of passing `RAILS_ENV` as a build argument to the dockerfile to figure out what should be bundled.
We switch to using `--target` to target a specific docker build stage that hardcodes the `RAILS_ENV` so it can't have an unexpected value.

We now have a stage for production, development and test. Each explicitly installs gems for the right group.
When building the docker image in production with script/docker/update
we get a new error:

```
 > [production 21/23] RUN   if [ "production" = "production" ]; then   SECRET_KEY_BASE="secret"   bundle exec rake assets:precompile;   fi:
43 3.045 Yarn executable was not detected in the system.
43 3.045 Download Yarn at https://yarnpkg.com/en/docs/install
43 3.326 Yarn executable was not detected in the system.
43 3.326 Download Yarn at https://yarnpkg.com/en/docs/install
43 3.668 rake aborted!
43 3.668 Uglifier::Error: Unexpected token: keyword (const). To use ES6 syntax, harmony mode must be enabled with Uglifier.new(:harmony => true).
…
43 3.668  11663   /**
43 3.668  11664    * ------------------------------------------------------------------------
43 3.668  11665    * Constants
43 3.668  11666    * ------------------------------------------------------------------------
43 3.668  11667    */
43 3.668     =>   const elementMap = new Map();
```

We follow the instruction to use `Uglifier.new(:harmony => true)` and
sure enought the error is silenced.
When starting the application with RAILS_ENV=production mode and we run `rake` we get an error that explains we don't have standard installed.

This is correct as out gemfile specifies that we only install standard in the dev and test environments.

This change only tries to load the standard rake task when in the dev or test environment.

Other take tasks such as `rails db:prepare` can still run in any environment as normal.
@lozette lozette force-pushed the fix/docker-production-test-gems branch 2 times, most recently from 4fbec67 to 70b6b2b Compare December 22, 2023 10:00
@lozette
Copy link
Contributor

lozette commented Dec 22, 2023

Been trying to get the tests passing on this to no avail, sorry!

@dragon-dxw dragon-dxw marked this pull request as draft January 10, 2024 15:02
@dragon-dxw
Copy link
Contributor Author

Converted to draft because this is no longer fit to be published, but someone might want to build on this work in future.

Also it'll silence the notifications I get for it...

@mec
Copy link
Contributor

mec commented Aug 23, 2024

Given the state and age of this work, I am closing this PR.

@mec mec closed this Aug 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants