CI: Run gen-server tests with Postgres #1336
Conversation
Context: some server tests succeeded when using sqlite but failed with postgresql. This commit runs the same tests using a postgresql database to ensure both database types are supported for these tests.
.github/workflows/main.yml
Outdated
| - name: Run gen-server tests with minio and redis | ||
| if: contains(matrix.tests, ':gen-server-') | ||
| run: | | ||
| PGPASSWORD=db_password psql -h localhost -U db_user -w db_name -c "SHOW ALL;" | grep ' jit ' | ||
| export TEST_SPLITS=$(echo $TESTS | sed "s/.*:gen-server-\([^:]*\).*/\1/") | ||
| yarn run test:gen-server | ||
| TYPEORM_TYPE=postgres TYPEORM_HOST=localhost TYPEORM_DATABASE=db_name TYPEORM_USERNAME=db_user TYPEORM_PASSWORD=db_password yarn run test:gen-server | ||
| env: | ||
| MOCHA_WEBDRIVER_HEADLESS: 1 | ||
| TESTS: ${{ matrix.tests }} | ||
| GRIST_DOCS_MINIO_ACCESS_KEY: administrator | ||
| GRIST_DOCS_MINIO_SECRET_KEY: administrator | ||
| TEST_REDIS_URL: "redis://localhost/11" | ||
| GRIST_DOCS_MINIO_USE_SSL: 0 | ||
| GRIST_DOCS_MINIO_ENDPOINT: localhost | ||
| GRIST_DOCS_MINIO_PORT: 9000 | ||
| GRIST_DOCS_MINIO_BUCKET: grist-docs-test | ||
|
|
There was a problem hiding this comment.
What do you think about splitting this up into two steps (e.g. Run gen-server tests with postgres and Run gen-server tests with sqlite)?
There was a problem hiding this comment.
I also think it logically makes more sense to split them, but I put them together to avoid excessive duplication in the env section. Do you feel strongly about it?
There was a problem hiding this comment.
Would yaml's anchor / alias solve your issue? This resource help understanding the concept: https://www.educative.io/blog/advanced-yaml-syntax-cheatsheet
- name: Run gen-server tests with minio and redis with sqlite
if: contains(matrix.tests, ':gen-server-')
run: |
export TEST_SPLITS=$(echo $TESTS | sed "s/.*:gen-server-\([^:]*\).*/\1/")
yarn run test:gen-server
env: &gen-server-env-vars
MOCHA_WEBDRIVER_HEADLESS: 1
TESTS: ${{ matrix.tests }}
GRIST_DOCS_MINIO_ACCESS_KEY: administrator
GRIST_DOCS_MINIO_SECRET_KEY: administrator
TEST_REDIS_URL: "redis://localhost/11"
GRIST_DOCS_MINIO_USE_SSL: 0
GRIST_DOCS_MINIO_ENDPOINT: localhost
GRIST_DOCS_MINIO_PORT: 9000
GRIST_DOCS_MINIO_BUCKET: grist-docs-test
- name: Run gen-server tests with minio and redis with postgresql
if: contains(matrix.tests, ':gen-server-')
run: |
PGPASSWORD=db_password psql -h localhost -U db_user -w db_name -c "SHOW ALL;" | grep ' jit '
export TEST_SPLITS=$(echo $TESTS | sed "s/.*:gen-server-\([^:]*\).*/\1/")
yarn run test:gen-server
env:
<<: *gen-server-env-vars
TYPEORM_TYPE: postgres
TYPEORM_HOST: localhost
TYPEORM_DATABASE: db_name
TYPEORM_USERNAME: db_user
TYPEORM_PASSWORD: db_password
# Also change the TEST_REDIS_URL value?And you might want to factorize a bit more… or not… Sometimes we have to choose a tradeoff between readability and factorization.
There was a problem hiding this comment.
@fflorent Sadly anchors are not supported by GH actions: actions/runner#1182
Co-authored-by: George Gevoian <[email protected]>
.github/workflows/main.yml
Outdated
| - name: Run gen-server tests with minio and redis | ||
| if: contains(matrix.tests, ':gen-server-') | ||
| run: | | ||
| PGPASSWORD=db_password psql -h localhost -U db_user -w db_name -c "SHOW ALL;" | grep ' jit ' |
There was a problem hiding this comment.
What is this? Looks like a print for a test, I just want to be sure of that :)
There was a problem hiding this comment.
An extra bit of info to confirm JIT is turned off successfully. For me it was useful for debugging CI problems, and I just kept it.
This reverts commit 7575c20.
SleepyLeslie
force-pushed
the
add-postgresql-ci
branch
from
177e7f2 to
53646de
Compare
SleepyLeslie
force-pushed
the
add-postgresql-ci
branch
from
53646de to
144f3a1
Compare
| POSTGRES_USER: db_user | ||
| POSTGRES_PASSWORD: db_password | ||
| POSTGRES_DB: db_name | ||
| # JIT is enabled by default since Postgres 17 and has a huge negative impact on performance, |
There was a problem hiding this comment.
| # JIT is enabled by default since Postgres 17 and has a huge negative impact on performance, | |
| # JIT is enabled by default since Postgres 17 and has a huge negative impact on tests performance, |
There was a problem hiding this comment.
It affects not only test performance. See https://support.getgrist.com/self-managed/#what-is-a-home-database
Context
This is the successor of PR #1278.
Proposed solution
gen-servertests fromservertests, as discussed in CI: add server tests with Postgresql #1278gen-servertests with PostgresRelated issues
N/A
Has this been tested?