Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Run gen-server tests with Postgres #1336

Merged
merged 9 commits into from
Dec 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 60 additions & 2 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ jobs:
- ':lint:python:client:common:smoke:stubs:'
- ':server-1-of-2:'
- ':server-2-of-2:'
- ':gen-server:'
- ':nbrowser-^[A-D]:'
- ':nbrowser-^[E-L]:'
- ':nbrowser-^[M-N]:'
Expand Down Expand Up @@ -63,7 +64,7 @@ jobs:
run: yarn run lint:ci

- name: Make sure bucket is versioned
if: contains(matrix.tests, ':server-')
if: contains(matrix.tests, ':server-') || contains(matrix.tests, ':gen-server:')
env:
AWS_ACCESS_KEY_ID: administrator
AWS_SECRET_ACCESS_KEY: administrator
Expand Down Expand Up @@ -96,12 +97,51 @@ jobs:
if: contains(matrix.tests, ':stubs:')
run: MOCHA_WEBDRIVER_HEADLESS=1 yarn run test:stubs

- name: Run gen-server tests with sqlite, minio and redis
if: contains(matrix.tests, ':gen-server:')
run: |
yarn run test:gen-server
# Anchors should be used once available. Not supported yet as of December 2024.
# https://github.com/actions/runner/issues/1182
env:
MOCHA_WEBDRIVER_HEADLESS: 1
TESTS: ${{ matrix.tests }}
GRIST_DOCS_MINIO_ACCESS_KEY: administrator
GRIST_DOCS_MINIO_SECRET_KEY: administrator
TEST_REDIS_URL: "redis://localhost/11"
GRIST_DOCS_MINIO_USE_SSL: 0
GRIST_DOCS_MINIO_ENDPOINT: localhost
GRIST_DOCS_MINIO_PORT: 9000
GRIST_DOCS_MINIO_BUCKET: grist-docs-test

- name: Run gen-server tests with postgres, minio and redis
if: contains(matrix.tests, ':gen-server:')
run: |
PGPASSWORD=$TYPEORM_PASSWORD psql -h $TYPEORM_HOST -U $TYPEORM_USERNAME -w $TYPEORM_DATABASE -c "SHOW ALL;" | grep ' jit '
yarn run test:gen-server
env:
MOCHA_WEBDRIVER_HEADLESS: 1
TESTS: ${{ matrix.tests }}
GRIST_DOCS_MINIO_ACCESS_KEY: administrator
GRIST_DOCS_MINIO_SECRET_KEY: administrator
TEST_REDIS_URL: "redis://localhost/11"
GRIST_DOCS_MINIO_USE_SSL: 0
GRIST_DOCS_MINIO_ENDPOINT: localhost
GRIST_DOCS_MINIO_PORT: 9000
GRIST_DOCS_MINIO_BUCKET: grist-docs-test
TYPEORM_TYPE: postgres
TYPEORM_HOST: localhost
TYPEORM_DATABASE: db_name
TYPEORM_USERNAME: db_user
TYPEORM_PASSWORD: db_password

- name: Run server tests with minio and redis
if: contains(matrix.tests, ':server-')
run: |
export TEST_SPLITS=$(echo $TESTS | sed "s/.*:server-\([^:]*\).*/\1/")
MOCHA_WEBDRIVER_HEADLESS=1 yarn run test:server
yarn run test:server
env:
MOCHA_WEBDRIVER_HEADLESS: 1
TESTS: ${{ matrix.tests }}
GRIST_DOCS_MINIO_ACCESS_KEY: administrator
GRIST_DOCS_MINIO_SECRET_KEY: administrator
Expand Down Expand Up @@ -167,6 +207,24 @@ jobs:
--health-timeout 5s
--health-retries 5

postgresql:
image: postgres:latest
env:
POSTGRES_USER: db_user
POSTGRES_PASSWORD: db_password
POSTGRES_DB: db_name
# JIT is enabled by default since Postgres 17 and has a huge negative impact on performance,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# JIT is enabled by default since Postgres 17 and has a huge negative impact on performance,
# JIT is enabled by default since Postgres 17 and has a huge negative impact on tests performance,

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It affects not only test performance. See https://support.getgrist.com/self-managed/#what-is-a-home-database

# making many tests timeout.
# https://support.getgrist.com/self-managed/#what-is-a-home-database
POSTGRES_INITDB_ARGS: "-c jit=off"
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U db_user"
--health-interval 10s
--health-timeout 5s
--health-retries 5

candidate:
needs: build_and_test
if: ${{ success() && github.event_name == 'push' }}
Expand Down
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@
"test:stubs": "GRIST_TEST_LOGIN=1 ./test/test_env.sh mocha ${DEBUG:+-b --no-exit} $([ -z $DEBUG ] && echo --forbid-only) -g \"${GREP_TESTS}\" --slow 8000 -R test/xunit-file '_build/test/nbrowser_with_stubs/**/*.js'",
"test:client": "./test/test_env.sh mocha ${DEBUG:+'-b'} '_build/test/client/**/*.js'",
"test:common": "./test/test_env.sh mocha ${DEBUG:+'-b'} '_build/test/common/**/*.js'",
"test:server": "TEST_SUITE=server TEST_SUITE_FOR_TIMINGS=server TIMINGS_FILE=test/timings/server.txt ./test/test_env.sh mocha ${DEBUG:+'-b'} -g \"${GREP_TESTS}\" -R test/xunit-file '_build/test/server/**/*.js' '_build/test/gen-server/**/*.js'",
"test:server": "TEST_SUITE=server TEST_SUITE_FOR_TIMINGS=server TIMINGS_FILE=test/timings/server.txt ./test/test_env.sh mocha ${DEBUG:+'-b'} -g \"${GREP_TESTS}\" -R test/xunit-file '_build/test/server/**/*.js'",
"test:gen-server": "TEST_SUITE=gen-server TEST_SUITE_FOR_TIMINGS=gen-server TIMINGS_FILE=test/timings/gen-server.txt ./test/test_env.sh mocha ${DEBUG:+'-b'} -g \"${GREP_TESTS}\" -R test/xunit-file '_build/test/gen-server/**/*.js'",
"test:smoke": "./test/test_env.sh mocha _build/test/nbrowser/Smoke.js",
"test:docker": "./test/test_under_docker.sh",
"test:python": "sandbox_venv3/bin/python sandbox/grist/runtests.py ${GREP_TESTS:+discover -p \"test*${GREP_TESTS}*.py\"}",
Expand Down