diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 4665edc..aa5e691 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -51,7 +51,7 @@ jobs: touch .env.local - name: Run integration tests - working-directory: ./integration_tests + working-directory: ./tests/integration_tests run: | chmod +x ./tests.sh ./tests.sh diff --git a/docs/TESTS.md b/docs/TESTS.md index 98258b5..4d9054f 100644 --- a/docs/TESTS.md +++ b/docs/TESTS.md @@ -17,7 +17,8 @@ To add a new test, create a new test function inside the "backend.test.jest" fil ## Integration Testing ### About Integration Tests -The integrations tests comprise of a backend integration suite and an e2e testing suite. The backend integration tests focus on testing the integration between the persistence and database layers of the architecture, by spinning up the backend server, calling various routes, and evaluating whether changes are reflected in the database. The e2e tests focus on testing the entire layered architecture, by spinning up the frontend, backend, and database, and testing the key functionality as though a user is interacting with the system. + +The integrations tests comprise of a backend integration suite and an e2e testing suite. The backend integration tests focus on testing the integration between the persistence and database layers of the architecture, by spinning up the backend server, calling various routes, and evaluating whether changes are reflected in the database. The e2e tests focus on testing the entire layered architecture, by spinning up the frontend, backend, and database, and testing the key functionality as though a user is interacting with the system. ### Running Integration Tests @@ -28,11 +29,13 @@ The full e2e test is excluded from the test script dockerfile as it requires Aut ### Adding to Integration Tests #### Adding to backend integration tests + The backend integration tests are created using pytest. Ensure that basic testing of the success and errors are tested inside the appropriate ".py" file. Ensure tests are written to also test the error handling and returned HTTP code. After this try adding the test into one or more of the scenarios in the "test_full_suite.py" file. The added test should follow the format of the other tests in the file. Tests should have a meaningful name following the format of test_ROUTE_TYPE_MOREINFO. Where ROUTE = route to be tested, TYPE= get, post etc and MOREINFO = meaningful information about what the test is doing. This will help developers understand what the test is doing and why it is there. Ensure that each tests asserts the returned http code and the returned data. Inside each test if something is unclear, add a comment to explain what is happening. #### Adding to e2e tests + The e2e tests are created using PlayWright. Ensure that any new functionality is added to the full e2e test. Smaller e2e tests can be created if necessary. The easiest way to add tests to the frontend is to use playwrights vscode record feature to record the tests. This will generate the tests for you. After this, you can modify the tests to make them more readable and understandable. The tests should be placed inside the "integration_tests/frontend" directory. Where the filename represents the page type (e.g. test_home_page). To use playwright record please visit [PlayWright](https://playwright.dev/docs/getting-started-vscode). @@ -57,4 +60,4 @@ The tests are run automatically on every push or PR to the main branch. This wil ## Results of Tests -This section has been added for the marker. For proof and evidence of the tests please visit the GitHub Actions and select the test workflow. Here you will find both the basic unit tests and integration tests being run and their results. \ No newline at end of file +This section has been added for the marker. For proof and evidence of the tests please visit the GitHub Actions and select the test workflow. Here you will find both the basic unit tests and integration tests being run and their results. diff --git a/report/REPORT.md b/report/REPORT.md index 2c81e2d..5f0994b 100644 --- a/report/REPORT.md +++ b/report/REPORT.md @@ -12,17 +12,16 @@ Throughout the project there were required changes compared to what was outlined In the project proposal it was outlined that the users should be able to create an account and register their university affiliation. However, due to time constraints we were forced to cut out the university affiliation and centre the project around University of Queensland students. -The team had also realised that the allowing users to sign up would require us to handle the variety of security risks it would introduce such as storing user login data. Therefore, we decided to use AUTH0 to handle the login using external accounts such as Gmail or GitHub. This however, had the unintended affect that it removed our ability to allow users to update the account information or profile picture as that it was handled by AUTH0. The decision to use AUTH0 was made after careful consideration of its security practices and its effect on the project, and is detailed further in [ADR001](../model/adrs/ADR001_AUTHENTICATION.md). +The team had also realised that allowing users to sign up would require us to handle the variety of security risks it would introduce such as storing user login data. Therefore, we decided to use AUTH0 to handle the login using external accounts such as Gmail or GitHub. This however, had the unintended affect that it removed our ability to allow users to update the account information or profile picture as that it was handled by AUTH0. The decision to use AUTH0 was made after careful consideration of its security practices and its effect on the project, and is detailed further in [ADR001](../model/adrs/ADR001_AUTHENTICATION.md). -### Succes criteria +### Success criteria -All of the functional requirements outlined in the proposal were, in their current state, untestable and questionable in quality. For example, "Over 99% uptime over a measured period", so if we run it for a second and it doesn't die is it okay? "All core functionality tested", but what is described as 'core'? From this we changed some of the succes criteria to be better defined and easily testable. +The project proposal had many ambiguities and untestable success criteria. An example of this is "99% uptime over a measured period," however the proposal does not specifically quantify the minimum measured time. To better define the success criteria we made clarifications on the ambiguous ones, this can be seen below. | Old | New | |-------|---------| -| Over 99% uptime over a measured period | Over 99% uptime over a 1 day span under some load (link k6 code if done here) | -| All core functionality tested | 100% of backend functions should have their main uses tested (i.e. 200/201 response) and most ( >50%) sub cases. On top of this general automated frontend testing should be done to ensure their is a connection | -| sum about maintain | idk this shit is so dumb | +| Over 99% uptime over a measured period | Over 99% uptime over a 2 day span with the checks happening to both frontend and backend in 5 minute intervals | +| All core functionality tested | 100% of backend functions should have their main functionality tested (i.e. 200/201 response). e2e testing should cover base use cases of logging in, creating course, exam, question and a comment. | ## Architecture Options diff --git a/integration_tests/Dockerfile b/tests/integration_tests/Dockerfile similarity index 100% rename from integration_tests/Dockerfile rename to tests/integration_tests/Dockerfile diff --git a/integration_tests/__init__.py b/tests/integration_tests/__init__.py similarity index 100% rename from integration_tests/__init__.py rename to tests/integration_tests/__init__.py diff --git a/integration_tests/backend/__init__.py b/tests/integration_tests/backend/__init__.py similarity index 100% rename from integration_tests/backend/__init__.py rename to tests/integration_tests/backend/__init__.py diff --git a/integration_tests/backend/test_comments.py b/tests/integration_tests/backend/test_comments.py similarity index 100% rename from integration_tests/backend/test_comments.py rename to tests/integration_tests/backend/test_comments.py diff --git a/integration_tests/backend/test_courses.py b/tests/integration_tests/backend/test_courses.py similarity index 100% rename from integration_tests/backend/test_courses.py rename to tests/integration_tests/backend/test_courses.py diff --git a/integration_tests/backend/test_exams.py b/tests/integration_tests/backend/test_exams.py similarity index 100% rename from integration_tests/backend/test_exams.py rename to tests/integration_tests/backend/test_exams.py diff --git a/integration_tests/backend/test_full_suite.py b/tests/integration_tests/backend/test_full_suite.py similarity index 100% rename from integration_tests/backend/test_full_suite.py rename to tests/integration_tests/backend/test_full_suite.py diff --git a/integration_tests/backend/test_health_routes.py b/tests/integration_tests/backend/test_health_routes.py similarity index 100% rename from integration_tests/backend/test_health_routes.py rename to tests/integration_tests/backend/test_health_routes.py diff --git a/integration_tests/backend/test_questions.py b/tests/integration_tests/backend/test_questions.py similarity index 100% rename from integration_tests/backend/test_questions.py rename to tests/integration_tests/backend/test_questions.py diff --git a/integration_tests/backend/test_users.py b/tests/integration_tests/backend/test_users.py similarity index 100% rename from integration_tests/backend/test_users.py rename to tests/integration_tests/backend/test_users.py diff --git a/integration_tests/base.py b/tests/integration_tests/base.py similarity index 100% rename from integration_tests/base.py rename to tests/integration_tests/base.py diff --git a/integration_tests/docker-compose.yml b/tests/integration_tests/docker-compose.yml similarity index 94% rename from integration_tests/docker-compose.yml rename to tests/integration_tests/docker-compose.yml index bcc6f18..1efdbdf 100644 --- a/integration_tests/docker-compose.yml +++ b/tests/integration_tests/docker-compose.yml @@ -4,7 +4,7 @@ services: backend: build: dockerfile: Dockerfile - context: ../backend + context: ../../backend environment: PORT: 8080 DB_TYPE: "postgres" @@ -28,7 +28,7 @@ services: frontend: build: dockerfile: Dockerfile - context: ../frontend + context: ../../frontend environment: NEXT_PUBLIC_API_URL: "http://localhost:8080" ports: diff --git a/integration_tests/frontend/__init__.py b/tests/integration_tests/frontend/__init__.py similarity index 100% rename from integration_tests/frontend/__init__.py rename to tests/integration_tests/frontend/__init__.py diff --git a/integration_tests/frontend/test_full_e2e.py b/tests/integration_tests/frontend/test_full_e2e.py similarity index 100% rename from integration_tests/frontend/test_full_e2e.py rename to tests/integration_tests/frontend/test_full_e2e.py diff --git a/integration_tests/frontend/test_home_page.py b/tests/integration_tests/frontend/test_home_page.py similarity index 100% rename from integration_tests/frontend/test_home_page.py rename to tests/integration_tests/frontend/test_home_page.py diff --git a/integration_tests/tests.sh b/tests/integration_tests/tests.sh similarity index 100% rename from integration_tests/tests.sh rename to tests/integration_tests/tests.sh