Skip to content

Commit

Permalink
Merge pull request #119 from CSSE6400/86
Browse files Browse the repository at this point in the history
moving integration tests into tests dir
  • Loading branch information
86LAK authored Jun 2, 2024
2 parents 62c375a + e2909cf commit f5c3f93
Show file tree
Hide file tree
Showing 19 changed files with 13 additions and 11 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ jobs:
touch .env.local
- name: Run integration tests
working-directory: ./integration_tests
working-directory: ./tests/integration_tests
run: |
chmod +x ./tests.sh
./tests.sh
7 changes: 5 additions & 2 deletions docs/TESTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ To add a new test, create a new test function inside the "backend.test.jest" fil
## Integration Testing

### About Integration Tests
The integrations tests comprise of a backend integration suite and an e2e testing suite. The backend integration tests focus on testing the integration between the persistence and database layers of the architecture, by spinning up the backend server, calling various routes, and evaluating whether changes are reflected in the database. The e2e tests focus on testing the entire layered architecture, by spinning up the frontend, backend, and database, and testing the key functionality as though a user is interacting with the system.

The integrations tests comprise of a backend integration suite and an e2e testing suite. The backend integration tests focus on testing the integration between the persistence and database layers of the architecture, by spinning up the backend server, calling various routes, and evaluating whether changes are reflected in the database. The e2e tests focus on testing the entire layered architecture, by spinning up the frontend, backend, and database, and testing the key functionality as though a user is interacting with the system.

### Running Integration Tests

Expand All @@ -28,11 +29,13 @@ The full e2e test is excluded from the test script dockerfile as it requires Aut
### Adding to Integration Tests

#### Adding to backend integration tests

The backend integration tests are created using pytest.

Ensure that basic testing of the success and errors are tested inside the appropriate ".py" file. Ensure tests are written to also test the error handling and returned HTTP code. After this try adding the test into one or more of the scenarios in the "test_full_suite.py" file. The added test should follow the format of the other tests in the file. Tests should have a meaningful name following the format of test_ROUTE_TYPE_MOREINFO. Where ROUTE = route to be tested, TYPE= get, post etc and MOREINFO = meaningful information about what the test is doing. This will help developers understand what the test is doing and why it is there. Ensure that each tests asserts the returned http code and the returned data. Inside each test if something is unclear, add a comment to explain what is happening.

#### Adding to e2e tests

The e2e tests are created using PlayWright.

Ensure that any new functionality is added to the full e2e test. Smaller e2e tests can be created if necessary. The easiest way to add tests to the frontend is to use playwrights vscode record feature to record the tests. This will generate the tests for you. After this, you can modify the tests to make them more readable and understandable. The tests should be placed inside the "integration_tests/frontend" directory. Where the filename represents the page type (e.g. test_home_page). To use playwright record please visit [PlayWright](https://playwright.dev/docs/getting-started-vscode).
Expand All @@ -57,4 +60,4 @@ The tests are run automatically on every push or PR to the main branch. This wil

## Results of Tests

This section has been added for the marker. For proof and evidence of the tests please visit the GitHub Actions and select the test workflow. Here you will find both the basic unit tests and integration tests being run and their results.
This section has been added for the marker. For proof and evidence of the tests please visit the GitHub Actions and select the test workflow. Here you will find both the basic unit tests and integration tests being run and their results.
11 changes: 5 additions & 6 deletions report/REPORT.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,16 @@ Throughout the project there were required changes compared to what was outlined

In the project proposal it was outlined that the users should be able to create an account and register their university affiliation. However, due to time constraints we were forced to cut out the university affiliation and centre the project around University of Queensland students.

The team had also realised that the allowing users to sign up would require us to handle the variety of security risks it would introduce such as storing user login data. Therefore, we decided to use AUTH0 to handle the login using external accounts such as Gmail or GitHub. This however, had the unintended affect that it removed our ability to allow users to update the account information or profile picture as that it was handled by AUTH0. The decision to use AUTH0 was made after careful consideration of its security practices and its effect on the project, and is detailed further in [ADR001](../model/adrs/ADR001_AUTHENTICATION.md).
The team had also realised that allowing users to sign up would require us to handle the variety of security risks it would introduce such as storing user login data. Therefore, we decided to use AUTH0 to handle the login using external accounts such as Gmail or GitHub. This however, had the unintended affect that it removed our ability to allow users to update the account information or profile picture as that it was handled by AUTH0. The decision to use AUTH0 was made after careful consideration of its security practices and its effect on the project, and is detailed further in [ADR001](../model/adrs/ADR001_AUTHENTICATION.md).

### Succes criteria
### Success criteria

All of the functional requirements outlined in the proposal were, in their current state, untestable and questionable in quality. For example, "Over 99% uptime over a measured period", so if we run it for a second and it doesn't die is it okay? "All core functionality tested", but what is described as 'core'? From this we changed some of the succes criteria to be better defined and easily testable.
The project proposal had many ambiguities and untestable success criteria. An example of this is "99% uptime over a measured period," however the proposal does not specifically quantify the minimum measured time. To better define the success criteria we made clarifications on the ambiguous ones, this can be seen below.

| Old | New |
|-------|---------|
| Over 99% uptime over a measured period | Over 99% uptime over a 1 day span under some load (link k6 code if done here) |
| All core functionality tested | 100% of backend functions should have their main uses tested (i.e. 200/201 response) and most ( >50%) sub cases. On top of this general automated frontend testing should be done to ensure their is a connection |
| sum about maintain | idk this shit is so dumb |
| Over 99% uptime over a measured period | Over 99% uptime over a 2 day span with the checks happening to both frontend and backend in 5 minute intervals |
| All core functionality tested | 100% of backend functions should have their main functionality tested (i.e. 200/201 response). e2e testing should cover base use cases of logging in, creating course, exam, question and a comment. |

## Architecture Options

Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ services:
backend:
build:
dockerfile: Dockerfile
context: ../backend
context: ../../backend
environment:
PORT: 8080
DB_TYPE: "postgres"
Expand All @@ -28,7 +28,7 @@ services:
frontend:
build:
dockerfile: Dockerfile
context: ../frontend
context: ../../frontend
environment:
NEXT_PUBLIC_API_URL: "http://localhost:8080"
ports:
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.

0 comments on commit f5c3f93

Please sign in to comment.