From 20131bbf31f8990abba669d75be801e739710e32 Mon Sep 17 00:00:00 2001 From: Luke Winship Date: Thu, 31 Aug 2023 15:52:06 +0100 Subject: [PATCH 1/7] docs: Explain how FlowStages work and their rationale --- .../test/helpers/flow-stages/README.md | 75 +++++++++++++++++++ .../helpers/flow-stages/flow-stage-run.js | 2 + .../test/helpers/flow-stages/flow-stage.js | 8 +- 3 files changed, 84 insertions(+), 1 deletion(-) create mode 100644 packages/openactive-integration-tests/test/helpers/flow-stages/README.md diff --git a/packages/openactive-integration-tests/test/helpers/flow-stages/README.md b/packages/openactive-integration-tests/test/helpers/flow-stages/README.md new file mode 100644 index 0000000000..c4f4a1b441 --- /dev/null +++ b/packages/openactive-integration-tests/test/helpers/flow-stages/README.md @@ -0,0 +1,75 @@ +# Flow Stages + +A Flow is a sequence of API calls that are made by Test Suite, for a given test, against either the Booking System or the Broker Microservice. A Flow Stage encapsulates the configuration and logic for one of those API calls. e.g. there is a FlowStage for C1 (which is in the Booking System), a FlowStage for Fetch Opportunities (which uses the Broker Microservice), etc. + +As an example, for [`opportunity-free-test`](packages/openactive-integration-tests/test/features/payment/free-opportunities/implemented/opportunity-free-test.js), when running in [Simple Booking Flow](https://openactive.io/open-booking-api/EditorsDraft/#simple-booking-flow), the following Flow Stages are utilised: + +1. Fetch Opportunities +2. C1 +3. Assert Opportunity Capacity (after C1) +4. C2 +5. Assert Opportunity Capacity (after C2) +6. B +7. Assert Opportunity Capacity (after B) + +Each of these FlowStages does the following, when run in a test: + +1. Receives some input from previous FlowStages (if applicable) + - This input can be provided to each FlowStage via `getInput(..)`, which is provided in the constructor. e.g. to have a C1 FlowStage receive the output of a Fetch Opportunities FlowStage, use the following in the C1 FlowStage's constructor: + ```js + const fetchOpportunities = new FetchOpportunitiesFlowStage({ /* ... */ }); + const c1 = new C1FlowStage({ + getInput: () => ({ + orderItems: fetchOpportunities.getOutput().orderItems, + }), + // ... other args + }); + ``` +2. Performs an API call + - Using `runFn(..)`, which is provided in the constructor +3. Transforms the output + - This is also performed in `runFn(..)`, which is provided in the constructor +4. Sends (some of) this output to subsequent FlowStages + - This output is automatically saved by the FlowStage as the output from `runFn(..)`, which is provided in the constructor, and can be retrieved by calling `getOutput(..)` on the FlowStage instance. + +FlowStages can then be queried after they've run in order to: + +1. Check that the FlowStage was successful + - This is overridable per-FlowStage. For example, B's Flow Stage considers the run successful if the HTTP response has status 201 + - FlowStage method: `itSuccessChecks()` +2. Perform validation checks on the output + - This is overridable per-FlowStage. In all cases, this is a case of calling [Validator](https://github.com/openactive/data-model-validator) on the HTTP output. + - FlowStage method: `itValidationTests()` + +## FlowStageRunnable + +An abstraction that can be either a **Flow Stage**, [**Book Recipe**](./book-recipe.js) or a [**Flow Stage Run**](./flow-stage-run.js). This represents a single FlowStage or sequence of FlowStages that can be run. + +This encapsulation allows us to, for example, more easily reason about tests in which the "book" stage could use either [Simple Booking Flow](https://openactive.io/open-booking-api/EditorsDraft/#simple-booking-flow) or [Booking Flow with Approval](https://openactive.io/open-booking-api/EditorsDraft/#booking-flow-with-approval), which involve different sets of API calls. + +## Jest Tests + +Flows, consisting of Flow Stages, run the underlying API calls which are being tested, via Jest in the various Test Suite tests. + +Jest tests involve a custom course of execution, in which setup occurs in `beforeEach`/`beforeAll` hooks, tests are run in `it` hooks, etc. Flow Stages are designed to work with this. + +Here is how Flow Stages slot into Jest's test execution lifecycle: + +- (Outside of hooks): A `FlowStage` class instance is created. The configuration of this FlowStage defines what will happen when it is run. +- `describe` hook: Contains the execution and checking logic for a given test. + - `beforeAll` hook: Runs the FlowStage. + - `it` hooks: Run success, validation, and any additional tests on the output of the FlowStage. + +## FlowStageUtils + +[FlowStageUtils](./flow-stage-utils.js) is a collection of utility functions which simplify and standardise the process of using FlowStages to write Test Suite tests. + +Of particular importance is the `describeRunAndRunChecks(..)` function, which creates a Jest `describe(..)` hook for a given **FlowStageRunnable** which performs the running and checking of the runnable as described in the **Jest Tests** section. + +## FlowStageRecipes + +Setting up a flow, consisting of multiple FlowStages where each feeds input into the next, can require a lot of boilerplate and repeated logic. In most cases, parts of these flows can be packaged up as they will be the same in lots of different tests. + +These packaged up flows are stored in [FlowStageRecipes](./flow-stage-recipes.js). + +An example is `initialiseSimpleC1C2BookFlow(..)`, which sets up a FetchOpportunities -> C1 -> C2 -> Book flow, which works with either [Simple Booking Flow](https://openactive.io/open-booking-api/EditorsDraft/#simple-booking-flow) or [Booking Flow with Approval](https://openactive.io/open-booking-api/EditorsDraft/#booking-flow-with-approval). This is used in many tests which simply make a booking with a certain configuration and simply check that it was successful or failed as expected. diff --git a/packages/openactive-integration-tests/test/helpers/flow-stages/flow-stage-run.js b/packages/openactive-integration-tests/test/helpers/flow-stages/flow-stage-run.js index 3847b595b4..4fb3bcd9fb 100644 --- a/packages/openactive-integration-tests/test/helpers/flow-stages/flow-stage-run.js +++ b/packages/openactive-integration-tests/test/helpers/flow-stages/flow-stage-run.js @@ -6,6 +6,8 @@ const { assertIsNotNullish } = require('../asserts'); */ /** + * A set of FlowStages and/or FlowStageRuns, which can be treated as a single sequence to be run. + * * @template {{ [stageName: string]: UnknownFlowStageType | FlowStageRun }} TStages */ class FlowStageRun { diff --git a/packages/openactive-integration-tests/test/helpers/flow-stages/flow-stage.js b/packages/openactive-integration-tests/test/helpers/flow-stages/flow-stage.js index 65cf847a61..a0daca3513 100644 --- a/packages/openactive-integration-tests/test/helpers/flow-stages/flow-stage.js +++ b/packages/openactive-integration-tests/test/helpers/flow-stages/flow-stage.js @@ -103,6 +103,9 @@ class FlowStage { * This input goes into `getInput`. It's a function as it will be called when * the FlowStage is run (rather than when the FlowStage is set up). Therefore, * it will have acccess to the output of any prerequisite stages. + * + * Note that this is usually used to get the output of a prerequisite stage, but + * is flexible to other use cases. * @param {string} args.testName Labels the jest `describe(..)` block * @param {(input: TInput) => Promise} args.runFn * @param {(flowStage: FlowStage) => void} args.itSuccessChecksFn @@ -198,7 +201,10 @@ class FlowStage { * * If there is a prerequisite stage, it will be run first. * - * The result is cached. + * The result is cached. This means that FlowStages will only be run once, while + * allowing for some flexibility on how they are run. For example, a test could only + * run the `run()` method on the last FlowStage in a flow, and this would automatically + * run all pre-requisite stages. */ run = pMemoize(async () => { // ## 1. Run prerequisite stage From 72b58740f95a3b3b46a2974e600a1ee1506bcac8 Mon Sep 17 00:00:00 2001 From: Luke Winship Date: Thu, 31 Aug 2023 17:20:30 +0100 Subject: [PATCH 2/7] integrate with architecture.md --- packages/openactive-integration-tests/ARCHITECTURE.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/packages/openactive-integration-tests/ARCHITECTURE.md b/packages/openactive-integration-tests/ARCHITECTURE.md index f62c55bf8d..170899c097 100644 --- a/packages/openactive-integration-tests/ARCHITECTURE.md +++ b/packages/openactive-integration-tests/ARCHITECTURE.md @@ -59,7 +59,8 @@ These consist of: - [Chakram](http://dareid.github.io/chakram/): This is a HTTP test framework designed for Mocha (however it works fine on Jest) - [Request helper](test/helpers/request-helper.js): This makes requests, and records the request + response against the logger. There are methods to directly make requests, along with methods for each API endpoint. -- [Flow Stages](test/helpers/flow-stages/flow-stage.js): A part of the booking flow (e.g. the C1 request). Use it to call the relevant API endpoint. When called, it stores the results, which can then be applied to successive Flow Stages (e.g. C1 output can be used for the C2 request). +- [Flow Stages](test/helpers/flow-stages/README.md): A part of the booking flow (e.g. the C1 request). Use it to call the relevant API endpoint. When called, it stores the results, which can then be applied to successive Flow Stages (e.g. C1 output can be used for the C2 request). + # Test flow - [Feature helper](test/helpers/feature-helper.js): This wraps up the initialisation of the test, implementing the describe blocks and initialising the logger. From 91da3e140bebaf062423454355dbd38b09df7649 Mon Sep 17 00:00:00 2001 From: Luke Winship Date: Thu, 31 Aug 2023 18:42:55 +0100 Subject: [PATCH 3/7] talk betterer Co-authored-by: civsiv --- .../test/helpers/flow-stages/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/openactive-integration-tests/test/helpers/flow-stages/README.md b/packages/openactive-integration-tests/test/helpers/flow-stages/README.md index c4f4a1b441..ae1c7122c4 100644 --- a/packages/openactive-integration-tests/test/helpers/flow-stages/README.md +++ b/packages/openactive-integration-tests/test/helpers/flow-stages/README.md @@ -1,6 +1,6 @@ # Flow Stages -A Flow is a sequence of API calls that are made by Test Suite, for a given test, against either the Booking System or the Broker Microservice. A Flow Stage encapsulates the configuration and logic for one of those API calls. e.g. there is a FlowStage for C1 (which is in the Booking System), a FlowStage for Fetch Opportunities (which uses the Broker Microservice), etc. +A Flow is a sequence of API calls that are made by Test Suite, for a given test, against either the Booking System or the Broker Microservice. A Flow is composed of several Flow Stages. A Flow Stage encapsulates the configuration and logic for one of those API calls. e.g. there is a FlowStage for C1 (which is in the Booking System), a FlowStage for Fetch Opportunities (which uses the Broker Microservice), etc. As an example, for [`opportunity-free-test`](packages/openactive-integration-tests/test/features/payment/free-opportunities/implemented/opportunity-free-test.js), when running in [Simple Booking Flow](https://openactive.io/open-booking-api/EditorsDraft/#simple-booking-flow), the following Flow Stages are utilised: From a1b4c805952e0b743f4ab181dd559607b990ac57 Mon Sep 17 00:00:00 2001 From: Luke Winship Date: Thu, 31 Aug 2023 18:45:14 +0100 Subject: [PATCH 4/7] not overridable. silly --- .../test/helpers/flow-stages/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/openactive-integration-tests/test/helpers/flow-stages/README.md b/packages/openactive-integration-tests/test/helpers/flow-stages/README.md index ae1c7122c4..7e930ca968 100644 --- a/packages/openactive-integration-tests/test/helpers/flow-stages/README.md +++ b/packages/openactive-integration-tests/test/helpers/flow-stages/README.md @@ -35,10 +35,10 @@ Each of these FlowStages does the following, when run in a test: FlowStages can then be queried after they've run in order to: 1. Check that the FlowStage was successful - - This is overridable per-FlowStage. For example, B's Flow Stage considers the run successful if the HTTP response has status 201 + - This is set per-FlowStage. For example, B's Flow Stage considers the run successful if the HTTP response has status 201 - FlowStage method: `itSuccessChecks()` 2. Perform validation checks on the output - - This is overridable per-FlowStage. In all cases, this is a case of calling [Validator](https://github.com/openactive/data-model-validator) on the HTTP output. + - This is set per-FlowStage. In all cases, this is a case of calling [Validator](https://github.com/openactive/data-model-validator) on the HTTP output. - FlowStage method: `itValidationTests()` ## FlowStageRunnable From fb9404e424240c13f57075514fd2e536fd3209fa Mon Sep 17 00:00:00 2001 From: Luke Winship Date: Thu, 31 Aug 2023 18:46:13 +0100 Subject: [PATCH 5/7] talkety Co-authored-by: civsiv --- .../test/helpers/flow-stages/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/openactive-integration-tests/test/helpers/flow-stages/README.md b/packages/openactive-integration-tests/test/helpers/flow-stages/README.md index 7e930ca968..3275dc6a44 100644 --- a/packages/openactive-integration-tests/test/helpers/flow-stages/README.md +++ b/packages/openactive-integration-tests/test/helpers/flow-stages/README.md @@ -43,7 +43,7 @@ FlowStages can then be queried after they've run in order to: ## FlowStageRunnable -An abstraction that can be either a **Flow Stage**, [**Book Recipe**](./book-recipe.js) or a [**Flow Stage Run**](./flow-stage-run.js). This represents a single FlowStage or sequence of FlowStages that can be run. +A FlowStageRunnable is an abstraction that can be either a **Flow Stage**, [**Book Recipe**](./book-recipe.js) or a [**Flow Stage Run**](./flow-stage-run.js). This represents a single FlowStage or sequence of FlowStages that can be run. This encapsulation allows us to, for example, more easily reason about tests in which the "book" stage could use either [Simple Booking Flow](https://openactive.io/open-booking-api/EditorsDraft/#simple-booking-flow) or [Booking Flow with Approval](https://openactive.io/open-booking-api/EditorsDraft/#booking-flow-with-approval), which involve different sets of API calls. From 8dc148d69d8138b85feb4303a9a91552517319e5 Mon Sep 17 00:00:00 2001 From: Luke Winship Date: Thu, 31 Aug 2023 18:47:14 +0100 Subject: [PATCH 6/7] semicolon king --- .../test/helpers/flow-stages/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/openactive-integration-tests/test/helpers/flow-stages/README.md b/packages/openactive-integration-tests/test/helpers/flow-stages/README.md index 3275dc6a44..620c606b9c 100644 --- a/packages/openactive-integration-tests/test/helpers/flow-stages/README.md +++ b/packages/openactive-integration-tests/test/helpers/flow-stages/README.md @@ -51,7 +51,7 @@ This encapsulation allows us to, for example, more easily reason about tests in Flows, consisting of Flow Stages, run the underlying API calls which are being tested, via Jest in the various Test Suite tests. -Jest tests involve a custom course of execution, in which setup occurs in `beforeEach`/`beforeAll` hooks, tests are run in `it` hooks, etc. Flow Stages are designed to work with this. +Jest tests involve a custom course of execution, in which setup occurs in `beforeEach`/`beforeAll` hooks; tests are run in `it` hooks; etc. Flow Stages are designed to work with this. Here is how Flow Stages slot into Jest's test execution lifecycle: From e1614609d9de4892c828d487ed36e9e9087890c0 Mon Sep 17 00:00:00 2001 From: Luke Winship Date: Thu, 31 Aug 2023 18:50:16 +0100 Subject: [PATCH 7/7] tsk Co-authored-by: civsiv --- .../test/helpers/flow-stages/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/openactive-integration-tests/test/helpers/flow-stages/README.md b/packages/openactive-integration-tests/test/helpers/flow-stages/README.md index 620c606b9c..2ca5250aa2 100644 --- a/packages/openactive-integration-tests/test/helpers/flow-stages/README.md +++ b/packages/openactive-integration-tests/test/helpers/flow-stages/README.md @@ -68,7 +68,7 @@ Of particular importance is the `describeRunAndRunChecks(..)` function, which cr ## FlowStageRecipes -Setting up a flow, consisting of multiple FlowStages where each feeds input into the next, can require a lot of boilerplate and repeated logic. In most cases, parts of these flows can be packaged up as they will be the same in lots of different tests. +Setting up a Flow, consisting of multiple FlowStages where each feeds input into the next, can require a lot of boilerplate and repeated logic. In most cases, parts of these flows can be packaged up as they will be the same in lots of different tests. These packaged up flows are stored in [FlowStageRecipes](./flow-stage-recipes.js).