Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Capacity e2e test #689

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Capacity e2e test #689

wants to merge 7 commits into from

Conversation

saraswatpuneet
Copy link
Contributor

@saraswatpuneet saraswatpuneet commented Nov 19, 2024

Problem

Adding test for capacity checks.

  • Test basic capacity check for a tx with no capacity
  • Test basic capacity with some available capacity

Note: this does not test concurrent calls to payWithCapacity

@saraswatpuneet saraswatpuneet changed the title [WIP] Capacity e2e edge test Capacity e2e test Nov 19, 2024
@saraswatpuneet saraswatpuneet marked this pull request as ready for review November 19, 2024 20:41
@saraswatpuneet saraswatpuneet changed the title Capacity e2e test [WIP] Capacity e2e test Nov 19, 2024
@saraswatpuneet saraswatpuneet changed the title [WIP] Capacity e2e test Capacity e2e test Nov 19, 2024
]);
const requiredFee = await blockChainRpcQueryService.getCapacityCostForExt(tx.toHex());
expect(requiredFee.inclusionFee.adjustedWeightFee).toBeGreaterThan(0);
const feeRequiredToStake = 50n * requiredFee.inclusionFee.adjustedWeightFee;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rougly stake exactly the same amount as needed for tx for the test

@@ -21,18 +21,18 @@ import { u8aToHex } from '@polkadot/util';
export const FREQUENCY_API_WS_URL = process.env.FREQUENCY_API_WS_URL || 'ws://0.0.0.0:9944';
export const BASE_SEED_PHRASE = process.env.SEED_PHRASE || '//Alice';

export async function setupProviderAndUsers(numUsers = 4) {
export async function setupProviderAndUsers(seedPhrase = BASE_SEED_PHRASE, numUsers = 4) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe better to make seedPhrase the second argument so existing calls don't need to change? (And maybe baseSeedPhrase is a better argument name)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These mods aren't technically needed at all if we change this all to be a unit test (see my other comment), but it does still seem like a worthy enhancement to this setup function.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this could just be a unit test... it's not testing the actual application flow, just a single function checkTxCapacityLimit, so there's no benefit to initializing the entire application stack, other than to connect to the chain, and the return values from the chain can easily be mocked in a unit test.

An e2e test would be more like making a REST api call to the app (or queueing a task in BullMQ) and looking for the expected end state. Unfortunately we don't have an e2e framework set up to test the workers yet, but there is an issue for it.

Maybe just make this a unit test for now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree on that but wasn't sure if unit tests in ci are currently tested against the chain ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To test the functionality of checkTxCapacityLimit, we don't need to connect to the chain, just mock the return of the BlockchainRpcQueryService.capacityInfo() and BlockchainRpcQueryService.getCapacityCostForExt()methods using jest.spyOn(...)

You could just add the tests to libs/blockchain/blockchain.service.spec.ts

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right, I can just mock that away 👍🏽, word 'chain' gives a false sense of some critical mass

Will get that in my updates

Copy link
Contributor Author

@saraswatpuneet saraswatpuneet Nov 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JoeCap08055 second thought, I think this is more about an e2e of that rpc call from a second perspective, given there are couple of interactions happening here, and this is the only use case of this rpc computeCapacityFeeDetails, wont this classify as e2e ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we're getting into semantics here, but E2E tests should test complete application flows, not just single functions. If anything, a test of a single function that relies on external services (not mocked) would probably be an integration test (of which we currently have none).

The test as you've set it up is really more testing the logic pathways internal to the function than it is testing the actual RPC call to the blockchain. In that sense I still believe it still is better qualified as a unit test.

(This does beg the question of whether we need a middle tier of "integration tests" as distinct from both unit & e2e tests. In general I believe that we do, but realistically we just don't have the resources to support that.)

(This is my view of testing):

  • Unit tests -> test logic paths within individual functions; the application environment and all dependencies external to the function under test are mocked. Test results are usually checking return value of functions, etc.
  • Integration tests -> test complete multi-function pathways, including dependencies external to the function(s) under test, potentially including properly configured external services (ie, DBs, network services, etc). The application environment itself may be partially mocked, in the sense that we're not necessarily running the complete application, but just enough of the app framework to test a particular pathway. Test results are checking both return values of invoked function pathways, as well as some externally observable outcomes (data, events, network states, etc).
  • E2E tests -> test application behavior using only application APIs or data scenarios. The complete application environment is spun up. Test results are based on externally observable outcomes only (application API responses, events, data states)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants