Skip to content

Commit

Permalink
updated docs for how to use prebuilt images (#78)
Browse files Browse the repository at this point in the history
  • Loading branch information
@sreuland
sreuland authored Sep 27, 2023
1 parent 352cc1d commit 3fd3ba4
Show file tree
Hide file tree
Showing 2 changed files with 52 additions and 38 deletions.
6 changes: 4 additions & 2 deletions .github/workflows/Dockerfile.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,14 @@ env:
HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
# the only time system test image needs to be defined for a push is when it's a non 'v' tag creation
PUSH_ENABLED: ${{ secrets.DOCKERHUB_TOKEN && github.ref_type == 'tag' && startsWith(github.ref_name, 'v') != true && 'true' || 'false' }}
IMAGE: ${{ format('{0}/{1}:{2}', secrets.DOCKERHUB_TOKEN && 'docker.io' || 'ghcr.io', github.repository, github.event.ref || 'latest') }}
IMAGE: ${{ format('{0}/{1}:{2}', secrets.DOCKERHUB_TOKEN && 'docker.io' || 'ghcr.io', github.repository, github.ref_type == 'tag' && github.ref_name || github.event.ref || 'latest') }}
CORE_GIT_REF: https://github.com/stellar/stellar-core.git#master
CORE_COMPILE_CONFIGURE_FLAGS: "--disable-tests --enable-next-protocol-version-unsafe-for-production"
CORE_COMPILE_CONFIGURE_FLAGS: "--disable-tests"
SOROBAN_RPC_GIT_REF: https://github.com/stellar/soroban-tools.git#main
RUST_TOOLCHAIN_VERSION: stable
SOROBAN_CLI_GIT_REF: https://github.com/stellar/soroban-tools.git#main
QUICKSTART_GIT_REF: https://github.com/stellar/quickstart.git#master
JS_SOROBAN_CLIENT_NPM_VERSION: https://github.com/stellar/js-soroban-client.git#main
jobs:
complete:
if: always()
Expand All @@ -50,6 +51,7 @@ jobs:
RUST_TOOLCHAIN_VERSION=${{ env.RUST_TOOLCHAIN_VERSION }} \
SOROBAN_CLI_GIT_REF=${{ env.SOROBAN_CLI_GIT_REF }} \
SYSTEM_TEST_IMAGE=${{ env.PUSH_ENABLED == 'true' && env.IMAGE || 'stellar/system-test:dev' }} \
JS_SOROBAN_CLIENT_NPM_VERSION=${{ env.JS_SOROBAN_CLIENT_NPM_VERSION }} \
QUICKSTART_GIT_REF=${{ env.QUICKSTART_GIT_REF }} build;
- if: ${{ env.PUSH_ENABLED == 'true' }}
name: Save Docker Image to file
Expand Down
84 changes: 48 additions & 36 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# System Test

### Runing tests using the docker image:
To run tests, requires two steps:
(1) First build the system test docker image with the correct versions of core,
horizon, soroban rpc, rust toolchain, soroban cli, this will create a docker image named
### Runing system tests:
Identify the system-test image you want to use for running tests:
- Use a prebuilt system test image published as tags under `dockerhub.io/stellar/system-test`
- Build the system test docker image locally with specific versions of core, horizon, soroban rpc, rust toolchain, soroban cli, this will create a docker image named
`stellar/system-test:dev`.
All `GIT_REF` variables can refer to either a fully qualified local path to checked out git repo, or a fully qualified github remote repo url `https://github.com/repo#<ref>`
```
Expand All @@ -23,7 +23,7 @@ To run tests, requires two steps:

example of build using specific git refs, mainline from repos in this example, or use tags, branches, etc:
```
$ make CORE_GIT_REF=https://github.com/stellar/stellar-core.git#f1dc39f0f146815e5e3a94ed162e2f0639cb433f \
make CORE_GIT_REF=https://github.com/stellar/stellar-core.git#f1dc39f0f146815e5e3a94ed162e2f0639cb433f \
CORE_COMPILE_CONFIGURE_FLAGS="--disable-tests --enable-next-protocol-version-unsafe-for-production" \
SOROBAN_RPC_GIT_REF=https://github.com/stellar/soroban-tools.git#main \
RUST_TOOLCHAIN_VERSION=stable \
Expand All @@ -33,9 +33,9 @@ To run tests, requires two steps:
build
```

example of build using the existing quickstart:soroban-dev image which has latest released soroban server versions and builds soroban cli from local directory of checked out soroban-tools repo:
example of build using an existing quickstart image, this can dramatically speed up the build time, as the existing quickstart image will provide the pre-compiled rpc, and core runtimes already:
```
$ make QUICKSTART_IMAGE=stellar/quickstart:soroban-dev \
make QUICKSTART_IMAGE=stellar/quickstart:soroban-dev \
RUST_TOOLCHAIN_VERSION=1.66.0 \
SOROBAN_CLI_GIT_REF=/Users/user/soroban-tools build
```
Expand All @@ -46,8 +46,10 @@ To run tests, requires two steps:
SOROBAN_RPC_GIT_REF=https://github.com/stellar/soroban-tools.git#main
RUST_TOOLCHAIN_VERSION=stable
QUICKSTART_GIT_REF=https://github.com/stellar/quickstart.git#master
GO_GIT_REF=https://github.com/stellar/go.git#soroban-xdr-next
CORE_COMPILE_CONFIGURE_FLAGS="--disable-tests --enable-next-protocol-version-unsafe-for-production"
# the GO_GIT_REF provides the reference on the stellar/go repo from which
# to build horizon
GO_GIT_REF=https://github.com/stellar/go.git#master
CORE_COMPILE_CONFIGURE_FLAGS="--disable-tests"
CORE_GIT_REF=https://github.com/stellar/stellar-core.git#master
JS_SOROBAN_CLIENT_NPM_VERSION=https://github.com/stellar/js-soroban-client.git#main
```
Expand All @@ -58,7 +60,7 @@ To run tests, requires two steps:
SOROBAN_CLI_CRATE_VERSION=0.4.0
# this will override the default Node JS vm version used for running the JS code:
NODE_VERSION=14.20.0
NODE_VERSION=16.20.2
# soroban js client version can be set to a published npm version on https://www.npmjs.com/package/soroban-client
JS_SOROBAN_CLIENT_NPM_VERSION=latest
Expand All @@ -70,40 +72,34 @@ To run tests, requires two steps:
# platform arch as the build host is running on, i.e. linux/amd64 or linux/arm64.
# Otherwise, build will fail if image is not available for matching host platform.
#
# this will skip building core, horizon, rpc and quickstart from git source and instead
# this will skip building from source for core(CORE_GIT_REF), rpc(SOROBAN_RPC_GIT_REF) and quickstart(QUICKSTART_GIT_REF), instead
# will use the versions already compiled in the existing quickstart docker image provided:
QUICKSTART_IMAGE=<docker registry>/<docker image name>:<docker tag>
# this will skip building core from git source and instead
# this will skip building core from CORE_GIT_REF and instead
# will use the bin already compiled at /usr/local/bin/stellar-core in the existing docker image provided:
CORE_IMAGE=<docker registry>/<docker image name>:<docker tag>
# this will skip building soroban-rpc from git source and instead
# this will skip building soroban-rpc from SOROBAN_RPC_GIT_REF and instead
# will use the bin already compiled at /bin/soroban-rpc in the existing docker image provided:
SOROBAN_RPC_IMAGE=<docker registry>/<docker image name>:<docker tag>
# this will skip building soroban-cli from git source and instead
# this will skip building soroban-cli from SOROBAN_CLI_GIT_REF and instead
# will use the bin already compiled at /usr/local/cargo/bin/soroban in the existing docker image provided:
SOROBAN_CLI_IMAGE=<docker registry>/<docker image name>:<docker tag>
# this will skip building horizon from git source and instead
# this will skip building horizon from GO_GIT_REF and instead
# will use the bin already compiled at /go/bin/horizon in the existing docker image provided:
HORIZON_IMAGE=<docker registry>/<docker image name>:<docker tag>
# this will skip building friendbot from git source and instead
# this will skip building friendbot from GO_GIT_REF and instead
# will use the bin already compiled at /app/friendbot in the existing docker image provided:
FRIENDBOT_IMAGE=<docker registry>/<docker image name>:<docker tag>
```

(2) Run the system test docker image:
```
docker run --rm -it --name e2e_test stellar/system-test:dev --VerboseOutput false
```

Optional parameters to pass when running system-test image, `stellar/system-test:<tag>`:

Optional settings to pass when running system-test image, `stellar/system-test:<tag>`:

To specify git version of the smart contract source code used as test fixtures.
To specify git version of the smart contract source code used in soroban test fixtures.
`--SorobanExamplesGitHash {branch, tag, git commit hash}`
`--SorobanExamplesRepoURL "https://github.com/stellar/soroban-examples.git"`

Expand All @@ -112,14 +108,12 @@ To specify which system test feature/scenarios to run, it is a regex of the feat
or
`--TestFilter "^TestDappDevelop$/^DApp developer compiles, deploys and invokes a contract#01$"`

The ending wildcard allows for all combinations of example data for a scenario outline, without that it would just run the first example data set in a scenario outline.

The default target network for system tests is a new/empty instance of standalone network hosted inside the docker container, tests will use the default root account already seeded into standalone network. Alternatively, can override the network settings here:
* Tests will use an internally hosted core node connected to standalone or futurenet network:
`--TargetNetwork {standalone|futurenet}`
* Tests will use an external rpc instance and the container will not run core, horizon, rpc services internally:
The default target network for system tests is a new/empty instance of local network hosted inside the docker container, tests will use the default root account already seeded into local network. Alternatively, can override the network settings for local and remote usages:
- Tests will use an internally hosted core watcher node:
`--TargetNetwork {standalone|futurenet|testnet}`
- Tests will use an external rpc instance and the container will not run core, horizon, rpc services internally:
`--TargetNetworkRPCURL {http://<rpc_host:rpc_port>/soroban/rpc}`
* Tests use these settings in either target network mode, and these are by default set to work with standalone:
- Tests use these settings in either target network mode, and these are by default set to work with local:
`--TargetNetworkPassphrase "{passphrase}"`
`--TargetNetworkTestAccountSecret "{your test account key pair info}"`
`--TargetNetworkTestAccountPublic "{your test account key pair info}"`
Expand All @@ -128,9 +122,28 @@ Debug mode, the docker container will exit with error code when any pre-setup or
you can enable DEBUG_MODE flag, and the container will stay running, prompting you for enter key before shutting down, make sure you invoke docker with `-it` so the prompt will reach your command line. While container is kept running, you can shell into it via `docker exec -it <container id or name>` and view log files of services in the stack such as core, rpc located in container at `/var/log/supervisor`.
`--DebugMode=true`


The docker run follows standard exit code conventions, so if all tests pass in the container run, exit code from command line execution will be 0, otherwise, if any failures in container or tests, then exit code will be greater than 0.

#### Running Test Examples
- Run tests against an instance of core and rpc on a local network all running in the test container:

```
docker run --rm -t --name e2e_test stellar/system-test:<tag> \
--VerboseOutput true
```

- Run tests against a remote instance of rpc configured for testnet, this will not run core or rpc instances locally in the test container. It requires you provide a key pair of an account that is funded with Lumens on the target network for the tests to use as source account on transactions it will submit to target network:

```
docker run --rm -t --name e2e_test stellar/system-test:<tag> \
--VerboseOutput true \
--TargetNetworkRPCURL https://<rpc host url> \
--TargetNetworkPassphrase "Test SDF Network ; September 2015" \
--TargetNetworkTestAccountSecret <your test account key pair info> \
--TargetNetworkTestAccountPublic <your test account key pair info> \
--SorobanExamplesGitHash v20.0.0-rc2
```


### Development mode and running tests directly from checked out system-test repo.
This approach allows to run the tests from source code directly on host as go tests, no docker image is used.
Expand All @@ -139,17 +152,16 @@ This approach allows to run the tests from source code directly on host as go te

1. go 1.18 or above - https://go.dev/doc/install
2. rust toolchain(cargo and rustc), install the version per testing requirements or stable, - use rustup - https://www.rust-lang.org/tools/install
3. `soroban`, compile or install via cargo crate a version of soroban cli onto your machine and accessible from PATH.
3. `soroban` cli, compile or install via cargo crate a version of soroban cli onto your machine and accessible from PATH.
4. target network stack for the tests to access soroban-rpc instance. You can use an existing/running instance if reachable or can use the quickstart image `stellar/quickstart:soroban-dev` from dockerhub to run the latest stable target network stack locally, or build quickstart with specific versions of core, horizon and soroban rpc first [following these instructions](https://github.com/stellar/quickstart#building-custom-images) and run `stellar/quickstart:dev` locally.
```
docker run --rm -it -p 8000:8000 --name stellar stellar/quickstart:dev --standalone --enable-soroban-rpc
```
5. locally checkout stellar/system-test GH repo and go into top folder - `git clone https://github.com/stellar/system-test.git;cd system-test`
#### Running tests
#### Running tests locally as go programs
```
# example values used here are for when running quickstart:soroban-dev standalone locally
system-test $ SorobanExamplesGitHash="main" \
SorobanExamplesRepoURL="https://github.com/stellar/soroban-examples.git" \
TargetNetworkPassPhrase="Standalone Network ; February 2017" \
Expand All @@ -170,6 +182,6 @@ This example uses a feature/scenario filter also to limit which tests are run.
#### Debugging tests
Have provided a debug config [launch.json](.vscode/launch.json) for example reference on how to run a test with the go/dlv debugger to step through, etc.
A debug config [launch.json](.vscode/launch.json) is provided for example reference on how to run a test with the go/dlv debugger.

0 comments on commit 3fd3ba4

Please sign in to comment.