diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 39beedc2..b5fd2baa 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -56,7 +56,11 @@ brew update && brew upgrade openapi-generator ``` -## How to run integration test +# How to run server or integration test + +## Prepare Cadence/Temporal environment +iWF server depends on Cadence or Temporal. You need at least one to be ready for running with iWF . +Or maybe both just for testing to ensure the code works for both Cadence and Temporal. ### Option 1: Run with our docker-compose file (Recommended) @@ -65,61 +69,41 @@ Simply run `docker compose -f docker-compose/integ-dependencies.yml up -` will: * Start both Cadence & Temporal as dependencies * Set up required system search attributes * Set up customized search attributes for integration test(`persistence_test.go`) -* Temporal WebUI: http://localhost:8080/ +* Temporal WebUI: http://localhost:8233/ * Cadence WebUI: http://localhost:8088/ -NOTE: You need to wait for up to 60s before you can run with Cadence because of -this [issue](https://github.com/uber/cadence/issues/5076). - -Then run the whole integ test suite against Cadence+Temporal service by this command: - -`make integTests` - -Or run `make temporalIntegTests` if you don't want to wait :/ - ### Option 2: Run with your own Temporal service -NOTE: For local testing it's recommended to use [temporal docker-compose](https://github.com/temporalio/docker-compose) +First of all, you need a Temporal service if you haven't had it: -NOTE: For Temporalite following the [instruction](https://github.com/temporalio/temporalite). If you see -error `error setting up schema`, try use command `temporalite start --namespace default -f my_test.db` instead to start. +Option 1 (recommended): use [Temporal CLI](https://github.com/temporalio/cli) -- `temporal server start-dev` -NOTE: Temporal docker compose uses http://localhost:8080/ but Temporalite uses http://localhost:8233/ for WebUI +Option 2: use [temporal docker-compose](https://github.com/temporalio/docker-compose) -Assuming you already have a Temporal service : -1. Make sure you have a default namespace +Assuming you are using `default` namespace: -```shell -tctl --ns default n re -``` - -2. Make sure you have registered system search attributes required by iWF server +1. Make sure you have registered system search attributes required by iWF server ```shell -tctl adm cl asa -n IwfWorkflowType -t Keyword -tctl adm cl asa -n IwfGlobalWorkflowVersion -t Int -tctl adm cl asa -n IwfExecutingStateIds -t Keyword - + temporal operator search-attribute create -name IwfWorkflowType -type Keyword + temporal operator search-attribute create -name IwfGlobalWorkflowVersion -type Int + temporal operator search-attribute create -name IwfExecutingStateIds -type KeywordList ``` -3. For `persistence_test.go` integTests, you need to register below custom search attributes. +2. For `persistence_test.go` integTests, you need to register below custom search attributes. ```shell -tctl adm cl asa -n CustomKeywordField -t Keyword -tctl adm cl asa -n CustomIntField -t Int -tctl adm cl asa -n CustomBoolField -t Bool -tctl adm cl asa -n CustomDoubleField -t Double -tctl adm cl asa -n CustomDatetimeField -t Datetime -tctl adm cl asa -n CustomStringField -t text + temporal operator search-attribute create -name CustomKeywordField -type Keyword + temporal operator search-attribute create -name CustomIntField -type Int + temporal operator search-attribute create -name CustomBoolField -type Bool + temporal operator search-attribute create -name CustomDoubleField -type Double + temporal operator search-attribute create -name CustomDatetimeField -type Datetime + temporal operator search-attribute create -name CustomStringField -type Text ``` -4. If you run into any issues with Search Attributes registration, use the below command to check the existing Search - attributes:`tctl adm cl get-search-attributes` - -Then run the whole integ test suite against Cadence+Temporal service by this command: - -`make temporalIntegTests` +3. If you run into any issues with Search Attributes registration, use the below command to check the existing Search + attributes:`temporal operator search-attribute list` ### Option 3: Run with your own Cadence service @@ -149,9 +133,31 @@ you run the test too early, you may see error: `"IwfWorkflowType is not a valid `CustomKeywordField, CustomIntField, CustomBoolField, CustomBoolField, CustomDoubleField, CustomDatetimeField, CustomStringField` 6. If you run into any issues with Search Attributes registration, use the below command to check the existing Search - attributes: - `cadence cl get-search-attr` + attributes: `cadence cl get-search-attr` + +## Run the server + +The first step you may want to explore is to run it locally! + +To run the server with Temporal +* If you are in an IDE, you can run the main function in `./cmd/main.go` with argument `start`. +* Or in terminal `go run cmd/server/main.go start` +* Or build the binary and run it by `make bins` and then run `./iwf-server start` + +To run with Cadence, make sure you specify the cadence config `--config config/development_cadence.yaml start`: +* In an IDE, you can run the main function in `./cmd/main.go` with argument ` --config config/development_cadence.yaml start`. +* Or in terminal `go run cmd/server/main.go --config config/development_cadence.yaml start` +* Or build the binary and run it by`make bins` and then run `./iwf-server --config config/development_cadence.yaml start` + +## Run the integration tests +For development, you may want to run the test locally for debugging, especially your PR has failed the tests in CI pipeline. + +:warning: NOTE: When running with local Cadence, you may need to wait for up to 60s for Search attributes to be ready, because of +this [issue](https://github.com/uber/cadence/issues/5076). + +* To run the whole integ test suite against Cadence+Temporal service by this command `make integTests` +* To run the whole suite for Temporal only `make temporalIntegTests` +* To run the whole suite for Cadence only `make cadenceIntegTests` +* To run a specify test case or a test file, you can utilize the IDE or `go test` command. -Then run the whole integ test suite against Cadence+Temporal service by this command: -`make cadenceIntegTests` diff --git a/README.md b/README.md index 9ef17773..ed077ed1 100644 --- a/README.md +++ b/README.md @@ -54,8 +54,8 @@ With some other existing technologies, you solve it using message queue(like SQS * Need to re-enqueue the message for larger backoff * Using visibility timeout for durable timer * Need to re-enqueue the message for once to have 24 hours timer -* Need to create one topic for every step -* Need additional storage for processing ready signal +* Need to create one queue for every step +* Need additional storage for waiting & processing ready signal * Only go to 3 or 4 if both conditions are met * Also need DLQ and build tooling around diff --git a/docker-compose/init-ci-temporal.sh b/docker-compose/init-ci-temporal.sh index ec47bfcf..a9fa1d00 100755 --- a/docker-compose/init-ci-temporal.sh +++ b/docker-compose/init-ci-temporal.sh @@ -1,11 +1,8 @@ #!/bin/bash -# use new version of tctl so that we can skip the prompt -tctl config set version next - checkExists () { # see https://github.com/temporalio/temporal/issues/4160 - if tctl search-attribute list | grep -q "$1"; then + if temporal operator search-attribute list | grep -q "$1"; then return 0 else return 1 @@ -15,23 +12,24 @@ fi echo "now trying to register iWF system search attributes..." for run in {1..120}; do sleep 1 - tctl search-attribute create -name IwfWorkflowType -type Keyword -y + temporal operator search-attribute create -name IwfWorkflowType -type Keyword sleep 0.1 - tctl search-attribute create -name IwfGlobalWorkflowVersion -type Int -y + temporal operator search-attribute create -name IwfGlobalWorkflowVersion -type Int + sleep 0.1 + temporal operator search-attribute create -name IwfExecutingStateIds -type KeywordList sleep 0.1 - tctl search-attribute create -name IwfExecutingStateIds -type Keyword -y + temporal operator search-attribute create -name CustomKeywordField -type Keyword sleep 0.1 - tctl search-attribute create -name CustomKeywordField -type Keyword -y + temporal operator search-attribute create -name CustomIntField -type Int sleep 0.1 - tctl search-attribute create -name CustomIntField -type Int -y + temporal operator search-attribute create -name CustomBoolField -type Bool sleep 0.1 - tctl search-attribute create -name CustomBoolField -type Bool -y + temporal operator search-attribute create -name CustomDoubleField -type Double sleep 0.1 - tctl search-attribute create -name CustomDoubleField -type Double -y + temporal operator search-attribute create -name CustomDatetimeField -type Datetime sleep 0.1 - tctl search-attribute create -name CustomDatetimeField -type Datetime -y + temporal operator search-attribute create -name CustomStringField -type Text sleep 0.1 - tctl search-attribute create -name CustomStringField -type text -y if checkExists "IwfWorkflowType" ] && checkExists "IwfGlobalWorkflowVersion" && checkExists "IwfExecutingStateIds" && checkExists "CustomKeywordField" && checkExists "CustomIntField" && checkExists "CustomBoolField" && checkExists "CustomDoubleField" && checkExists "CustomDatetimeField" && checkExists "CustomStringField" ] ; then echo "All search attributes are registered" diff --git a/docker-compose/init-compose.sh b/docker-compose/init-compose.sh index e8a924b1..ef29ad83 100755 --- a/docker-compose/init-compose.sh +++ b/docker-compose/init-compose.sh @@ -1,11 +1,8 @@ #!/bin/bash -# use new version of tctl so that we can skip the prompt -tctl config set version next - checkExists () { # see https://github.com/temporalio/temporal/issues/4160 - if tctl search-attribute list | grep -q "$1"; then + if temporal operator search-attribute list | grep -q "$1"; then return 0 else return 1 @@ -16,11 +13,11 @@ echo "now trying to register iWF system search attributes..." for run in {1..60}; do sleep 1 - tctl search-attribute create -name IwfWorkflowType -type Keyword -y - sleep 0.1 - tctl search-attribute create -name IwfGlobalWorkflowVersion -type Int -y + temporal operator search-attribute create -name IwfWorkflowType -type Keyword sleep 0.1 - tctl search-attribute create -name IwfExecutingStateIds -type Keyword -y + temporal operator search-attribute create -name IwfGlobalWorkflowVersion -type Int + sleep 0.1 + temporal operator search-attribute create -name IwfExecutingStateIds -type KeywordList sleep 0.1 if checkExists "IwfWorkflowType" ] && checkExists "IwfGlobalWorkflowVersion" && checkExists "IwfExecutingStateIds" ] ; then echo "All search attributes are registered"