transformer

Transformer application - simple data copy from one datastore to another, i.e. (aerospike to mysql)

This application was build to provide end to end testing example of etl application.

The end to end test provide use cases to test aerospike to mysql backup with and without transformation.

Prerequisites:

Enable ssh logic you your use on your machine (on osx System Preference / Sharing / Remote Login )

Install docker service

Download endly

Provide a username and password to login to your box.

endly -c=localhost

Verify that secret file were created

cat ~/.secret/localhost.json

Create 'mysql' secret credentials, provide root as username and non empty password for docker mysqladmin

endly -c=mysql

Run reporter webservice workflow

Run the following command:

git clone https://github.com/viant/endly
cd endly/example/etl/transformer/e2e/

run test with manager workflow:

endly -w=manager

run test with inline workflowrun request

endly -r=run

To check manager workflow tasks list

endly -w=manager -t='?'

#Troubleshooting

to check you aerospike just run

docker exec -it mydb4 aql SHOW sets;

docker exec -it mydb3 mysql show tables;