Often times when you show this demo you may want to start from a zero data situation to best illustrate how data flows in accumulates. To accomplish this here is a tear down script that will reset all the components necessary to have re-establish a clean data environment.
Step 1: Remove source01 by issuing the following DC/OS CLI command
- dcos marathon app remove source01
Step 2: Remove rat01 by issuing the following DC/OS CLI command
- dcos marathon app remove rat01
Step 3: Verify that source01 and rat01 are both removed from the application listing in the Marathon dashboard
Step 4: Remove the Elasticsearch index and all of it's assoicated data by issuing the following command from the ssh tunnel shell of the master node
- curl -XDELETE 'http://spatiotemporal-store.elasticsearch.mesos:9200/taxi'
- to verify you can run: curl -XGET 'http://spatiotemporal-store.elasticsearch.mesos:9200/*' and see that you get empty results
Step 5: Kafka consumer group offsets need to be cleared out so old data is not propogated to consumer when they reconnect using the same consumer group. To do this the Kafka offsets are managed in the distributed config store state of the DC/OS cluster called Zookeeper. There is Zookeeper Exhibitor dashboard that can be used to accomplish this. To connect to Zookeeper Exhibitor:
- On Azure, with the SSH tunnel in place you can connect to the DC/OS dashboard with http://localhost:9001/exhibitor.
- On Amazon, obtain the public IP address of your master(s) and connect with /exhibitor.
Step 6: In Zookeeper Exhibitor click on the 'Explorer' tab, expand the folder, expand the 'dcos-service-kafka' folder, expand the 'consumers' folder, and click on the 'taxi-consumer-group':
Step 7: With 'taxi-consumer-group' selected click on the 'Modify...' button, in the dialog box select a 'Type' value of 'Delete', fill in the Username/Ticket Code/Reason with a value of 'a' for each, and click the 'Next...' button.
Step 8: Click 'OK' on the 'Modify Node' confirmation dialog:
All previous demo run state is now cleaned up so you can proceed to re-run the demo starting with the instructions at schedule a real-time analytic task & a source that emits events.