V0.2.0b0 (#28)

* bump up the version * fix version format * remove fgops * fix handoff source search. fix dependency versions * attempt to fix broken doc on readthedocs * add a blankline after each include * add shell notation at the code blocks * add shell to begin quote * fix more quotes
anelendata · Aug 6, 2020 · 03ef42b · 03ef42b
1 parent c1078f4
commit 03ef42b
Show file tree

Hide file tree

Showing 37 changed files with 154 additions and 173 deletions.
diff --git a/.gitmodules b/.gitmodules
diff --git a/.travis.yml b/.travis.yml
@@ -24,4 +24,4 @@ script:
   - pytest -s tests/unit
   - pytest -s tests/local_run_no_install
   - pytest -s tests/local_run_install
-  - tests/install_test.sh
+    #  - tests/install_test.sh
diff --git a/HISTORY.md b/HISTORY.md
@@ -1,7 +1,17 @@
 ## History
 
+### 0.2.0b0 (2020-08-05)
+
+- Beta
+- Covers entire workflow from the local test to Fargate deployment
+- Clean extensible code-design (services and plugins)
+- Unit, integration, and installation tests
+- Interactive tutorial
+- Near-stable CLI version
+
 ### 0.1.2-alpha (2020-07-17)
 
+- Alpha
 - fix docker files to remove project dir
 - fix the issue of env var not passed to subprocesses
 
@@ -11,4 +21,4 @@ Just added a long description to setup.py for the pypi release.
 
 ### 0.1.0-alpha (2020-07-16)
 
-The first package build.
+Pilot. The first package build.
diff --git a/bin/cf_create_deploy_group b/bin/cf_create_deploy_group
diff --git a/bin/cf_create_deploy_role b/bin/cf_create_deploy_role
diff --git a/bin/cf_create_task b/bin/cf_create_task
diff --git a/bin/cf_delete_deploy_group b/bin/cf_delete_deploy_group
diff --git a/bin/cf_delete_task b/bin/cf_delete_task
diff --git a/bin/cf_update_deploy_group b/bin/cf_update_deploy_group
diff --git a/bin/cf_update_deploy_role b/bin/cf_update_deploy_role
diff --git a/bin/cf_update_task b/bin/cf_update_task
diff --git a/bin/docker_task b/bin/docker_task
diff --git a/bin/ecs_run_task b/bin/ecs_run_task
diff --git a/bin/events_schedule_create b/bin/events_schedule_create
diff --git a/bin/events_schedule_remove b/bin/events_schedule_remove
diff --git a/bin/iam_assume_role b/bin/iam_assume_role
diff --git a/bin/logs_pull_latest_stream b/bin/logs_pull_latest_stream
diff --git a/bin/mkdevenv b/bin/mkdevenv
diff --git a/deploy/fargate b/deploy/fargate
diff --git a/docs/01_run_local.md b/docs/01_run_local.md
@@ -7,10 +7,10 @@ using project 01_word_count.
 
 Each project directory contains:
 
-```
+```shell
 > ls -l 01_word_count
 ```
-```
+```shell
 
  files
  project.yml
@@ -19,11 +19,11 @@ Each project directory contains:
 
 project.yml looks like:
 
-```
+```shell
 > cat 01_word_count/project.yml
 ```
 
-```
+```shell
 commands:
   - command: cat
     args: "./files/the_great_dictator_speech.txt"
@@ -43,16 +43,16 @@ Here,
 
 The example from 01_word_count runs a command line equivalent of:
 
-```
+```shell
 cat ./files/the_great_dictator_speech.txt | wc -w
 ```
 
 Now let's run. Try entering this command below:
 
-```
+```shell
 > handoff --project 01_word_count --workspace workspace run local
 ```
-```
+```shell
 
 INFO - 2020-08-06 03:35:12,691 - handoff.config - Reading configurations from 01_word_count/project.yml
 INFO - 2020-08-06 03:35:12,693 - handoff.config - Setting environment variables from config.
@@ -69,7 +69,7 @@ INFO - 2020-08-06 03:35:13,130 - handoff.config - Job ended at 2020-08-06 03:35:
 
 If you see the output that looks like:
 
-```
+```shell
 
  INFO - 2020-08-03 04:51:01,971 - handoff.config - Reading configurations from 01_word_count/project.yml
  ...
@@ -81,10 +81,10 @@ If you see the output that looks like:
 Then great! You just ran the first local test. It created a workspace
 directory that looks like:
 
-```
+```shell
 > ls -l workspace
 ```
-```
+```shell
 
  artifacts
  config
@@ -93,11 +93,11 @@ directory that looks like:
 
 And the word count is stored at workspace/artifacts/state. Here is the content:
 
-```
+```shell
 > cat  workspace/artifacts/state
 ```
 
-```
+```shell
 644
 ```
 
@@ -112,7 +112,7 @@ Check out on YouTube: https://www.youtube.com/watch?v=J7GY1Xg6X20
 
 And here is the first few paragraphs of the text:
 
-```
+```shell
 
 I’m sorry, but I don’t want to be an emperor. That’s not my business. I don’t want to rule or conquer anyone. I should like to help everyone - if possible - Jew, Gentile - black man - white. We all want to help one another. Human beings are like that. We want to live by each other’s happiness - not by each other’s misery. We don’t want to hate and despise one another. In this world there is room for everyone. And the good earth is rich and can provide for everyone. The way of life can be free and beautiful, but we have lost the way.
 
@@ -123,11 +123,11 @@ Greed has poisoned men’s souls, has barricaded the world with hate, has goose-
 
 Now to the second example. This time project.yml looks like:
 
-```
+```shell
 > cat 02_collect_stats/project.yml
 ```
 
-```
+```shell
 commands:
   - command: cat
     args: ./files/the_great_dictator_speech.txt
@@ -139,19 +139,19 @@ commands:
 
 ...which is shell equivalent to
 
-```
+```shell
 cat ./files/the_great_dictator_speech.txt | python ./files/stats_collector.py | wc -w
 ```
 
 The script for the second command stats_collector.py can be found in
 02_collect_stats/files directory and it is a Python script that looks like:
 
 
-```
+```shell
 > cat 02_collect_stats/files/stats_collector.py
 ```
 
-```
+```python
 #!/usr/bin/python
 import io, json, logging, sys, os
 
@@ -188,10 +188,10 @@ The raw text is then processed by the third command (wc -w) and it conts the num
 
 Now let's run. Try entering this command below:
 
-```
+```shell
 > handoff --project 02_collect_stats --workspace workspace run local
 ```
-```
+```shell
 
 INFO - 2020-08-06 03:35:13,401 - handoff.config - Reading configurations from 02_collect_stats/project.yml
 INFO - 2020-08-06 03:35:13,402 - handoff.config - Setting environment variables from config.
@@ -209,11 +209,11 @@ INFO - 2020-08-06 03:35:13,881 - handoff.config - Processed in 0:00:00.049824
 Let's check out the contents of the second command:
 
 
-```
+```shell
 > cat workspace/artifacts/collect_stats.json
 ```
 
-```
+```shell
 {"rows_read": 15}
 ```
 

diff --git a/docs/02_exchange_rates.md b/docs/02_exchange_rates.md
@@ -10,11 +10,11 @@ in Python vitual environment.
 
 We will use 03_exchange_rates project. project.yml looks like:
 
-```
+```shell
 > cat 03_exchange_rates/project.yml
 ```
 
-```
+```shell
 commands:
   - command: "tap-exchangeratesapi"
     args: "--config config/tap-config.json"
@@ -40,8 +40,9 @@ deploy:
 
 ...which is shell equivalent to
 
+```shell
     tap-exchangeratesapi | python files/stats_collector.py | target-csv
-
+```
 
 
 Before we can run this, we need to install tap-exchangeratesapi and target-csv.
@@ -53,10 +54,10 @@ commands.
 
 To install everything, run this command:
 
-```
+```shell
 > handoff -p 03_exchange_rates -w workspace_03 workspace install
 ```
-```
+```shell
 
 INFO - 2020-08-06 03:35:14,158 - handoff.config - Reading configurations from 03_exchange_rates/project.yml
 INFO - 2020-08-06 03:35:14,240 - botocore.credentials - Found credentials in shared credentials file: ~/.aws/credentials
@@ -85,10 +86,10 @@ Successfully installed jsonschema-2.6.0 pendulum-1.2.0 python-dateutil-2.8.1 pyt
 
 Now let's run the task. Try entering this command below:
 
-```
+```shell
 > handoff -p 03_exchange_rates -w workspace_03 run local
 ```
-```
+```shell
 
 INFO - 2020-08-06 03:35:29,258 - handoff.config - Reading configurations from 03_exchange_rates/project.yml
 INFO - 2020-08-06 03:35:29,339 - botocore.credentials - Found credentials in shared credentials file: ~/.aws/credentials
@@ -109,14 +110,14 @@ INFO - 2020-08-06 03:35:33,964 - handoff.config - Processed in 0:00:04.268474
 
 This process should have created a CSV file in artifacts directory:
 
-```
+```shell
 
 exchange_rate-20200806T033530.csv
 ```
 
 ...which looks like:
 
-```
+```shell
 
 CAD,HKD,ISK,PHP,DKK,HUF,CZK,GBP,RON,SEK,IDR,INR,BRL,RUB,HRK,JPY,THB,CHF,EUR,MYR,BGN,TRY,CNY,NOK,NZD,ZAR,USD,MXN,SGD,AUD,ILS,KRW,PLN,date
 0.0127290837,0.0725398406,1.3197211155,0.4630976096,0.0618218792,2.9357569721,0.2215388446,0.007434429,0.0401958831,0.0863047809,135.1005146082,0.7041915671,0.050374336,0.6657569721,0.0625373506,1.0,0.29312749,0.0088188911,0.0083001328,0.0399311089,0.0162333997,0.0642571381,0.0655312085,0.0889467131,0.0142670983,0.158440405,0.0093592297,0.2132744024,0.0130336985,0.0134852258,0.032375498,11.244189907,0.0371372842,2020-07-10T00:00:00Z

diff --git a/docs/03_set_up_aws_account.md b/docs/03_set_up_aws_account.md
@@ -72,7 +72,7 @@ Enter Access key ID and Secret access key you created in the last step.
 For region, use one of these keys (for USA users, us-east-1 would do):
 
 
-```
+```shell
 
 ap-northeast-1
 ap-northeast-2