Merge pull request #13 from wizzdev-pl/devel

Merge-devel-v1.0.4

.gitignore

@@ -4,6 +4,7 @@ terraform/.terraform/*
 
 # terraform bin
 terraform/terraform
+terraform/terraform.exe
 
 #JSON aws configuration file
 MicroPython/src/aws_config.json
@@ -12,6 +13,8 @@ MicroPython/src/aws_config.json
 *.pyc
 venv/
 .idea/
+*.vscode/
+.env
 
 
 Misc files

MicroPython/README.md

@@ -1,64 +1,76 @@
 ### Requirements
-- Installed python3.6-dev
-- Installed python3.6-venv
-- Installed python3-pip
+- Created virtual environment
 - User added to dialup group
 - Configured AWS cloud
 
 
 ### To develop in PyCharm:
-1. Create and activate virtual environment with Python 3.6 using PyCharm GUI
-or do it manually.
-````bash
-cd Micropython
-virtualenv venv
-pip3 install -r requirements.txt
-````
+1. Activate previously created environment using venv or anaconda.
+
+    ```
+    cd Micropython
+    ```
+    This step is platform dependent:
+    #### Windows
+    * For anaconda:
+        ```
+        conda activate ENV_NAME
+        ```
+
+    * For venv:
+        ```
+        ../terraform/scripts/ENV_NAME/Scripts/activate.bat
+        ```
+
+    #### Linux/ Mac OS
+    * For anaconda:
+        ```
+        conda activate ENV_NAME
+        ```
+
+    * For venv:
+        ```
+        source ../terraform/scripts/ENV_NAME/bin/activate
+        ```
 
 2. Open the project MicroPython with PyCharm and mark *src* and *ulib_mocks* directory as Sources Root in Pycharm.
 
 ### AWS and terraform 
 Make sure that your AWS cloud is configured. For more information please go to 
-ReadMe in "terraform" directory [here](../terraform/README.md).
+README in "terraform" directory [here](../terraform/README.md).
 
 If you already have configured AWS infrastructure, make sure that:
-- terraform binaries (https://www.terraform.io/downloads.html) are in ".terraform" directory.
+- terraform exists either in ".terraform" directory or installed through the package manager
 - you have configured ssh connections with AWS (aws configure)
 
 ### Basic Setup of the ESP32
 To set up a new board or flash the old one. 
 Make sure that your AWS cloud is configured, and your computer has AWS credentials.
 
-#### Creating virtual environment 
-If you already have made virtual environment, you can skip this step.
-```bash
-cd Micropython
-python3 -m venv venv
-```
-
-#### Activating virtual environment
-This step is platform dependent
-###### Windows
-```bash
-venv/Scripts/activate.bat
-```
 
-###### Linux/ Mac OS
-```bash
-source venv/bin/activate
-```
+#### Flashing the board
+Make sure that your board is connected to the computer and you have activated your virtual environment. 
+Check the port number - on Linux system, port can be checked through the simple script which will list all usb devices and their ports:
 
-#### Install requirements
 ```bash
-pip3 install -r requirements.txt
+#!/bin/sh
+
+for sysdevpath in $(find /sys/bus/usb/devices/usb*/ -name dev); do
+    (
+        syspath="${sysdevpath%/dev}"
+        devname="$(udevadm info -q name -p $syspath)"
+        [[ "$devname" == "bus/"* ]] && exit
+        eval "$(udevadm info -q property --export -p $syspath)"
+        [[ -z "$ID_SERIAL" ]] && exit
+        echo "/dev/$devname - $ID_SERIAL"
+    )
+done
 ```
 
-#### Flashing the board
-Make sure that your board is connected to computer. Check the port number.
+After finding the correct port, execute:
 
 ```bash
-source venv/bin/activate
-python3 scripts/upload_all.py -p <port>
+python scripts/upload_all.py -p <port>
 ```
 
 After flashing the board please reset it using button EN button.
@@ -80,40 +92,57 @@ http://192.168.4.1/web_pages/setup.html
 ```
 Next, type your WiFi network credentials into SSID and Password fields and press
 "Submit" button. Remember that this network should have an internet connection. 
-Otherwise, data will not be sent to the AWS.
+Otherwise, data will not be sent to the AWS. 
+If you’ve lost connection after submitting, don’t worry - that is supposed to happen.
 
 ### Logs from the device
 If you want to see logs from working device you should access serial port 
 communication with board. We recommend you programs listed below.
 
-#### Linux
+### Linux
 On Linux you can use picocom:
+
+* Ubuntu:
+```bash
+sudo apt-get install picocom 
+```
+
+* Fedora:
 ```bash
-sudo apt-get install picocom
-picocom port_name --baud 115200
+sudo dnf install picocom 
 ```
 
-#### Windows 
+To run:
+```bash
+picocom <port> --baud 115200
+```
+
+### Windows 
 On Windows you can use PuTTy _(https://www.putty.org/)_
 
-#### Mac 
+After installing connect to the device through:
+1. Select Session in Category
+2. Select Serial in Connection type
+3. Type in correct port (Serial line) and speed
+4. Open connection
+
+### Mac 
 On Linux you can also use picocom
 
 ### Checking measurement
-There are two ways to check measurements sent from the board. First is using AWS IoT core service,
-second is visualization page prepared by WizzDev.
+There are two ways to check measurements sent from the board. First is using AWS IoT core service, second is visualization page prepared by WizzDev.
 
 #### Using IoT Core
-Log in to your AWS account. Go to 'IoT Core' service. Next, choose "Test" and into "Subscription topic" 
-type:
+Log in to your AWS account. Go to 'IoT Core' service. Next, choose "Test" and into "Subscription topic" type:
 ```
 topic/data
 ```
 Finally, click "Subscribe to topic". After some time you should see dataframes send by ESP32 board.
 
 #### Visualization page
-Visualization page url is generated by AWS during infrastructure build. So URL is connected with your
-AWS account. You can find this URL in places listed below:
+Visualization page url is generated by AWS during infrastructure build. So URL is connected with your AWS account. 
+You can find this URL in places listed below:
 - in file MicroPython/src/aws_config.json as **visualization_url**
+- in the "CloudFront" service on your AWS account under the **Domain Name** field
 
 

MicroPython/requirements.txt

@@ -1,5 +1,5 @@
 wheel
 esptool==2.8
+gitpython==3.1.17
 adafruit-ampy==1.0.5
 docopt>=0.6.2,<0.7
-gitpython

MicroPython/scripts/generate_terraform.py

@@ -1,9 +1,13 @@
 import os
+import pathlib
 import json
 import subprocess
+import shutil
+import sys
 
 TERRAFORM_DIR = "../terraform"
-ROOT_DIR = os.path.abspath(os.path.join(os.path.dirname(os.path.abspath(__file__)), '..'))
+ROOT_DIR = os.path.abspath(os.path.join(
+    os.path.dirname(os.path.abspath(__file__)), '..'))
 TERRAFORM_BINARY_NAME = "terraform"
 TERRAFORM_LOCAL_EXE = "./terraform"
 
@@ -29,10 +33,28 @@ def save_terraform_output_as_file(_output_file_name: str):
         outfile.write(tf_output_str)
 
 
+def _check_terraform():
+    """Checks if terraform binary exists in the system"""
+
+    terraform_path = shutil.which(TERRAFORM_BINARY_NAME)
+    if terraform_path is not None:
+        # Terraform installed on the system
+        return TERRAFORM_BINARY_NAME
+    elif pathlib.Path(TERRAFORM_LOCAL_EXE).exists():
+        # Terraform binary exists inside terraform dir
+        return TERRAFORM_LOCAL_EXE
+    else:
+        print("No terraform binary file found! Aborting.")
+        sys.exit(1)
+
+
 def create_terraform_output() -> str:
-    subprocess.run([TERRAFORM_LOCAL_EXE, "init"])
-    subprocess.run([TERRAFORM_LOCAL_EXE, "workspace", "select", "production"])
-    proc = subprocess.Popen([TERRAFORM_LOCAL_EXE, 'output', '-json'], stdout=subprocess.PIPE)
+    terraform = _check_terraform()
+
+    subprocess.run([terraform, "init"])
+    subprocess.run([terraform, "workspace", "select", "production"])
+    proc = subprocess.Popen(
+        [terraform, 'output', '-json'], stdout=subprocess.PIPE)
     output = proc.stdout.read()
     return get_string_from_byte(output)
 

MicroPython/scripts/pyboard.py

@@ -322,6 +322,9 @@ def enter_raw_repl(self):
             self.serial.read(n)
             n = self.serial.inWaiting()
 
+        # Need 1 sec to enter raw repl or error occurs
+        time.sleep(1)
+
         self.serial.write(b"\r\x01")  # ctrl-A: enter raw REPL
         data = self.read_until(1, b"raw REPL; CTRL-B to exit\r\n>")
         if not data.endswith(b"raw REPL; CTRL-B to exit\r\n>"):

MicroPython/scripts/requirements.txt

@@ -1,3 +1,3 @@
 wheel
-esptool
+esptool==2.8
 gitpython

MicroPython/scripts/upload_micropython.py

@@ -8,6 +8,7 @@
 MICROPYTHON_BIN_FILE_DIR = "MicroPython_firmware/"
 ROOT_DIR = os.path.abspath(os.path.join(os.path.dirname(os.path.abspath(__file__)), '..'))
 
+
 #read -n 1 -s -r -p "Reset ESP32 into bootloader mode - Hold BOOT button and click EN button. Release BOOT. Then press any key to continue"
 def erase_chip_advanced(port):
     print('Erasing Chip...')

MicroPython/scripts/upload_scripts.py

@@ -1,10 +1,9 @@
 import argparse
-import json
 import os
-import subprocess
 import sys
 import git
 from git.refs.tag import TagReference
+from pathlib import Path
 
 import pyboard
 

MicroPython/src/boot.py

@@ -1,6 +1,6 @@
-from gc import enable, collect, threshold, mem_free, mem_alloc
-from logging import basicConfig, getLogger, DEBUG, debug
 from uos import statvfs
+from logging import basicConfig, getLogger, DEBUG, debug
+from gc import enable, collect, threshold, mem_free, mem_alloc
 
 from common import utils