-
Notifications
You must be signed in to change notification settings - Fork 9
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit 64ec0eb
Showing
111 changed files
with
13,697 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# Sphinx build info version 1 | ||
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. | ||
config: 84e2c76a5f957d4cd237c1d518258990 | ||
tags: 645f666f9bcd5a90fca523b33c5a78b7 |
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Large diffs are not rendered by default.
Oops, something went wrong.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Empty file.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,113 @@ | ||
# Cluster Pi Setup | ||
|
||
The Illuminator can be deployed to a cluster of Raspberry Pi's. It requires one Raspberry Pi acting as a **server** or 'master' and several **clients** Raspberry Pi's. | ||
Raspberry Pi's must be connected and configured as a local network, and the | ||
*server* must be configured to have permissions to access and control the *clients* through the Secure Shell Protocol (SSH). | ||
|
||
During simulation, the *server* engage with the *clients* to run the simulations defined in the [simulation configuration file](user/config-file.md), and | ||
information is exchanged between Rasberry Pi's using network sockets. | ||
The **server** provides a Dashboard to visualize the results, and saves them to a `.csv` files for later analysis. | ||
|
||
<div align="center"> | ||
<img align="center" src="_static/img/Structure.jpg" width="500"> | ||
</div> | ||
|
||
|
||
## Hardware Requirements | ||
- A Raspberry Pi to use as a *server*. | ||
- One or more Raspebrry Pi's to use as *clients*. | ||
- A networkw switch to connect all Rasberry Pi's as a local network. | ||
|
||
## Set up | ||
|
||
Conduct the following steps on each Raspberry Pi to deploy the illuminator in the cluster. | ||
|
||
1. [Install Raspberry pi OS using Raspberry Pi imager.](https://www.raspberrypi.com/software/) | ||
2. Set an static IP address for the Raspberry Pi. Use the following command on the terminal to open the `dhcpcd.conf` file: | ||
|
||
```shell | ||
sudo nano /etc/dhcpcd.conf | ||
``` | ||
|
||
In the `dhcpcd.conf` file, find the information to change the IP address to static as following: | ||
|
||
```shell | ||
interface etho | ||
static ip_address=192.168.0.1/24 # change the IP address as you want | ||
``` | ||
|
||
Give all users execute permission to all the documents in `runshfile/` in order to make sure the *server* can access the *client* model. | ||
|
||
```shell | ||
chmod -R a+X *dir* | ||
``` | ||
|
||
Finally, reboot the Raspberry Pi using `sudo reboot` on the terminal. | ||
3. [Configure SSH connections so that the *server* can connect to the *clients* without a password.](https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys-2) | ||
|
||
4. Install the Illuminator Python package, and the addional dependencies: | ||
|
||
```shell | ||
# if connected to the internet | ||
pip install illuminator | ||
|
||
# or, if from source code | ||
pip install Illuminator/ | ||
``` | ||
|
||
```shell | ||
# aditional dependencies | ||
pip install tk python-csv python-math scipy wandb itertools | ||
``` | ||
5. Use the following command on the *server's* terminal to check the connections for each of the *clients:* | ||
|
||
```shell | ||
# notice that the followng assumes that each client has a | ||
# user named 'illuminator' | ||
ssh illuminator@ip #ip represent your follower IP address set in step 2 | ||
``` | ||
6. Run the `build_runshfile.py` file in the configuration directory on the *server*, this will generate a `run.sh` script. Give the appropiate `config.yaml` file containing the simulation scenario definition: | ||
|
||
```shell | ||
python3 build_runshfile.py <config.yaml> | ||
``` | ||
|
||
The `runs.sh` file contains a list of commands that will start the models required by a simulation defined in the `config.yaml`, such as: | ||
|
||
```shell | ||
# add example | ||
``` | ||
|
||
TODO: integrate this: | ||
|
||
The lx terminal command does the following: | ||
|
||
Lxterminal itself starts a terminal on a remote machine. | ||
|
||
So ‘lxterminal -e ssh [email protected]’ would use SSH to login to machine 192.168.0.1 with the user illuminator which has no password (room for improvement 😉 ). | ||
|
||
The second part: | ||
|
||
'./Desktop/illuminatorclient/configuration/runshfile/runWind.sh 192.168.0.1 5123 /home/illuminator/Desktop/Final_illuminator'& | ||
|
||
Is starting the script ./Desktop/illuminatorclient/configuration/runshfile/runWind.sh on the remote machine with the following parameters: | ||
|
||
IP address 192.168.0.1 | ||
Port 5123 | ||
Path of mosaik /home/illuminator/Desktop/Final_illuminator | ||
|
||
The & at the end starts the process in the background, so that the run.sh script does not wait for the command to finish but executes the next command immediately. | ||
|
||
For example runWind.sh looks like this: | ||
|
||
#! /bin/bash | ||
cd $3/Wind | ||
python wind_mosaik.py $1:$2 –remote | ||
|
||
There you see the three parameters in action. | ||
|
||
|
||
## Contact and Support | ||
|
||
For more comprehensive support, please contact us at [[email protected]](mailto:[email protected]). Additionally, you can reach out to the main contributors for specific inquiries: | ||
* [Dr.ir. Milos Cvetkovic](mailto:[email protected]) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,80 @@ | ||
# Docstrings | ||
|
||
The entirety of this project follows the [Numpy docstring style guide](https://numpydoc.readthedocs.io/en/latest/format.html), so for more information or questions please refer to the provided link. | ||
|
||
## Short summary | ||
|
||
The style guide states that all comments should start with triple quotation marks, seen below: | ||
|
||
```python | ||
def add(a, b): | ||
""" | ||
The sum of two numbers. | ||
""" | ||
return a + b | ||
``` | ||
|
||
The docstrings should also have a clearly separated sections for other parts of the code (if any). These may include, but is not limited to: Parameters, Attributes, Returns, Raises. | ||
|
||
Each separated section should start with the Title of the section, followed by a row of dashes such as in the following section: | ||
|
||
```python | ||
def multiply(a, b): | ||
""" | ||
Computes the multiplication of two numbers and returns its value. | ||
|
||
... | ||
Parameters | ||
---------- | ||
a : int | ||
The first integer of the multiplication formula | ||
b : int | ||
The second integer of the multiplication formula | ||
|
||
Returns | ||
------- | ||
int | ||
The two values multiplied. | ||
""" | ||
return a * b | ||
``` | ||
|
||
When appropriate, we should also ensure to include the name and/or type of variables for the any of the aforementioned sections. | ||
|
||
Lastly, type hints are also a useful addition to any code. They can be used to "hint" to other developers what is expected as input and/or output when a function is used. | ||
```python | ||
def sum(a:int, b:int) -> float: | ||
``` | ||
As seen from the example above we can immediately conclude that for this function to work it will need an integer `a` and `b`, with the return value being of type float | ||
|
||
Combined, the docstrings may look something like this: | ||
|
||
```python | ||
def sum(a:int, b:int) -> float: | ||
""" | ||
Computes the sum of `a` and `b` and returns the outcome | ||
|
||
... | ||
|
||
Parameters | ||
---------- | ||
a : int | ||
The first integer of the sum formula | ||
b : int | ||
The second integer of the sum formula | ||
|
||
Returns | ||
------- | ||
result : float | ||
The sum of a + b | ||
""" | ||
result = a + b | ||
return result | ||
``` | ||
|
||
## Missing data in older docstrings | ||
There is still a lot of missing data for older docstrings, which has not been completed due to missing domain knowledge. | ||
In order to contribute to those, one should simply search for any file which contains `???` in its docstrings and change it to whatever is appropriate. | ||
In most cases, the description is missing since we could still acquire datatypes of attributes/parameters based on context clues or debugging/testing. However some bits of code are unused which means that it is also missing the type hints/docstring object types. | ||
|
||
There exists an excel sheet within the `docs` folder called `Illuminator Model Classification.xlsx` which contains a semi-filled list of variables and their descriptions which can be used to help finish the incomplete docstrings. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
# Set Up Environment | ||
Follow these steps to set up a development environment. | ||
|
||
**Requirements** | ||
* Python >= 3.11 | ||
* Recent version of PIP | ||
|
||
1. Clone the repository: | ||
|
||
```shell | ||
git clone [email protected]:Illuminator-team/Illuminator.git | ||
``` | ||
|
||
2. Go the root tof the repository: | ||
|
||
```shell | ||
cd Illuminator/ | ||
``` | ||
|
||
3. install the development dependencies in editable mode: | ||
|
||
```shell | ||
pip install -e . | ||
``` | ||
|
||
## Running Unit Tests | ||
|
||
We use `Pytest` to write and test the source code. To run all unit-tests, run the following command at the root of the repository: | ||
|
||
```shell | ||
pytest tests/ | ||
``` | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
# Testing: basic principles and ideas | ||
In order to properly to contribute to the Illuminator project, one must also create, update or delete tests based on what it is they are trying to achieve. For each new contribution, new tests will be necessary. Updating old behaviour also might require updating older unit tests. For this project we have decided to use `pytest` as our testing library. | ||
|
||
## Types of tests | ||
In total there are three types of tests that are considered industry standards: Unit tests, Integration Tests, End-to-End (E2E) tests. Each type of test serves a different purpose and has a different priority. A short overview of these tests can be seen in the image below: | ||
|
||
<div align="center"> | ||
<img align="center" src="../_static/img/testing_pyramid.jpg" width="300"> | ||
</div> | ||
|
||
|
||
### Unit tests | ||
These types of tests are the most numerous and are simplest to create. Their run time is very fast, they will ideally have no dependencies on outside factors and they can test the smallest bits of code. In an ideal world, these tests should exist for every new method/function that is created. These tests will test the expected behaviour of a method, which not only includes the "positive" behaviour, but also the negative ones as well (i.e. what happens if the method does not get what it needs?) | ||
|
||
> Why do we create unit tests? | ||
|
||
As mentioned in the summary of unit tests, we create them to test functions and methods which we have written. Although it might excessive to test every line of every method, Unit tests ultimately serve as a sanity check for the developers of old and new code. | ||
|
||
#### An example: | ||
We are creating a new model of some type of `Battery` monitoring device. This model expects the `Battery` model to give information about its State-Of-Charge whenever the battery is discharged using the **discharge_battery()** method. If we have written a unit test for the method **discharge_battery()** where we explicitly state that we expect it to return the State-Of-Charge, then we can guarantee our new model will be able to get that information without having to read the code written in the `Battery` model. | ||
|
||
### Integration tests | ||
|
||
Unlike Unit tests, integration tests are intended to test multiple things at once while heavily limiting our "mocking". If we think of "unit tests" as testing individual "units", then the integration test involves multiple units at once. Because of this, we can see that integration tests tend to cover large chunks of code, hence why there is less integration tests compared to unit tests. In order to truly understand integration tests we must first explain the concept of "mocking". | ||
|
||
> What is mocking? | ||
|
||
Mocking, put simply, is used to mimic what an outside function or variable is supposed to be, without actually calling it. This is what allows unit tests to be independent of other methods and classes, and what separates it from integration and E2E tests. | ||
|
||
Using the previous `Battery` **discharge_battery()** method as an example, if we want to create a test for our new `monitoring device` model which uses that method, we must specify in our test that instead of actually calling the **dischare_battery()** method, we will fake (mock) the call by skipping over it and instead return values we set. | ||
|
||
> Why do we create integration tests? | ||
|
||
Integration tests can be used for multiple reasons. Perhaps a method is far too complex to write as a unit test. Maybe there are specific interactions we wish to see in a "normal" environment instead of a simulated one. In some cases we can even test multiple classes at the same time as bigger integration tests. In the example above with the `monitoring device` and `Battery` model, we would not mock anything and just let the two classes interact regularly. | ||
|
||
As mentioned before, integration tests should mock very little. They should be written in a way that is close to how the real code would work, hence why mocking is avoided if possible. An example where one might want to use mocking within an integration test is with data. If we wish to test a class which needs data from a large dataset, we can instead tell our integration test to mimic that data by creating some fakes (think of 2 or 3 lines of a CSV file) | ||
|
||
### End-to-End tests | ||
Unlike Unit and Integration tests, these should never mock data. These tests should run the code exactly at it is, with realistic inputs and outputs (which we must check) and ideally use as much of the codebase as it can. These tests should be very few in numbers because they are complex to write, and tend to have a longer runtime. These are essentially automated versions of "manual" testing. | ||
|
||
|
||
## How to write tests and future expectations | ||
The explanation on writing tests can be found on [this page](writing-tests.md). | ||
|
||
At the moment all the test are located within the `tests/Models/` folder. They are not separated into smaller sub-folders due to pytest's [good practice](https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html) recommendations. In the future, this will will make more sense as we expect the number of tests needed to drastically drop due to repetition between models. | ||
|
||
|
Oops, something went wrong.