Skip to content

Commit

Permalink
deploy: f17e57e
Browse files Browse the repository at this point in the history
  • Loading branch information
manuGil committed Nov 28, 2024
0 parents commit 64ec0eb
Show file tree
Hide file tree
Showing 111 changed files with 13,697 additions and 0 deletions.
4 changes: 4 additions & 0 deletions .buildinfo
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: 84e2c76a5f957d4cd237c1d518258990
tags: 645f666f9bcd5a90fca523b33c5a78b7
Binary file added .doctrees/cluster-setup.doctree
Binary file not shown.
Binary file not shown.
Binary file added .doctrees/developer/set-up.doctree
Binary file not shown.
Binary file added .doctrees/developer/testing-explained.doctree
Binary file not shown.
Binary file added .doctrees/developer/writing-tests.doctree
Binary file not shown.
Binary file added .doctrees/environment.pickle
Binary file not shown.
Binary file added .doctrees/index.doctree
Binary file not shown.
883 changes: 883 additions & 0 deletions .doctrees/nbsphinx/tutorials/example.ipynb

Large diffs are not rendered by default.

Binary file added .doctrees/nbsphinx/tutorials_example_13_0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .doctrees/nbsphinx/tutorials_example_19_0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .doctrees/nbsphinx/tutorials_example_7_0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .doctrees/quick-start.doctree
Binary file not shown.
Binary file added .doctrees/references/engine-api.doctree
Binary file not shown.
Binary file added .doctrees/references/models.doctree
Binary file not shown.
Binary file added .doctrees/tutorials/example.doctree
Binary file not shown.
Binary file added .doctrees/user/config-file.doctree
Binary file not shown.
Binary file added .doctrees/user/models.doctree
Binary file not shown.
Binary file added .doctrees/user/simulations.doctree
Binary file not shown.
Empty file added .nojekyll
Empty file.
Binary file added _images/tutorials_example_13_0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added _images/tutorials_example_19_0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added _images/tutorials_example_7_0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
113 changes: 113 additions & 0 deletions _sources/cluster-setup.md.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Cluster Pi Setup

The Illuminator can be deployed to a cluster of Raspberry Pi's. It requires one Raspberry Pi acting as a **server** or 'master' and several **clients** Raspberry Pi's.
Raspberry Pi's must be connected and configured as a local network, and the
*server* must be configured to have permissions to access and control the *clients* through the Secure Shell Protocol (SSH).

During simulation, the *server* engage with the *clients* to run the simulations defined in the [simulation configuration file](user/config-file.md), and
information is exchanged between Rasberry Pi's using network sockets.
The **server** provides a Dashboard to visualize the results, and saves them to a `.csv` files for later analysis.

<div align="center">
<img align="center" src="_static/img/Structure.jpg" width="500">
</div>


## Hardware Requirements
- A Raspberry Pi to use as a *server*.
- One or more Raspebrry Pi's to use as *clients*.
- A networkw switch to connect all Rasberry Pi's as a local network.

## Set up

Conduct the following steps on each Raspberry Pi to deploy the illuminator in the cluster.

1. [Install Raspberry pi OS using Raspberry Pi imager.](https://www.raspberrypi.com/software/)
2. Set an static IP address for the Raspberry Pi. Use the following command on the terminal to open the `dhcpcd.conf` file:

```shell
sudo nano /etc/dhcpcd.conf
```

In the `dhcpcd.conf` file, find the information to change the IP address to static as following:

```shell
interface etho
static ip_address=192.168.0.1/24 # change the IP address as you want
```

Give all users execute permission to all the documents in `runshfile/` in order to make sure the *server* can access the *client* model.

```shell
chmod -R a+X *dir*
```

Finally, reboot the Raspberry Pi using `sudo reboot` on the terminal.
3. [Configure SSH connections so that the *server* can connect to the *clients* without a password.](https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys-2)

4. Install the Illuminator Python package, and the addional dependencies:

```shell
# if connected to the internet
pip install illuminator

# or, if from source code
pip install Illuminator/
```

```shell
# aditional dependencies
pip install tk python-csv python-math scipy wandb itertools
```
5. Use the following command on the *server's* terminal to check the connections for each of the *clients:*

```shell
# notice that the followng assumes that each client has a
# user named 'illuminator'
ssh illuminator@ip #ip represent your follower IP address set in step 2
```
6. Run the `build_runshfile.py` file in the configuration directory on the *server*, this will generate a `run.sh` script. Give the appropiate `config.yaml` file containing the simulation scenario definition:

```shell
python3 build_runshfile.py <config.yaml>
```

The `runs.sh` file contains a list of commands that will start the models required by a simulation defined in the `config.yaml`, such as:

```shell
# add example
```

TODO: integrate this:

The lx terminal command does the following:

Lxterminal itself starts a terminal on a remote machine.

So ‘lxterminal -e ssh [email protected]’ would use SSH to login to machine 192.168.0.1 with the user illuminator which has no password (room for improvement 😉 ).

The second part:

'./Desktop/illuminatorclient/configuration/runshfile/runWind.sh 192.168.0.1 5123 /home/illuminator/Desktop/Final_illuminator'&

Is starting the script ./Desktop/illuminatorclient/configuration/runshfile/runWind.sh on the remote machine with the following parameters:

IP address 192.168.0.1
Port 5123
Path of mosaik /home/illuminator/Desktop/Final_illuminator

The & at the end starts the process in the background, so that the run.sh script does not wait for the command to finish but executes the next command immediately.

For example runWind.sh looks like this:

#! /bin/bash
cd $3/Wind
python wind_mosaik.py $1:$2 –remote

There you see the three parameters in action.


## Contact and Support

For more comprehensive support, please contact us at [[email protected]](mailto:[email protected]). Additionally, you can reach out to the main contributors for specific inquiries:
* [Dr.ir. Milos Cvetkovic](mailto:[email protected])
80 changes: 80 additions & 0 deletions _sources/developer/developer-docstrings.md.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Docstrings

The entirety of this project follows the [Numpy docstring style guide](https://numpydoc.readthedocs.io/en/latest/format.html), so for more information or questions please refer to the provided link.

## Short summary

The style guide states that all comments should start with triple quotation marks, seen below:

```python
def add(a, b):
"""
The sum of two numbers.
"""
return a + b
```

The docstrings should also have a clearly separated sections for other parts of the code (if any). These may include, but is not limited to: Parameters, Attributes, Returns, Raises.

Each separated section should start with the Title of the section, followed by a row of dashes such as in the following section:

```python
def multiply(a, b):
"""
Computes the multiplication of two numbers and returns its value.

...
Parameters
----------
a : int
The first integer of the multiplication formula
b : int
The second integer of the multiplication formula

Returns
-------
int
The two values multiplied.
"""
return a * b
```

When appropriate, we should also ensure to include the name and/or type of variables for the any of the aforementioned sections.

Lastly, type hints are also a useful addition to any code. They can be used to "hint" to other developers what is expected as input and/or output when a function is used.
```python
def sum(a:int, b:int) -> float:
```
As seen from the example above we can immediately conclude that for this function to work it will need an integer `a` and `b`, with the return value being of type float

Combined, the docstrings may look something like this:

```python
def sum(a:int, b:int) -> float:
"""
Computes the sum of `a` and `b` and returns the outcome

...

Parameters
----------
a : int
The first integer of the sum formula
b : int
The second integer of the sum formula

Returns
-------
result : float
The sum of a + b
"""
result = a + b
return result
```

## Missing data in older docstrings
There is still a lot of missing data for older docstrings, which has not been completed due to missing domain knowledge.
In order to contribute to those, one should simply search for any file which contains `???` in its docstrings and change it to whatever is appropriate.
In most cases, the description is missing since we could still acquire datatypes of attributes/parameters based on context clues or debugging/testing. However some bits of code are unused which means that it is also missing the type hints/docstring object types.

There exists an excel sheet within the `docs` folder called `Illuminator Model Classification.xlsx` which contains a semi-filled list of variables and their descriptions which can be used to help finish the incomplete docstrings.
33 changes: 33 additions & 0 deletions _sources/developer/set-up.md.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Set Up Environment
Follow these steps to set up a development environment.

**Requirements**
* Python >= 3.11
* Recent version of PIP

1. Clone the repository:

```shell
git clone [email protected]:Illuminator-team/Illuminator.git
```

2. Go the root tof the repository:

```shell
cd Illuminator/
```

3. install the development dependencies in editable mode:

```shell
pip install -e .
```

## Running Unit Tests

We use `Pytest` to write and test the source code. To run all unit-tests, run the following command at the root of the repository:

```shell
pytest tests/
```

47 changes: 47 additions & 0 deletions _sources/developer/testing-explained.md.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Testing: basic principles and ideas
In order to properly to contribute to the Illuminator project, one must also create, update or delete tests based on what it is they are trying to achieve. For each new contribution, new tests will be necessary. Updating old behaviour also might require updating older unit tests. For this project we have decided to use `pytest` as our testing library.

## Types of tests
In total there are three types of tests that are considered industry standards: Unit tests, Integration Tests, End-to-End (E2E) tests. Each type of test serves a different purpose and has a different priority. A short overview of these tests can be seen in the image below:

<div align="center">
<img align="center" src="../_static/img/testing_pyramid.jpg" width="300">
</div>


### Unit tests
These types of tests are the most numerous and are simplest to create. Their run time is very fast, they will ideally have no dependencies on outside factors and they can test the smallest bits of code. In an ideal world, these tests should exist for every new method/function that is created. These tests will test the expected behaviour of a method, which not only includes the "positive" behaviour, but also the negative ones as well (i.e. what happens if the method does not get what it needs?)

> Why do we create unit tests?

As mentioned in the summary of unit tests, we create them to test functions and methods which we have written. Although it might excessive to test every line of every method, Unit tests ultimately serve as a sanity check for the developers of old and new code.

#### An example:
We are creating a new model of some type of `Battery` monitoring device. This model expects the `Battery` model to give information about its State-Of-Charge whenever the battery is discharged using the **discharge_battery()** method. If we have written a unit test for the method **discharge_battery()** where we explicitly state that we expect it to return the State-Of-Charge, then we can guarantee our new model will be able to get that information without having to read the code written in the `Battery` model.

### Integration tests

Unlike Unit tests, integration tests are intended to test multiple things at once while heavily limiting our "mocking". If we think of "unit tests" as testing individual "units", then the integration test involves multiple units at once. Because of this, we can see that integration tests tend to cover large chunks of code, hence why there is less integration tests compared to unit tests. In order to truly understand integration tests we must first explain the concept of "mocking".

> What is mocking?

Mocking, put simply, is used to mimic what an outside function or variable is supposed to be, without actually calling it. This is what allows unit tests to be independent of other methods and classes, and what separates it from integration and E2E tests.

Using the previous `Battery` **discharge_battery()** method as an example, if we want to create a test for our new `monitoring device` model which uses that method, we must specify in our test that instead of actually calling the **dischare_battery()** method, we will fake (mock) the call by skipping over it and instead return values we set.

> Why do we create integration tests?

Integration tests can be used for multiple reasons. Perhaps a method is far too complex to write as a unit test. Maybe there are specific interactions we wish to see in a "normal" environment instead of a simulated one. In some cases we can even test multiple classes at the same time as bigger integration tests. In the example above with the `monitoring device` and `Battery` model, we would not mock anything and just let the two classes interact regularly.

As mentioned before, integration tests should mock very little. They should be written in a way that is close to how the real code would work, hence why mocking is avoided if possible. An example where one might want to use mocking within an integration test is with data. If we wish to test a class which needs data from a large dataset, we can instead tell our integration test to mimic that data by creating some fakes (think of 2 or 3 lines of a CSV file)

### End-to-End tests
Unlike Unit and Integration tests, these should never mock data. These tests should run the code exactly at it is, with realistic inputs and outputs (which we must check) and ideally use as much of the codebase as it can. These tests should be very few in numbers because they are complex to write, and tend to have a longer runtime. These are essentially automated versions of "manual" testing.


## How to write tests and future expectations
The explanation on writing tests can be found on [this page](writing-tests.md).

At the moment all the test are located within the `tests/Models/` folder. They are not separated into smaller sub-folders due to pytest's [good practice](https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html) recommendations. In the future, this will will make more sense as we expect the number of tests needed to drastically drop due to repetition between models.


Loading

0 comments on commit 64ec0eb

Please sign in to comment.