-
Notifications
You must be signed in to change notification settings - Fork 9
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit 0ff6e72
Showing
110 changed files
with
11,029 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# Sphinx build info version 1 | ||
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. | ||
config: 84e2c76a5f957d4cd237c1d518258990 | ||
tags: 645f666f9bcd5a90fca523b33c5a78b7 |
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,106 @@ | ||
# Cluster Pi Setup | ||
|
||
The Illuminator can be deployed to a cluster of Raspberry Pi's. It requires one Raspberry Pi acting as a **server** or 'master' and several **clients** Raspberry Pi's. | ||
Raspberry Pi's must be connected and configured as a local network, and the | ||
*server* must be configured to have permissions to access and control the *clients* through the Secure Shell Protocol (SSH). | ||
|
||
During simulation, the *server* engage with the *clients* to run the simulations defined in the [simulation configuration file](user/config-file.md), and | ||
information is exchanged between Rasberry Pi's using network sockets. | ||
The **server** provides a Dashboard to visualize the results, and saves them to a `.csv` files for later analysis. | ||
|
||
|
||
<div align="center"> | ||
<img align="center" src="_static/img/Structure.jpg" width="500"> | ||
</div> | ||
|
||
## Hardware Requirements | ||
- A Raspberry Pi to use as a *server*. | ||
- One or more Raspebrry Pi's to use as *clients*. | ||
- A networkw switch to connect all Rasberry Pi's as a local network. | ||
|
||
## Set up | ||
|
||
Conduct the following steps on each Raspberry Pi to deploy the illuminator in a cluster. These instructions require to install *Illuminator* from source. | ||
|
||
:::{warning} | ||
The steps were defined before the release of version `3.0.0` and therefore issues might arise when using the latest Illumiator version. | ||
::: | ||
|
||
1. [Install Raspberry pi OS using Raspberry Pi imager.](https://www.raspberrypi.com/software/) | ||
2. Set an static IP address for the Raspberry Pi. Use the following command on the terminal to open the `dhcpcd.conf` file: | ||
|
||
```shell | ||
sudo nano /etc/dhcpcd.conf | ||
``` | ||
|
||
In the `dhcpcd.conf` file, find the information to change the IP address to static as following: | ||
|
||
```shell | ||
interface etho | ||
static ip_address=192.168.0.1/24 # change the IP address as you want | ||
``` | ||
|
||
Give all users execute permission to all the documents in `runshfile/` in order to make sure the *server* can access the *client* model. | ||
|
||
```shell | ||
chmod -R a+X *dir* | ||
``` | ||
|
||
Finally, reboot the Raspberry Pi using `sudo reboot` on the terminal. | ||
3. [Configure SSH connections so that the *server* can connect to the *clients* without a password.](https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys-2) | ||
|
||
4. Install the Illuminator Python package from source, and the addional dependencies: | ||
|
||
```shell | ||
# or, if from source code | ||
pip install Illuminator/ | ||
``` | ||
|
||
```shell | ||
# aditional dependencies | ||
pip install tk python-csv python-math scipy wandb itertools | ||
``` | ||
5. Use the following command on the *server's* terminal to check the connections for each of the *clients:* | ||
|
||
```shell | ||
# notice that the followng assumes that each client has a | ||
# user named 'illuminator' | ||
ssh illuminator@<ip> #<ip> represent the client's IP address set in step 2 | ||
``` | ||
6. Run the `build_runshfile.py` file in the configuration directory on the *server*, this will generate a `run.sh` script. Pass the appropiate `config.yaml` file containing the configuration for the simulation scenario: | ||
|
||
```shell | ||
python3 build_runshfile.py <config.yaml> | ||
``` | ||
|
||
The `runs.sh` file contains a list of commands that will start the models required by a simulation defined in the `config.yaml`, such as: | ||
|
||
```shell | ||
# Example | ||
lxterminal -e ssh [email protected] './Desktop/illuminatorclient/configuration/runshfile/runBattery.sh 192.168.0.1 5123 /home/illuminator/Desktop/Final_illuminator'& | ||
lxterminal -e ssh [email protected] './Desktop/illuminatorclient/configuration/runshfile/runBattery.sh 192.168.0.2 5123 /home/illuminator/Desktop/Final_illuminator'& | ||
``` | ||
|
||
:::{important} | ||
**Explanation** | ||
|
||
`lxterminal` starts a terminal on a remote machine (a client). So `lxterminal -e ssh [email protected]` would use SSH to login to machine `192.168.0.1` with the user `illuminator` which has no password (this should be improved). | ||
|
||
Three values are passed to the `ssh` command (the part between single quoates): `'./Desktop/illuminatorclient/configuration/runshfile/runWind.sh 192.168.0.1 5123 /home/illuminator/Desktop/Final_illuminator'&`. | ||
This starts the script `./Desktop/illuminatorclient/configuration/runshfile/runWind.sh` on the remote machine with the following parameters: | ||
* IP address: `192.168.0.1` | ||
* Port: `5123` | ||
* Path of mosaik file: `/home/illuminator/Desktop/Final_illuminator` | ||
|
||
The `& `at the end starts the process in the background, so that the `run.sh` script does not wait for the command to finish but executes the next command immediately. | ||
|
||
For example, the `runWind.sh` looks like this: | ||
|
||
```shell | ||
#! /bin/bash | ||
cd $3/Wind | ||
python wind_mosaik.py $1:$2 –remote | ||
``` | ||
There you see the three parameters in action. | ||
::: | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
# Cluster Set Up | ||
|
||
We are currently working on automating some of the steps to deploy the **Illuminator** on a Raspberry Pi cluster. | ||
The main idea of this line of development is to simplify the number of steps required to install and enable simulations, after the networking of the cluster is completed. The following are some of the goals we will like to achieve: | ||
|
||
1. **Installation:** it shall be possible to install the 'Illuminator' on each *client* and *server* with a single command on the terminal. For example: `pip install illuminator`. This will remove the burden of copying the source code to to each client and server imposed by the current implementation. Achieving this will make goal (2) more feasible. | ||
2. **Model accessibility:** it shall be possible for the *server* to run models in any *client* by using the `lxterminal` command, with a command such as: | ||
`lxterminal -e ssh [email protected] 'illuminator cluster run <model-name> --remote'`. Where the `--remote` flag must tell *Mosaik* that the simulation will run in a distributed environment (cluster). In the current implementation, one has to specified the path to the Python file containing the model (usually a directoy on the client). However, that is harder to maintain because new versions of the Illuminator rely on the source code and not on Python wheels or TARs. Therefore, the focus here should be on making sure that once the Illuminator is installed to the Python path, then all models are accessible at the OS level and they can be run in *remote* model. | ||
3. **Business logic for cluster scenarios:** shall develop the business logic to use the information in (`connect`) in the [scenario configuration file](../user/config-file.md) to start simulators on the clienst; such that, for example, calling `illuminator cluster <scenario.yaml>` reads on which client a model should be started, and start the relevant simulators on that client and runs the simulation. | ||
|
||
:::{note} | ||
The ideas above can be use by contributors to participate in the development of *Illuminator*. Better ideas and solutions always welcome. Please contact the [Illuminator Development Team](mailto:[email protected]) if you want to contribute. | ||
::: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
# Dashboard | ||
|
||
We are looking for an light-weigth opensource source solution for implementing a dasboard for the *Illuminator*. The main purspose of the dashboard is to display the simulations result during runtime, therefore a solution must provide visualisations in real-time. We haven't explore the posibilites, but [Graphana](https://grafana.com/) can be a candidate. | ||
|
||
:::{note} | ||
The ideas above can be use by contributors to participate in the development of *Illuminator*. Better ideas and solutions always welcome. Please contact the [Illuminator Development Team](mailto:[email protected]) if you want to contribute. | ||
::: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,80 @@ | ||
# Docstrings | ||
|
||
The entirety of this project follows the [Numpy docstring style guide](https://numpydoc.readthedocs.io/en/latest/format.html), so for more information or questions please refer to the provided link. | ||
|
||
## Short summary | ||
|
||
The style guide states that all comments should start with triple quotation marks, seen below: | ||
|
||
```python | ||
def add(a, b): | ||
""" | ||
The sum of two numbers. | ||
""" | ||
return a + b | ||
``` | ||
|
||
The docstrings should also have a clearly separated sections for other parts of the code (if any). These may include, but is not limited to: Parameters, Attributes, Returns, Raises. | ||
|
||
Each separated section should start with the Title of the section, followed by a row of dashes such as in the following section: | ||
|
||
```python | ||
def multiply(a, b): | ||
""" | ||
Computes the multiplication of two numbers and returns its value. | ||
|
||
... | ||
Parameters | ||
---------- | ||
a : int | ||
The first integer of the multiplication formula | ||
b : int | ||
The second integer of the multiplication formula | ||
|
||
Returns | ||
------- | ||
int | ||
The two values multiplied. | ||
""" | ||
return a * b | ||
``` | ||
|
||
When appropriate, we should also ensure to include the name and/or type of variables for the any of the aforementioned sections. | ||
|
||
Lastly, type hints are also a useful addition to any code. They can be used to "hint" to other developers what is expected as input and/or output when a function is used. | ||
```python | ||
def sum(a:int, b:int) -> float: | ||
``` | ||
As seen from the example above we can immediately conclude that for this function to work it will need an integer `a` and `b`, with the return value being of type float | ||
|
||
Combined, the docstrings may look something like this: | ||
|
||
```python | ||
def sum(a:int, b:int) -> float: | ||
""" | ||
Computes the sum of `a` and `b` and returns the outcome | ||
|
||
... | ||
|
||
Parameters | ||
---------- | ||
a : int | ||
The first integer of the sum formula | ||
b : int | ||
The second integer of the sum formula | ||
|
||
Returns | ||
------- | ||
result : float | ||
The sum of a + b | ||
""" | ||
result = a + b | ||
return result | ||
``` | ||
|
||
## Missing data in older docstrings | ||
There is still a lot of missing data for older docstrings, which has not been completed due to missing domain knowledge. | ||
In order to contribute to those, one should simply search for any file which contains `???` in its docstrings and change it to whatever is appropriate. | ||
In most cases, the description is missing since we could still acquire datatypes of attributes/parameters based on context clues or debugging/testing. However some bits of code are unused which means that it is also missing the type hints/docstring object types. | ||
|
||
There exists an excel sheet within the `docs` folder called `Illuminator Model Classification.xlsx` which contains a semi-filled list of variables and their descriptions which can be used to help finish the incomplete docstrings. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
# Set Up Environment | ||
Follow these steps to set up a development environment. | ||
|
||
**Requirements** | ||
* Python >= 3.11 | ||
* Recent version of PIP | ||
|
||
1. Clone the repository: | ||
|
||
```shell | ||
git clone [email protected]:Illuminator-team/Illuminator.git | ||
``` | ||
|
||
2. Go the root tof the repository: | ||
|
||
```shell | ||
cd Illuminator/ | ||
``` | ||
|
||
3. install the development dependencies in editable mode: | ||
|
||
```shell | ||
pip install -e . | ||
``` | ||
|
||
## Running Unit Tests | ||
|
||
We use `Pytest` to write and test the source code. To run all unit-tests, run the following command at the root of the repository: | ||
|
||
```shell | ||
pytest tests/ | ||
``` | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,119 @@ | ||
# Software Architecture | ||
|
||
The Illuminator is modular Python applications to simulate energy systems. In this seciton, we provide an overview of its sotware architecture. The diagram below describes the components of the Illuminator using the terminology of the [C4 model](https://c4model.com/). | ||
|
||
<div align="center"> | ||
<img align="center" src="../_static/img/components.png" width="800"> | ||
</div> | ||
|
||
|
||
At a highest level, the Illuminator consists of three internal applications: *Model Builder, Simulation Engie, and Dashboard*; which depend on an an external application for executing simualation, the *Mosaik Simulation Framework* | ||
Users of the Illuminator interact with the *Model Builder* and the *Simulation Engine* for developig model and define simulation scenarios. Illuminator's applications interact with the *Mosaik Simulation Framework* to run simulations and collect the results. | ||
|
||
## Users | ||
Users of the Illuminator take one of two roles: | ||
|
||
* **Model Developer**: uses the Illuminator to develope new energy models that can be used in a simulation. | ||
* **Energy Analyst**: use the Illuminator to define simulation scenarios and run simulations. | ||
|
||
## Components | ||
|
||
### Mosaik Simulation Framework | ||
|
||
A framework that serves as a core platform for executing energy system simulations. [Mosaik](https://mosaik.readthedocs.io/en/latest/index.html) is an external dependency, and as such the Illuminator interacts with it through its API. | ||
|
||
### Model Builder Application | ||
|
||
A Python application that model developers use to create/modify energy models for the Illuminator. New models are developed using the **Builder** componente, which provides a custom interface for creating and registering energy models to the **Model Library**. The purpose of the **Builder** component is to ease the development of energy models using a jargon that **energy system engineers** are more familiar with. For example, using term such as *inputs, outputs, states, etc.* to define new models. | ||
|
||
The **Model Library** component stores energy models that can be use in a simulation, so that they can be accessed by the **Mosaik Simulation Framework** during runtime. | ||
Models in the **Model Library** are containers of metadata and business logic. | ||
|
||
No computations related to simulations are performed by the *model builder application*. | ||
|
||
### Simulation Engine Application | ||
|
||
A Python application to run simulations via the Mosaik API. This application consists of four components. The **Scenario API** provides a wrapper to prepare and start simulations in the **Mosaik Simulation Framework**. Simulations, computations and the management of output data are delegated to the **Mosaik Simulation Framework**. | ||
The Senario API uses the **Scenario Schema** to validate simulation scenarios writen as YAML files by the **Energy Analysis**. The **Scenario Schema** defines the format that YAML files must be written on. | ||
|
||
The **Illuminator CLI** is an appliccation implemented using [Typer](https://typer.tiangolo.com/), which provides a command line interface to run simulation locally, and parcially automates the deployment of the Illuminator in a Raspberry Pi cluster. The **Illuminator CLI** uses the **Scernario API** and the **Cluster PI** components to provide functionality. | ||
|
||
Finally, the **Cluster Pi** component consists of a set of tools for setting up the Illuminator to tha Raspberry Pi Cluster, where simulation scenarios will be run. | ||
|
||
### Dashboard | ||
|
||
An application used by the **Energy Analyst** to visualise results and logs of simulations in real-time. This is has not been implemented in the current version. | ||
|
||
----- | ||
## Use Cases | ||
|
||
There are three comon use cases for the users of the Illuminator: | ||
|
||
1. Extending the model library: a *Model developer* wants to add a new model to the **Model Library** | ||
2. Creating a simullation scenario: an *Energy Analyst* wants to define a simulation scenario using a YAML file and execute the simulation. | ||
3. Set up a raspberry Pi cluster: a **user** wants to set up the Illuminator in a cluster of Raspberry Pi's to run simulations. | ||
|
||
### 1. Extending the Model Library | ||
|
||
Energy models should be added to the **Model Library** as follows: | ||
|
||
1. Create a Python module with the name of the model. For example, `example_model.py` | ||
1. In the file, create an IlluminatorModel object for the model. This defines which inputs, output, parameters, states, triggers, etc. a particular model has. For example: | ||
|
||
```Python | ||
from illuminator.builder import IlluminatorModel | ||
# Defines a model'a paramters, inputs, outputs... | ||
example_model = IlluminatorModel( | ||
parameters={"param1": "addition"}, | ||
inputs={"in1": 10, "in2": 20}, | ||
outputs={"out1": 0}, | ||
states={"out1": 0}, | ||
time_step_size=1, | ||
time=None | ||
) | ||
``` | ||
|
||
2. Create a class that inherits from `ModelConstructor`, and impement the `step()` method. The new class will become a *model type* in the Illuminator. Instances of this model type will be created by the **Scenario API** | ||
|
||
For example, | ||
|
||
```python | ||
from illuminator builder import ModelConstructor | ||
|
||
class ExampleModel (ModelConstructor): | ||
|
||
def step(): | ||
"""Computes this in every time step""" | ||
|
||
# The computation logic goes here: | ||
|
||
# return the time for the next time step | ||
return time + self._model.time_step_size | ||
``` | ||
|
||
3. Update the `illuminator/models/__init__.py` to import the new model type. For example: | ||
|
||
```python | ||
from .example_model import ExampleModel | ||
|
||
__all__ = ['BatteryModel', | ||
'Collector', | ||
'ExampleModel' # add new model | ||
``` | ||
|
||
3. To test the new model has been added correctly, try to import into a Python module: | ||
|
||
```python | ||
# Python file | ||
frmo illuminator.models import ExampleModel # test model import | ||
|
||
# run the file to check if importing is successful | ||
``` | ||
|
||
### 2. Creating Simulation Scenarios | ||
|
||
Refer to [simulation configuration file](../user/config-file.md). | ||
|
||
### 3. Setting Up Cluster Pi | ||
|
||
Refer to [Cluster Pi setup](../cluster-setup.md). |
Oops, something went wrong.