Skip to content

Commit

Permalink
Merge pull request #9 from tobirohrer/feature/variable-energy-price
Browse files Browse the repository at this point in the history 
Time varying price profile for building simulation
  • Loading branch information
@tobirohrer
tobirohrer authored Oct 8, 2023
2 parents fac360c + 0c3c53c commit d03af17
Show file tree
Hide file tree
Showing 14 changed files with 17,728 additions and 816 deletions.
75 changes: 39 additions & 36 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,13 @@

<img src="docs/imgs/overview.drawio.png" alt="isolated" width="600"/>

The Building Energy Storage Simulation serves as open source OpenAI gym (now [gymnasium](https://github.com/Farama-Foundation/Gymnasium)) environment for Reinforcement Learning. The environment represents a building with an energy storage (in form of a battery) and a solar energy system. The aim is to control the energy storage so that the self consumption of the energy generated by the solar energy system is optimized.
The Building Energy Storage Simulation serves as OpenAI gym (now [gymnasium](https://github.com/Farama-Foundation/Gymnasium)) environment
for Reinforcement Learning. The environment represents a building with an energy storage (in form of a battery) and a
solar energy system. The building is connected to a power grid with time varying electricity prices. The task is to
control the energy storage so that the total cost of electricity are minimized.

The inspiration of this project and the data profiles come from the [CityLearn](https://github.com/intelligent-environments-lab/CityLearn) environment. Anyhow, this project focuses on the ease of usage and the simplicity of its implementation. Therefore, this project serves as playground for those who want to get started with reinforcement learning for energy management system control.

## Documentation

The documentation is available at [https://building-energy-storage-simulation.readthedocs.io/](https://building-energy-storage-simulation.readthedocs.io/)

## Installation

By using pip just:
Expand All @@ -27,13 +26,6 @@ git clone https://github.com/tobirohrer/building-energy-storage-simulation.git &
pip install -e .[docs,tests]
```

## Contribute & Contact

As I just started with this project, I am very happy for any kind of
contribution! In case you want to contribute, or if you have any
questions, contact me via
[discord](https://discord.com/users/tobirohrer#8654).

## Usage

```python
Expand All @@ -56,19 +48,26 @@ The simulation contains a building with an energy load profile attached to it. T
- primarily using electricity generated by the solar energy system,
- and secondary by using the remaining required electricity "from the grid"

The simulated building contains a battery which can be used to store energy. By **charging** energy, you can (temporarily)
increase the energy demand of the building, and by **discharging** you can (temporarily) decrease the energy demand of
the building. **The task is to find strategies of when to charge and when to discharge the battery.** Hereby,
the goal is to utilize the battery so energy usage from the solar energy system is maximized and usage of the energy
grid is minimized. Note, that excess energy from the solar energy system which is not used by the electricity load or
used to charge the battery is considered lost. So better use the solar energy to charge the battery in this case ;-)
When energy is taken from the grid, costs are incurred that can vary depending on the time (if a price profile is passed
as `electricity_price` to `BuildingSimulation`). The simulated building contains a battery which be controlled by
**charging** and **discharging** energy. The goal is to find control strategies optimize the use of the energy storage
by e.g. charging whenever electricity prices are high or whenever there is a surplus of solar generation. It is important
to note that no energy can be fed into the grid. This means any surplus of solar energy which is not used to charge the
battery is considered lost.

### Reward

$$ r_t = -1 * electricity\_consumed_t * electricity\_price_t$$

Note, that the term `electricity_consumed` cannot be negative. This means, excess energy from the solar
energy system which is not consumed by the electricity load or by charging the battery is considered lost
(`electricity_consumed` is 0 in this case).

### Action Space

| Action | Min | Max |
| ----------- | ----------- | ----------- |
| Charge | -1 | 1 |
| Action | Min | Max |
|----------|----------|--------|
| Charge | -1 | 1 |

The actions lie in the interval of [-1;1]. The action represents a fraction of the maximum energy which can be retrieved from the battery (or used to charge the battery) per time step.

Expand All @@ -78,30 +77,34 @@ The actions lie in the interval of [-1;1]. The action represents a fraction of t

### Observation Space

| Index | Observation | Min | Max |
| ----------- | ----------- | ----------- | ----------- |
| 0 | State of Charge (in kWh)| 0| Max Battery Capacity |
| [1; n]| Forecast Electric Load (in kWh) | 0 | Max Load in Profile |
| [n+1; 2*n]| Forecast Solar Generation (in kWh) |0| Max Solar Generation in Profile |
| Index | Observation | Min | Max |
|-------------|---------------------------------------|---------------------------|------------------------------|
| 0 | State of Charge (in kWh) | 0 | `battery_capacity` |
| [1; n] | Forecast Electric Load (in kWh) | Min of Load Profile | Max of Load Profile |
| [n+1; 2*n] | Forecast Solar Generation (in kWh) | Min of Generation Profile | Max of Generation Profile |
| [2n+1; 3*n] | Electricity Price (in € cent per kWh) | Min of Price Profile | Max of Price Profile |


The length of the observation depends on the length of the forecast used. By default, the simulation uses a forecast length of 4.
This means 4 time steps of an electric load forecast and 4 time steps of a solar generation forecast are included in the observation.
The length of the observation depends on the length of the forecast ($n$) used. By default, the simulation uses a forecast length of 4.
This means 4 time steps of an electric load forecast, 4 time steps of a solar generation forecast and 4 time steps of the
electric price profile are included in the observation.
In addition to that, the information about the current state of charge of the battery is contained in the observation.

The length of the forecast can be defined by setting the parameter `num_forecasting_steps` of the `Environment()`.


### Reward
### Episode Ends

As our goal is to use as less energy as possible, the reward is defined by the energy consumed at every time step (times -1):
The episode ends if the `max_timesteps` of the `Environment()` are reached.

$$ r_t = -1 * electricity\_consumed_t $$
## Code Documentation

It is important to note, that the term `electricity_consumed` cannot be negative. This means, excess energy from the solar
energy system which is not consumed by the electricity load or by charging the battery is considered lost
(`electricity_consumed` is 0 in this case).
The documentation is available at [https://building-energy-storage-simulation.readthedocs.io/](https://building-energy-storage-simulation.readthedocs.io/)

### Episode Ends
## Contribute & Contact

As I just started with this project, I am very happy for any kind of
contribution! In case you want to contribute, or if you have any
questions, contact me via
[discord](https://discord.com/users/tobirohrer#8654).

The episode ends if the `max_timesteps` of the `Environment()` are reached.
52 changes: 33 additions & 19 deletions building_energy_storage_simulation/building_simulation.py
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from typing import Tuple, Iterable
from typing import Tuple, Iterable, Union

import numpy as np

Expand All @@ -14,6 +14,8 @@ class BuildingSimulation:
:type electricity_load_profile: Iterable
:param solar_generation_profile: Generation profile in kWh per time step.
:type solar_generation_profile: Iterable
:param electricity_price: Electricity price in € cent per kWh.
:type electricity_price: Iterable or float
:param battery_capacity: The capacity of the battery in kWh.
:type battery_capacity: float
:param max_battery_charge_per_timestep: Maximum amount of energy (kWh) which can be obtained from the battery or
Expand All @@ -24,25 +26,32 @@ class BuildingSimulation:
def __init__(self,
electricity_load_profile: Iterable = load_profile('electricity_load_profile.csv', 'Load [kWh]'),
solar_generation_profile: Iterable = load_profile('solar_generation_profile.csv', 'Generation [kWh]'),
electricity_price: Union[Iterable, float] = load_profile('electricity_price_profile.csv', 'Day Ahead Auction'),
battery_capacity: float = 100,
max_battery_charge_per_timestep: float = 20
):

self.electricity_load_profile = np.array(electricity_load_profile)
self.solar_generation_profile = np.array(solar_generation_profile)

if isinstance(electricity_price, (float, int)):
self.electricity_price = np.full(len(self.solar_generation_profile), electricity_price)
else:
self.electricity_price = np.array(electricity_price)

self._check_data_profiles_same_length()

self.battery = Battery(capacity=battery_capacity,
max_battery_charge_per_timestep=max_battery_charge_per_timestep)

assert len(self.solar_generation_profile) == len(self.electricity_load_profile), \
"Solar generation profile and electricity load profile must be of the same length."

self.step_count = 0
self.start_index = 0
pass

def reset(self):
"""
1. Resetting the state of the building by calling `reset()` method from the building class.
1. Resetting the state of the building by calling `reset()` method from the battery class.
2. Resetting the `step_count` to 0. The `step_count` is used for temporal orientation in the electricity
load and solar generation profile.
Expand All @@ -58,8 +67,7 @@ def simulate_one_step(self, action: float) -> Tuple[float, float]:
1. Charging or discharging the battery depending on the amount.
2. Calculating the amount of energy consumed by the building in this time step.
3. Trimming the amount of energy to 0, in case it is negative.
4. Calculating the amount of excess energy which is considered lost.
5. Increasing the step counter.
4. Increasing the step counter.
:param action: Fraction of energy to be stored or retrieved from the battery. The action lies in [-1;1]. The
action represents the fraction of `max_battery_charge_per_timestep` which should be used to charge or
Expand All @@ -69,22 +77,28 @@ def simulate_one_step(self, action: float) -> Tuple[float, float]:
:returns:
Tuple of:
1. Amount of energy consumed in this time step. This is calculated by: `battery_energy`
+ `electricity_load` - `solar_generation`. Note that negative values are trimmed to 0. This means, that energy
can not be "gained". Excess energy from the solar energy system which is not used
+ `electricity_load` - `solar_generation`. Note that negative values are trimmed to 0. This means,
no energy can be fed into the grid. Excess energy from the solar energy system which is not used
to charge the battery is considered lost. Better use it to charge the battery ;-)
2. Amount of excess energy which is considered lost.
2. Electricity price in € cent per kWh.
:rtype: (float, float)
"""
electricity_load_of_this_timestep = self.electricity_load_profile[self.start_index + self.step_count]
solar_generation_of_this_timestep = self.solar_generation_profile[self.start_index + self.step_count]
electricity_load_this_timestep = self.electricity_load_profile[self.start_index + self.step_count]
solar_generation_this_timestep = self.solar_generation_profile[self.start_index + self.step_count]
electricity_price_this_timestep = self.electricity_price[self.start_index + self.step_count]

electricity_consumed_for_battery = self.battery.use(action * self.battery.max_battery_charge_per_timestep)
electricity_consumption = electricity_consumed_for_battery + electricity_load_of_this_timestep - \
solar_generation_of_this_timestep
excess_energy = 0
if electricity_consumption < 0:
excess_energy = -1 * electricity_consumption
electricity_consumption = 0
electricity_consumption_this_timestep = electricity_consumed_for_battery + electricity_load_this_timestep - \
solar_generation_this_timestep

if electricity_consumption_this_timestep < 0:
electricity_consumption_this_timestep = 0
self.step_count += 1
return electricity_consumption, excess_energy
return electricity_consumption_this_timestep, electricity_price_this_timestep

def _check_data_profiles_same_length(self):
if len(self.solar_generation_profile) == len(self.electricity_load_profile) == len(self.electricity_price):
pass
else:
raise ValueError('Data profiles passed to simulation are not of the same length.')
59 changes: 47 additions & 12 deletions building_energy_storage_simulation/data/data_preprocessing.ipynb
View file
Original file line number Diff line number Diff line change
@@ -1,13 +1,5 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "7a38795d",
"metadata": {},
"source": [
"The files processed in this notebook were obtained from [https://github.com/intelligent-environments-lab/CityLearn](https://github.com/intelligent-environments-lab/CityLearn)"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -19,6 +11,16 @@
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "markdown",
"id": "33a01875-61fa-4364-9114-8d02290100a5",
"metadata": {},
"source": [
"# Electricity Load and Solar Generation Data\n",
"\n",
"This data was obtained from [https://github.com/intelligent-environments-lab/CityLearn](https://github.com/intelligent-environments-lab/CityLearn)"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand Down Expand Up @@ -86,15 +88,48 @@
"solar_generation_profile.to_csv('preprocessed/solar_generation_profile.csv')"
]
},
{
"cell_type": "markdown",
"id": "f9ea2dad-74fb-40a3-a404-82c3dc672ab1",
"metadata": {},
"source": [
"# Price Data \n",
"\n",
"Data represents stock marked prices from Germany in 2022 taken from [Energy Charts](https://www.energy-charts.info/)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8ed67333-d1a5-4e45-b6b4-15df645893db",
"metadata": {},
"outputs": [],
"source": [
"electricity_price_profile = pd.read_csv('raw/energy-charts_Electricity_production_and_spot_prices_in_Germany_in_2022.csv')\n",
"\n",
"electricity_price_profile = electricity_price_profile['Day Ahead Auction'][1:]\n",
"electricity_price_profile = pd.to_numeric(electricity_price_profile)\n",
"electricity_price_profile = electricity_price_profile / 1000 * 100 # convert from MW to kW and from € to cents"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "79a1f7b4-e25c-4eb1-bdf9-d4c3e71a6570",
"metadata": {},
"outputs": [],
"source": [
"electricity_price_profile.to_csv('preprocessed/electricity_price_profile.csv')"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d39a0d2f",
"id": "feeba086-0635-4f3d-ad07-3fdca3448bbc",
"metadata": {},
"outputs": [],
"source": [
"weather_profile = pd.read_csv('raw/weather_data.csv')\n",
"weather_profile"
"pd.read_csv('preprocessed/electricity_price_profile.csv')"
]
}
],
Expand All @@ -114,7 +149,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.18"
"version": "3.10.13"
}
},
"nbformat": 4,
Expand Down
Loading

0 comments on commit d03af17

Please sign in to comment.