use_cases
See - Actors for a list of actors
- High-level Summary
- Input data
- Configuring individual and collections of models
- Performing model runs and analysing results
- Technical Details
- Actor manages data
- Actor manages infrastructure sector models
- Actor manages system-of-systems models
- Actor manages model runs
- Some time later, Actor performs a model run
- Actor analyses results
- Actor Manages Scenario Sets
- Actor Manages Narrative Sets
- Actor Manages Spatial dimensions
- Actor Manages Temporal dimensions
A Scenario is a source of data which is used to manage uncertainties beyond the control of a decision maker. For all practical purposes, models treat scenarios as a source of model input values. As with other data sources, scenarios have a spatial and temporal resolution.
A Scenario Set is a collection of scenarios of the same type with the same facets. When configuring a SosModel, free inputs to a model will be associated with a Scenario Set, deferring the choice of scenario to the definition of the Model Run.
Chris has a set of electricity demand data for different anticipated usage scenarios. He defines a new Scenario Set using the Scenario manager and names it electricity demand. Within the 'electricity demand' Scenario Set he creates a new Scenario for each anticipated scenario and names them electricity demand (low), electricity demand (med) and electricity demand (high). He adds a facet to the first scenario and notes that the same facet pops up at all other scenarios. He completes the entries for each scenario, by uploading the associated data sets and makes sure that the spatial, temporal resolutions and units are set right.
- Actor navigates to the data management page
- «system» displays list of existing Scenario Sets
- Actor creates a new scenario set and gives it a unique name and description
- Actor manages scenarios
- «system» writes changes to «store»
3 a. Actor edits an existing Scenario Set
- Actor selects existing scenario from the list
- Actor amends attributes
- Actor save changes
- «system» write changes to store
3 b. Actor deletes an existing Scenario Set
- Actor clicks delete button on existing scenario set from the list
- «system» asks for a confirmation
- Actor confirms delete
- «system» write changes to store
- «system» displays list of existing scenarios
- Actor creates a new scenario
- Actor adds facet to scenario
- «system» adds same facet to all existing scenarios
- Actor selects a flat file containing value, locational and time data
- Actor selects relevant metadata including appropriate spatial and temporal resolutions
- «system» reads in data and writes to «store»
2 a. Actor edits an existing scenario
- Actor selects existing scenario from the list
- Actor amends attributes
- Actor save changes
- «system» write changes to store
2 b. Actor deletes an existing scenario
- Actor clicks delete button on existing scenario from the list
- «system» asks for a confirmation
- Actor confirms delete
- Actor saves scenario set
- «system» write changes to store
3 b. Actor removes a facet from a Scenario
- Actor clicks delete button on existing facet from the list
- «system» asks for a confirmation
- Actor confirms delete
- «system» removes faces from all scenarios within the set
- Actor save changes to scenario set
- «system» write changes to store
Narratives provide a mechanism that allows users access to individual model parameters across a system-of-systems model in an organised manner. Narratives are held centrally and make visible the key model assumptions which can override the default values for model parameters. (Note that the only difference of model parameters from model inputs is that the data are provided by a user, and not by another model via a dependency).
By defining a new narrative variant, users can define one or more narrative policies that override the default values of the model parameters. These data are stored in a sparse manner, where only those entries that are present override the defaults.
Within a modelrun, where multiple narratives contain the same parameters, narratives are ordered into a hierarchy by the user to indicate which assumptions should take precedence.
Freddy is conducting a system of systems analysis of energy and digital communications sectors. He has already configured two models of energy and digital systems and has defined a number of model parameters for each of the models.
Freddy wants to explore possible futures using these coupled models under high and low technological change in these sectors. He creates a 'technological change' narrative for his system-of-system model called 'SimEnergyDigital'. The narrative form shows the available parameters from each of the energy and digital comms models, and he picks from the list to choose only those relevant under the heading of 'technological change'. These include parameters for cost, efficiencies and also a behavioural parameter governing the rate of consumer adoption of new technology. He then creates two narrative variants titled "high tech" and "low tech". Freddy now fills in values for the parameters he wishes to override.
Freddy can now define and run modelruns that include either his "high tech" or "low tech" narratives. The assumptions included in these narratives pass through to each of the simulation models within the system-of-system model at run time. The narratives provide a handy mechanism that enable Freddy to organise his analysis of the system-of-systems model of energy and digital communications.
Freddy has now extended both his energy and digital models to include financial
reporting. The models both require a discount rate parameter to be defined,
and this should only be defined once at the SosModel level.
Freddy navigates to his SosModel and clicks clicks "Add New Global Parameter". He is greeted with the same "Add Parameter" form seen on the Sector Model page, so fills in the name "discount rate" and other metadata, attaching the data file.
Now, instead of attaching a file to his narrative variant definition, he can choose the global parameter defined within his SosModel.
- Actor navigates to narratives page
- «system» displays list of existing Narratives ('Governance')
- Actor creates a new narrative and gives it a unique name and description (e.g. 'Technological Change')
- Actor adds a system-of-systems model to the narrative
- «system» displays list if all parameters contained within the simulation models
- Actor adds one or more parameters from the parameter list to narrative
- Actor adds one or more variants ('high tech', 'low tech')
- Actor links to data file for each of the parameters for each of the variants
- «system» writes changes to «store»
3 a. Actor edits an existing Narrative
- Actor selects existing Narrative from the list
- Actor amends attributes
- Actor save changes
- «system» write changes to «store»
3 b. Actor deletes an existing Narrative
- Actor selects existing Narrative from the list
- «system» requests delete confirmation
- «system» write changes to «store»
- «system» displays list of existing narrative variants
- Actor creates a new narrative variant
- Actor Configure Narrative Files
- «system» writes to «store»
2 a. Actor edits an existing narrative variant
- Actor selects existing narrative variant from the list
- Actor amends attributes
- Actor save changes
- «system» write changes to «store»
2 b. Actor deletes an existing narrative variant
- Actor selects existing narrative variant from the list
- «system» requests confirmation
- «system» write changes to «store»
- «system» displays existing narrative files
- Actor creates a new narrative file
- Actor enters new values for parameters
- «system» displays list of dimensions
- Actor creates new dimension
- Actor enters unique name, selects type (temporal or spatial) and enters path to the data file
- «system» validates and writes dimension to «store»
2 a. Edit existing dimension
- Actor selects existing dimension
- Actor edits attributes
- «system» validates and writes changes to «store»
2 b. Delete existing dimension
- Actor selects existing dimension
- «system» confirms deletion action
- «system» writes changes to «store»
An Intervention is a possible action which has a name, a number of
required attributes, including capital cost, economic_lifetime, capacity
and operational_life, other attributes and location, but no build date.
Interventions are intrinsically linked to sector models in that they correspond to one, and only one simulation model.
Interventions are collected into a package so that the Decision Manager can choose which to perform, where and when. An Intervention is the smallest divisible unit of a decision within the framework.
- Actor navigates to the intervention manager screen
- «system» displays list of existing interventions
- Actor adds new interventions with required attributes, suggested attributes and/or custom attributes
- Actor saves intervention
- «system» saves intervention to the «store»
3 a. Import interventions from a yaml file
- Actor clicks "import interventions" button, dialogue asks Actor to select a file, (dialogue filters to *.yaml, *.yml files).
- Actor selects one or more yaml files and clicks "import" button. Dialogue box closes.
- Interventions are imported and appear in intervention list.
3 b. Edit an existing intervention
- Actor selects the intervention to edit
- «system» retrieves intervention from store
- Actor makes changes
- «system» writes changes to «store»
3 c. Delete an existing intervention
- Actor selects the intervention to delete
- «system» requests confirmation
- «system» writes changes to «store»
3 a. One or more interventions already exist in list
- «system» displays list of duplicate interventions.
- Actor chooses duplicate from the list to retain
Harpreet is an energy modeller who wishes to couple the energy supply
simulation model she is developing with an existing water supply model. She
wants to investigate the resilience of the electricity system
to constraints on water abstractions during drought events. Harpreet has
written a wrapper for her model executable using the smif.SectorModel class
and identified one key input to the model (electricity_demand) and
two key outputs (water_demand, unmet_demand).
Harpreet also writes a spatial and a temporal dimension file
which apply to all inputs and outputs.
Harpreet logs onto the «system» and navigates to her project. She chooses to
create a new model configuration, which she names es_resilience. She then
directs the system to her wrapper file,
registers the spatial and temporal resolutions with the gazateer and uploads
the data.
She then adds the inputs and outputs, associating each of them with the spatial
and temporal resolutions registered previously. She checks the visual
representation of her model configuration and notes the one input and two
outputs detailed on the diagram. Hovering her mouse over, she can view the
meta-data associated with the input and outputs. She adds some notes to her
configuration, and sets the model version to match that of the model
from her version control system.
- How do we manage simulation model versions from the interface?
- Should we use a composite type approach, where a SosModel is a model, a SectorModel is a model and a SosModel comprises of a collection of models? In this situation, a scenario would also be a subclass of Model, with one ModelOutput deferring the actual data loaded to runtime (and specified in the model run).
Each of the sector models needs to be configured so that it
can be run by smif. Models are wrapped using the SectorModel class which
exposes initialise and simulate methods.
Models have inputs, outputs and parameters,
each of which have associated dimensions.
- Actor navigates to simulation model configuration interface page
- «system» displays a list of existing simulation model configurations
- Actor adds a new simulation model configuration
- Actor enters unique model name and the path to the model wrapper (to execute the model)
- «system» displays list of classnames in model wrapper
- Actor chooses the classname from list
- Actor Manages Inputs/Outputs
- Actor Manages Interventions
- Actor Manages Parameter Configurations
- Actor saves simulation model configuration
- «system» saves configuration to «store»
- «system» returns focus to scetor model list page now displaying added sector model configuration
3 a. Edit an existing simulation model configuration
- Actor views the list of model configurations
- Actor selects model configuration to edit
- «system» requests configuration data from «store»
- "Edit model configuration" form is displayed with entries for various attributes.
- Actor edits the entries and saves changes
- «system» saves changes to «store»
3 b. Delete an existing simulation model configuration
- Actor views the list of model configurations
- Actor selects model configuration to delete
- «system» requests confirmation of deletion
- Actor confirms deletion
- «system» saves changes to «store»
5 a. «system» cannot import model wrapper (compile error)
- Error reported back to Actor
- Return to step 2.
2 a. No model configurations exist
- «system» displays message saying that there are no existing configurations
8 a. store connection error
- Resolve connection problems
- «system» displays list of sector model inputs and outputs
- Actor adds new input or output, choosing from list of spatial and temporal resolutions, and units, name and description
- Actor saves input/output, «system displays list of sector model inputs»
2 a. Spatial/temporal resolution or unit not defined.
- «system» directs Actor to add spatial/temporal resoltions prior to trying to add input to sector model
- «system» allows partial save of sector model
The modeller implements the simulation method of the ScenarioModel class so that the wrapped simulation model can be run by smif. The implementation must receive parameter and scenario data passed into it for each timestep, and return results from the simulation model.
At runtime a simulation model needs access to simulation parameters, scenario data and data output by other models.
A SectorModel makes requests to the data layer using a friendly API.
In one ModelRun a SectorModel might get its data from scenario data (e.g. 'High Population projection'), in another ModelRun that might be the output of a demographics model which runs each timestep too. These details are hidden from the SectorModel by the DataHandle.
The containing CompositeModel (a SosModel or ModelSet) has responsibility for creating (or adapting) a DataHandle with the correct ModelRun name, current timestep, timesteps, ModelSet iteration.
Where a ModelRun or ModelRunner simulates multiple options for decisions, it is responsible for creating (or adapting) a DataHandle specialised for a given decision iteration.
For example, within SectorModel.simulate(), we can call:
data_handle.get_data('population')
data_handle.get_base_timestep_data('road_trips')
data_handle.get_previous_timestep_data('dynamic_model_output')
data_handle.get_parameter('travel_cost_coefficient_alpha')
which will return the correct data or parameters, whatever the data source and indifferently to whether the parameters are local or global, default or overridden by a Narrative.
- SectorModel gains access to required data for the relevant modelrun (and for any sub-tasks within the modelrun such as iterations while solving decision algorithms or dependency loops).
At runtime a simulation model needs to write out or return results.
A SectorModel makes requests to the data layer using the friendly DataHandle API. As with Request Data above, the containing CompositeModel or ModelRunner has responsibility for creating or adapting a DataHandle which will access the correct subset of data.
For example, within SectorModel.simulate(), we can call:
data_handle.set_results('population', data)
- SectorModel writes its results so that they are available to containing processes and other SectorModels.
At runtime a containing process (e.g. ModelRunner, ModelSet or SosModel) needs to access the outputs from its contained models.
At test time or in an interactive session (notebook. REPL) it is also convenient to have straightforward access to results.
As with Request Data and Return Results, the details of accessing model results should be hidden behind the DataHandle API.
For example, within ModelSet.simulate(), we could call:
data_handle.get_results('population', model_name='demographics', modelset_iteration=10)
data_handle.get_results('population', model_name='demographics', modelset_iteration=11)
data_handle.get_results('population', model_name='demographics', modelset_iteration=12)
in order to then test whether a model's results are converging.
- A ModelSet accesses results computed by any of the SectorModels which it contains.
Gerald wishes to expose several of the energy demand model parameters to those who will use his model. There are two key model parameters he wishes to add today, smart meter energy savings and technology uptake which needs to be specified for each end-use and technology.
Gerald logs onto the system and navigates to the energy demand model configuration. He navigates to the model parameter list, and adds a new parameter.
The first parameter, smart meter energy savings is a floating point number, with a suggested range from the literature of between 3%-10%. He adds the unique name smart meter energy savings, a helpful description of what this parameter does, the type (float), an absolute range [0, 1] and a suggested range [0.03-0.1], a default value of 0.03 and % for the units attribute.
Gerald adds another new parameter with the name technology uptake %%tech%% %%end-use%%. First, Gerald defines the two dimensions over which the technology uptake parameter will be expanded. He adds the set tech contains a list of three technologies [washing machine, dishwasher, incandescent lighting, LED lighting] and the set end-use contains [wet, lighting] to the parameter configuration. He then appends the default values of the parameter attributes to the parameter, including range: [0, 1], type: float, description: "Technology uptake for %%tech%% for end-use %%end-use%%". After saving the parameters are visible for each of the tech and end-use combinations e.g.
name: technology uptake washing machine wet
range: [0, 1]
type: float
description: "Technology uptake for washing machine for end-use wet"
Defining a model parameter is a special case of defining a model input or output. Model parameters may have one or more dimensions, have a default value, a unit, may require exhaustive definition, and have a data type.
Examples include::
technology uptake - the proportion of an enduse within a sector which is
serviced by a technology at a year, (the intermediate values are interpolated
in the model)
vehicle fleet composition - the proportion of engine types within a vehicle
type category for each year
One implementation could be:
An inputs.yaml file::
inputs:
name: technology_uptake
dimensions:
- year
- end-use
- sector
- technology
default value: 0
ranges: (0,1)
dtype: float
constraints: 'sum over technologies supplying end-use - sector sum == 1'
unit: '%'
dimensions:
year:
- 2010
- 2020
- 2030
- 2040
sector:
- residential
technology:
- washing machine
- dishwasher
- incandescent lighting
- LED lighting
end-use:
- wet
- lightingA data_file for each input:
technology_uptake.csv
year,end-use,sector,technology,value
2010,wet,residential,washing machine,0.35
- «system» displays list of model parameters
- Actor creates new parameter configuration with a unique (to the simulation model) name, plus attributes including description, range, suggested range, dimension, type (float, int, bool)
- Default data is added separately after definition of the parameter metadata
- «system» validates changes and writes parameter configuration to «store»
2 a. Edit an existing parameter
- Actor selects a parameter to edit from the list
- Actor amends parameter attributes
- «system» validates changes and writes changes to «store»
2 b. Delete an existing parameter
- Actor selects parameter to delete
- «system» requests confirmation of deletion
- «system» saves changes to «store»
1 a. No parameters exist yet
- «system» prompts Actor to create a new parameter configuration
2 a. Parameter fails validation checks e.g. default value not in range
- «system» prompts Actor to amend parameter attributes
A SosModel is a collection of sector models with a requirement for data input.
Chris, Harpreet's colleague, has finished work on the water supply model, and has already configured it using the «system». He's now ready to define a new system-of-systems model (SosModel), which will couple the two sector models together.
Chris logs into the «system» and selects the water-energy-resilience project.
He navigates to the SosModel configuration page and follows the instructions
shown as this is the first time he's configured a SosModel. The instructions
direct him to create a new SosModel configuration. Presented with a list
of previously defined sector models, he selects the energy model which was
configured by his colleague Harpreet called es_resilience. He reviews the
notes and description associated with the energy model and scans them for any
particular issues Harpreet identified.
He then selects his water model and confirms the additions to his SosModel
project.
Now two models exist in the SosModel configuration, Chris needs to link the
model input and outputs to one another to define dependencies. The «system»
displays a visual representation of the models inputs and outputs.
He connects the water_demand output from energy model to the water_demand
input of the water supply model.
A «system» warning notifies Chris that he has connected
inputs and outputs with a different spatio-temporal resolution. Luckily,
Chris is prepared, and has written a function to cope with this which he wishes
to use instead of the default assumption of area weighting. He selects the
function and adds it to the configuration. Chris now selects the remaining
input to the energy model, electricity demand, and categorises its source as
a scenario input. Then, he selects the rainfall input to the water supply
model and categorises its source as a scenario input. This will allow him to
select multiple different scenarios when running this SosModel i.e. defer the
selection of the model inputs to the definition of the model run.
Chris saves the SosModel configuration and goes for a cup of tea.
- Actor navigates to SosModel configuration
- «system» shows list of existing SosModel configurations
- Actor creates a new SosModel
- «system» displays a list of available simulation model configurations
- Actor selects one or more sector models from list
- Actor manages sector models as required
- «system» provides list of the aggregate model inputs required and the model outputs produced by the collection of sector models
- Actor manages dependencies
- «system» computes the running order of the models suggested by the graph of dependencies and indicates where iteration will be necessary
- When happy with the collection of sector models in the SosModel, the defined dependencies, the running order, and the remaining inputs and outputs, the Actor saves the configuration
- «system» validates SosModel configuration
- The focus returns to the list of existing SosModel configurations, now displaying the newly created SosModel
4 a. There are no existing simulation model configurations
- «system» asks Actor to add a new SosModel configuration
4 b. The required simulatiom model configuration does not exist
- «system» asks Actor to add a new SosModel configuration
7 None of the simulation model configurations have inputs or outputs
- Edge case - no use for smif in this situation. Simulation model configurations are invalid
10 Running order is infeasible
- Raise an error and return to linking of inputs and outputs step
13 store connection fails and unable to save configuration
- Save configuration locally to text file
3 a. Edit an existing SosModel
- Actor selects SosModel to edit
- «system» retrieves SosModel configuration
- Actor edits SosModel configuration and saves changes
- «system» writes changes to «store»
3 b. Copy an existing SosModel
- Actor selects SosModel to copy
- «system» duplicates SosModel configuration and requests unique name
- Actor enters unique name for new SosModel
- «system» writes changes to «store»
3 c. Delete an existing SosModel
- Actor views the list of SosModel configurations
- Actor selects SosModel to delete
- «system» requests confirmation of deletion
- Actor confirms deletion
- «system» saves changes to «store»
- «system» displays lists of model outputs and inputs and spatio-temporal resolutions for current SosModel
- Actor adds dependency by choosing a pair of inputs and outputs between two models
- Actor defines custom spatio-temporal conversion functions along each of the dependencies, or accepts the default assumption of area weighting
Harpreet and Chris now wish to conduct a number of model runs using their
combined energy and water system-of-systems model. They start by defining a
reference scenario model run. Harpreet navigates to the model run configuration
page, and observes the empty list of model runs. Clicking on the 'New Model Run'
button, she enters a description of the model run, and selects the SosModel
configuration es_resilience from a list. A visual graphic representing the
SosModel pops up showing the inputs rainfall and electricity demand and the
outputs unmet demand. The unmet demand parameter is automatically linked
to a results node. The rainfall and electricity demand are unlinked.
Harpreet selects the rainfall sub-category of climate scenario central case
from a series of drop down menus, and rainfall appears as a data source on
the visualisation. She drags a link between the two rainfall nodes to link
them, accepting the default area-weighting for spatio-temporal conversion.
She then selects the electricity demand data source from the New Data Source
menu and links that to the electricity demand node.
The Model Run collects scenarios, timesteps, narratives, and system-of-system models into a package which can be deployed and run.
Once a Model Run has been performed (simulated) successfully, then the configuration is locked to maintain data provenance. Model Runs can only be duplicated, edited and rerun.
Actor chooses to create a new model run
- Actor navigates to the model run screen
- «system» displays a list of model runs
- Actor creates a new model run, entering a unique name, and choosing a SosModel from the list of available SosModel
- Actor chooses from a list of time steps which define the years for which the model will run
- For all inputs to the SosModel which do not match to a data source (either an external data source or another model output), «system» displays a warning
- For each Narrative, Actor chooses a Variant
- For each Scenario, Actor choose a Variant
- For each model input: Actor provides data for model input
- Actor saves the model run configuration
- «system» saves new configuration to the «store»
- «system» adds displays configuration name and summary metadata in the Model Run list
-
No SosModel definitions have been configured
- Configure a SosModel
2 a. Configure a new model run from an existing model run
- Actor creates a new Model Run, checking a tick box indicating that she wishes to copy an existing Model Run
- «system» notifies Actor to choose an existing Model Run
- Actor selects from a list of model run configurations
- New model run is created on «system»
- «system» notifies Actor to enter a unique name for the model run
- Actor is able to edit the configuration and save/cancel the changes.
- «system» tags the model run with a warning symbol if it appears to duplicate an existing model run (nothing is different from the parent model run).
2 b. Copy an existing Model Run
2 c. Delete an existing Model Run
2 d. Edit an existing Model Run. Only non-running
Within a Model Run definition, any free-hanging model inputs in the chosen SosModel must be linked to scenario data
- Actor chooses to add scenario data for the model input.
- «system» displays list of available scenarios within the scenario set defined wihtin the sector model definition.
- Actor selects one of the scenarios
-
No scenarios have been defined
- «system» asks Actor to define scenario data sources
After finishing the configuration of the model run, Harpreet navigates to the
Model Run list, and scans through the list. She filters by model run status, to
show only runs with the status Ready. She selects the model run, and clicks
Submit Model Run. The system responds informing her that the run has been
submitted to the job queue
- Actor selects a model run
- Actor submits run for simulation
- «system» locks and deploys model run
- «system» notifies Actor that model run has been submitted
- «system» creates results entry for model run
- «system» notifies Actor of successful completion
- «system» populates results entry with data from «store»
3 a. Model run fails 3 b. Model is infeasible (optimisation)
- PlanningHorizon is a parameter of the ModelRun
- Behaviour of Decision Manager is a function of PlanningHorizon and DecisionModule type (planning/rules/optimisation)
- Actor builds SosModel object adding SectorModels and Dependencies
- Actor creates instance of ModelRunner and passes SosModel object to ModelRunner
- ModelRunner Run the SosModel and returns results
- «system» saves results to store
- Actor receives the run order from ModelRun along with SosModel, ModelHorizon and DecisionManager
- While the DecisionManager is not yet satisfied:
- Actor runs decision models, awaiting decisions
- Actor runs simulation models, awaiting results
- Actor creates a JobGraph of decision models from the DecisionManager
- Actor submits JobGraph to JobScheduler
- Actor awaits JobGraph completion
- Actor creates a JobGraph of simulation models from the SosModel
- Actor submits JobGraph to JobScheduler
- Actor awaits JobGraph completion
- JobScheduler (perhaps interfacing with other systems) eventually triggers 'run' to execute a model, in the context of a given model run (hence scenarios/parameters/other models), timestep and decision iteration, with a given shared store available for input, data and results.
- JobScheduler writes contextual information to a context.properties file
- JobScheduler starts Model process with environment variable and/or command line argument pointing to context.properties
- Model constructs
DataHandleor uses its own approach to data-access, initialises itself, runs, writes results.
- JobScheduler posts context and run-instruction to model server.
- Model server instantiates a model run and runs, etc., writes results.
Harpreet returns to the Model Run screen and filters on Successful
status. No model run appears, as her run has not yet finished. She removes the
filter and sorts by Submission Time and finds her Model Run at the top of
the list. She selects it and tries to click View Results, but the button is
greyed out.
Later, Harpreet returns to the Model Run screen and filters on Successful
status. He model run appears at the top of the list.
She selects it and clicks View Results. The black results dashboard is
displayed with space where a range of useful metrics could be shown. She
configures the dashboard with a number of useful result widgets, so that she can
ensure at a glance that the model is performing as it should.
- Actor navigates to results viewer
- «system» displays list of completed model runs with linked results
- Actor selects model run
- «system» displays model results
- Actor navigates to bundle viewer
- «system» displays list of bundles
- Actor Manage bundles
- Actor selects a bundle
- «system» retrieves bundle from «store» and displays results comparison
4 a. Bundle contains model runs with no results
- «system» shows results of only model runs with results and warning for others
5 a. Model run results are identical
- «system» displays warning that results difference cannot be shown as results are identical
A bundle is a collection of model runs within a project which enables results comparison of one or more model runs.
- Actor navigates to bundle viewer
- «system» displays list of existing bundles
- Actor creates new bundle
- «system» displays list of model runs of all statuses
- Actor selects one or more model runs to add to bundle
- «system» saves bundle to «store»
Actor is on the model run page and a successfully completed model run exists in the model run list
- «system» displays list of model runs
- Actor selects completed model run in the list of model runs
- «system» displays results dashboard for model run
The choice of decision mechanism changes the manner in which the model is run. For example, with pre-specified planning, the model state is pre-specified and the simulation can run straight through (could be parallelised along years if no state is passed from year to year).
With the rule-based approach, decision iterations are required until the rule thresholds pass successfully. Previous timesteps need to complete before moving to the next timestep.
With the MOEA approach, each decision iteration contains an entire pre-specified pathway over all timesteps in the model horizon which can be simulated in parallel. Bundles of decision iterations make up a generation, and new generations are generated upon completion of each bundle.
Carol defines a pre-specified planning pipeline by choosing combinations of interventions, locations and build-years according to the pool of available interventions and constraints imposed by the governance, technology and behavioural narratives.
Carol begins the model run. The model runner initalises the ModelRun and timesteps
which builds the SosModel and contained objects.
The ModelRunner also initialises the DecisionManager, which references
the sector-specific planning pipeline as its sole strategy and sets the
decision_iteration to 0.
The ModelRunner calls the SosModel simulate method for the first year contained
in the decision_iteration (note that a decision iteration can hold between 1
and H planning timesteps, where H is the length/cardinality of the set of
time intervals).
For each set in the list of ModelSets:
for each model in the set:
the sector model obtains the current state of the system by calling the DecisionManager.get_state() method
Initial conditions are the list of cumulative interventions that make up the structure of the infrastructure networks contained within the SectorModel.
Each decision iteration represents one route through the decision space.
The DataHandle provides a get_decisions() method, which returns the list of
interventions for the current planning timestep.
Derek defines a series of business rules in the rule-based decision layer which represent how the solid waste sector performs its regulatory duties. He does this by linking the value of model outputs with conditions and defining actions to be taken when those conditions are not met.
He starts up the model run. The model runner configures the decision manager with the rule-based algorithm. The algorithm compiles the rules, runs the SosModel simulation (which includes the solid waste model) for the first time period in a first iteration. A high 'emissions' model output causes one of the rule conditions to fail, and the algorithm chooses one intervetion from a set of interventions as the action. The algorithm reruns for a second iteration and the new action corrects the path of the model. Now all the conditions are met, the model completes successfully, and the sequence of decisions is returned to the model runner.
While the model is running, Derek observes the log which details the actions which are triggered by the model outputs and the performance of the model.
Jenny sets up her model-run with an optimisation algorithm which chooses decisions according to the criteria of least cost. The planning window is set to the default value, which is the same length as the model horizon. She sets the model running. The algorithm runs a number of iterations until it arrives at the lowest cost.
In each iteration, the algorithm selects, for each timestep in the horizon, a number of interventions from the list of available interventions and runs the simulation on the SosModel.
After each run of the SosModel, the algorithm now receives results of the simulation and assesses whether it should continue exploring the decision space or exploit the current best available decision.
Jenny views the progress of the algorithm via the logs, which detail the time, interventions and model results for each iteration and timestep the algorithm visited.
Once the algorithm terminates, it returns the sequence of interventions in each timestep that meet the objective function and constraints.
Jenny modifies her optimisation algorithm to try to represent the short-term decision horizon in the water supply sector. She sets the planning window to 10 years, meaning that in each timestep, the algorithm only looks 10 years ahead when assessing whether the current decision is optimal, all other information is disregarded. This results in the creation of a sub-problem with a horizon of 10 years rather than the entire model horizon. The algorithm iterates over this sub-problem until it finds an optimal solution, fixes the decisions in the initial timestep, and then moves to the next timestep whereupon the process begins again.
Fred wishes to explore the trade-offs between three objectives, cost, resilience and carbon emissions (the trilemma). He develops a multi-objective evolutionary algorithm which finds a pareto optimal set of decision options. In each iteration, called a generation, the algorithm evaluates a number of candidate decision options individuals, which denote which interventions are made when and where. In a typical analysis, the algorithm is run for 250 generations, where each generation contains 100 individuals. Given a time horizon of 50 years with 10 time steps of 5 years, this require 10 * 100 * 250 = 250,000 executions of the system-of-systems model.
The algorithm views the decisions available to the system-of-systems model as a binary array. This array is permuted with the population of individuals within a generation over generations. Each individual represents a different combination of the available decisions. The decision module is responsible for providing the algorithm with an interface to the system-of-systems model. The decision module accumulates the available interventions from each of the simulation models in the system-of-systems model and presents them two the algorithm as a binary array. For each individual combination chosen by the algorithm, the decision module then converts the binary array back into interventions which are passed to the simulation models at runtime. The decision module then passes back the objective values for the three objectives cost, resilience and carbon emissions.
Interventions are chosen in each timestep in a Planning View. Interventions are only persisted for the first timestep in the Planning View.
The set of Planning Views is a deterministic function of the model horizon and the planning window. If the planning window does not cover the model horizon, on each iteration, the algorithm moves to the PlanningView which begins with the next timestep and continues until the whole model horizon has been covered. The system state is copied from the end of the previous timestep.
An Iteration is a single pass through the timesteps in a planning view.
Modeller sets PlanningWindow parameter to 1
- ModelRunner requests PlanningView for SosModel configuration
- PlanningView returns decision epochs (set of timesteps for which to make decisions) for first year
- ModelRunner creates new Iteration with current year and planning view.
- ModelRunner sends iteration to decision manager with a request for decisions.
- Decision Manager adds decisions to Iteration.
- ModelRunner performs Iteration by sending decisions to SosModel with simulate command.
- SosModel sends ModelRunner results.
- ModelRunner attaches results to Iteration
1 a. Planning Window is 1 < x < ModelHorizon
- In each timestep, decisions are chosen for the current timestep, and then each timestep in the PlanningView is performed to assess performance. Algorithm then moves to the next timestep. 1 b. Planning Window is set to 0 (don't use window)
- ModelRunner runs over all timesteps in the model horizon
- Actor navigates to planning interface
- «system» displays list of strategies
- Actor creates a new pre-specified planning strategy with a unique name and a description
- For each of the sectors in turn, Actor associates one or more interventions from the list populated by the sector models with a timestep from the Model Run
- «system» writes the data to the «store»
- {as above}
- Interface doesn't let Actor create a new planning strategy and displays a warning message saying that no interventions have been defined.
- Model Runner initialises Decision Manager with strategy bundle
- DM requests the list of interventions from the <>
- Model Runner iterates over each iteration-timestep pairing yielded by the DM
- For each timestep, Model Runner solves the SosModel
- For each sector model in the SosModel, sector model gets the current state of the system from the decision manager.
Title: Actor: Scenario:
Extensions: (when things go wrong)
Precondition: (what must be true to begin this use case)
Postcondition: Stakeholders: Technology list: Scope: Level:
TODO: process these into use cases as above.
Labels:
- smif
Main objective is to support consistent analysis of variability:
- e.g. weather data with 100 realisations for a climate scenario, drives energy demand, generation, and hydrology hence water supply
Currently can imagine 100 model runs created by script - does the job.
Imaginative hack - a fake decision module that iterates over scenario realisations instead of/as well as making decisions.
Can we do better? Single model run with realisations as a dimension in the scenario/s, 'explode' sos model graph to include 100 copies of each model (in the case where each model is ignorant of the 'realisations' dimension going on around it).
Maybe just a presentational thing in the GUI - group model runs, assert that all attributes are identical except for varying along realisation.
Other motivations to group model runs: strategy/scenario exploration, sensitivity analysis, parameter sweeps...
Labels:
- smif
- smif ideas
Motivations:
- clean separation between coordination and simulation/model-running
- avoid need to load wrappers into the same python process as smif
- performance: (eventually) reduce layers of IO
- distribution/platforms: switch local method call to call subprocess, other job-runner, container/orchestration..
Design ideas:
- smif SDK essentially implements DataLayer connection/DataHandle creation, could be in multiple languages
- then building a model to be smif-runnable is a matter of importing a language-specific SDK and defining a CLI entrypoint, and munging from DataArray/list[dict] to model types. That is, smif-SDK has functions that directly read/write to the scratch store, and exposes the DataHandle API to the model at runtime for some simplicity.
- could avoid a layer of IO e.g. smif reads data from scratch backing store and passes to wrapper, wrapper writes to txt/CSV and starts model subprocess, model subprocess reads txt/CSV again...
- could look at distributed (in-memory?) object store as backing too - plasma, shm, ... maybe avoiding touching disk and a layer of de/serialization
So smif-runner or job-scheduler calls a series of subprocesses (no python dynamic class-loading) or runs a series of containers.
Labels:
- smif
- smif ideas
Add the following to a sample project:
- example of a custom dimension. Example project does not currently use any interval definitions, or csv based dimension data, only shapefiles
- demonstrate data conversion using default DataAdapters
- demonstrate custom data conversion using user specified DataAdapter
(See comments for more items to add).
Labels:
- errors
- logging
- smif
- smif ideas
At present, all logs get dumped into one stream which is very noisy. It would be good to separate the multiple log streams according to the simulation models and integration framework (and decisions).
Labels:
- logging
- smif
- smif ideas
Provide modellers with a logger handle that writes out custom log messages for an integrated sector model. This helps modellers to debug or analyse their models within a system-of-systems without having to scroll through smif system messages.
- One log per modelrun
- Each modelrun should results in a new empty logfile
- A tag should show the source of the log message
[energy_demand] Finished run in 23s
[water_supply] Start run timestep 2020
see also
Labels:
- documentation
- smif
- smif ideas
see if pre-commit install can be added to setup script
Labels:
- modelrun
- smif
- smif ideas
Add configuration metadata to modelrun results and protect the user from warm-start when the config changes (from PR #153)
Use an MD5 hash of the configuration folder, check on new run, and refuse warm run if the configuration has changed.
Labels:
- cli
- integration
- smif
- smif ideas
When the modelrun / sector_model / .. configuration changes, results from a previous modelrun cannot be used anymore to resume a modelrun using a --warm start.
Labels:
- cli
- integration
- smif
- smif ideas
The warm start now resumes the modelrun from the last timestep. Extending this on the last modelrun decision or iteration would speed this up even more.
Labels:
- smif
- smif ideas
Synthesise a dataset e.g. as might be used by energy demand, benchmark I/O costs of using csv and yaml
Labels:
- data_handle
- gui
- smif
- smif ideas
Delegate type checking to the data interface, and handle errors in HTTP/GUI. Some inline checking on forms.
Provide a "dashboard" facility which allows modellers to easily specify visualisations to analyse model runs
Labels:
- smif ideas
- visualisation
Obtaining quick feedback on the outputs of a model run is useful while conducting individual scenario analyses. Modellers may have interests in viewing information focussed on their sector, and so the ability to collate a dashboard which provides relevant summaries to the modeller is useful.
Labels:
- data_handle
- smif
- smif ideas
It's useful to log some data-level information (such as adding time intervals), but this results in data overload at runtime when using debug level.
We could add a low debugging level to handle these, or we could move most logging to the info level?
Generalise time to decision timesteps and simulation intervals, with intervals contained within each timestep.
Labels:
- fundamentals
- smif
- smif ideas
We can implement a more general treatment of time within smif where the timesteps used for decision making can be a super-set of the intervals over which the system is simulated. This would allow a move towards lock-step type use of smif where models increment over smaller intervals of time rather than the simulation of entire years before information is exchanged.
Labels:
- smif
- smif ideas
The framework should facilitate meta-studies (e.g. global sensitivity analysis) of the sector models to enable selection of the dependencies for inclusion in the study. For example, the global sensitivity analysis library SALib requires definition of parameter names and ranges and produces a sample which can then be used to run a Monte-Carlo type simulation of the model. Subsequent analysis of the input sample and results provides a number of useful sensitivity metrics.
