-
Notifications
You must be signed in to change notification settings - Fork 4
Experiment decomposition V2
!!Draft
##Workflow
An experiment is described by the workflow. Workflow is constructed from blocks ( graph nodes) and data bindings (graph arcs) between block outputs and input ports.
##Scope
From the abstract point of view, scope encapsulate a workflow graph (DAG). Meta blocks create sub-scope which create space to define sub-workflow. Sub-scope owner controls execution of sub-workflow. Thus scopes creates hierarchy.
##Scope variables
Most of the blocks produces variables. Each variable has data type and assigned to the particular scope. Variable can be bound to the input port of the complimentary data type it is visible by the target port block.
Each blocks lies in some scope. Most general one root is created by experiment. Nested scopes are created by meta blocks like cross-validation.
If block provides a variable it is visible in the block scope and any nested sub-scope. User can bind variables visible to the input ports.
Three options:
- Run single non-meta block
- Run entire experiment ( or run global scope)
- Run some meta-block (or run sub-scope)
The main addition of V2 is to introduce scope execution token. New token should be issued when some scope begins to execute. Changeable block parts and output variables should unique for pair (Experiment, Token).
#Data models
Simple handle to scope
- scope_name
- parent experiment id
- owner block uuid or None for root scope
Scope variables represent results of block execution.
- parent block
- parent scope
- execution token or None for prototype
- variable name
- variable type
- readiness state
- variable itself ( metadata and payload) according to the data type
Blocks is surely the most complex entity in the miXGENE system. Here we split it into the 3 parts:
- methods ( including block actions)
- State machine, which defines which action can be applied
- any other static properties
- Dynamic input ports
- Block parameters mutated into input ports
- Data bindings
- User provided parameters values
- Produced scope variables
- FSM state
- Block field changeable by its methods (but without the user access)
- execution token or None for prototype