Execution token branch notes

Main goal

Execute the same block instances in parallel independently by introduction of execution token. Each block component (field, input variable, output) which varies through execution runs now additionally depends on token. To execute a scope a new token is generated, block instance bound to the execution token is produced from user configured block instance.

Block instance hierarchy

1. Block class (BC)
   init block class
2. Block instance (BI) <-- user configuration
   create a copy of BI with real token
3. Execution time block instance (EBI) <-- execution token

For manual execution we should provide some "initial" token, still tricky with meta-blocks, since internal scope will be executed automatically.

Automated execution

generally there are no big changes, since ET will be changed by meta-block actions

Still if some nested block has input port which is bound to the block in outer scopes we should identify which EBI of variable providing block should be used. Hence need to keep execution token's hierarchy creaded by meta-block scopes.

##Block fields block structure defined by the private field BlockSpecification (former BlockSerialiser) block consists of 3 types of fields:

1. [static] static fields, defined once for block class
2. [nonET] independent of ExecutionToken (ET), some can be modified by the user
3. [ET] dependent of ET

Each field should be declared in the ORM like style, so that entire block could be serialised either for JSON or to store it in DB/filesystem.

• BC contains static fields and prototypes for all other fields
• BI produced from BC and fills nonET fields
• EBI produced from BI by bound to the particular execution token. EBI has copies of static, nonET fields and utilizes

###Block fields which describes its behavior: [static]

• is_abstract: intermediate block in the inheritance hierarchy, hidden from the users
• is_block_supports_auto_execution: defines that block can be executed during scope automated execution
• _trans: Transition system defines finite automate (FSM)
• _block_spec: BlockSpecification
• declaration of the all block field prototypes:
  _uuid = BlockField("uuid", FieldType.STR, None, is_immutable=True)
_bound_inputs = BlockField("bound_inputs", FieldType.SIMPLE_DICT, defaultdict())

###Basic block fields : [static]

• block_group: Group for the toolbox
• name: User readable title (todo: rename to title)

[nonET]

• block_base_name: Prefix for stored payloads
• uuid: Unique block identifier
• exp_id: Parent experiment identifier
• scope_name: Parent scope name
• [sub_scope_name: Name of the created scope, only for meta-blocks]
• _block_spec: Keeps knowledge about the block instance structure. Provides methods to serialize into json and parse from json entire block.
• _bound_inputs: dictionarty with linked varaibles to the input ports

[ET]

• state: state in the FSM
• output variables (TODO: maybe completely separate them from block?)
• celery task handlers
• fields defined for the particular BC, e.g. for FetchGSE: source_file = BlockField("source_file", FieldType.CUSTOM, None)

Common methods

• save_params
• validate_inputs (!TODO: look at WTForms validators implementation)
• on_params_is_valid
• on_error
• ...

####To access volatile variables a new methods introduced:

• get_block_field_var(name, execution_token)
• set_block_field_var(name, execution_token, new_value)

Do action

• extract block method do_action to the independent function.
• don't invoke next action recursively

###Folder structure

Resources folder +

• /user id
• /experiment id
• /block id
• /execution token --> filename

###Data structures

• Meta-data (common and unique fields)
• Payload ( [de]serialization to store)
• JSON/dict presentation for the web client + preview version
• ?