Conditional Branching in a Pipeline

[sgaseretto]

Does Elyra support in a way "conditional branching" in ? Imagine Notebook A performs some evaluation, if that evaluation returns a value above certain threshold, only run Notebook B, but if the value is below that threshold, only run Notebook C.

[ptitzler]

TL;DR: Not currently.

The only conditional processing that is currently performed evaluates the notebook (or Python script) execution return code, meaning if notebook or Python script execution fails downstream nodes are not processed.

A potential workaround you could implement would be the following:

                     /--- notebook B
notebook A  --
                     \--- notebook C

• notebook A creates and output file (containing the value) and declares the file as output
• notebooks B and C read the content of the file and evaluate the condition (the file is automatically made available to them)

Edit 1: I'll post an example pipeline later today for illustrative purposes

[ptitzler]

The pipeline example is located in https://github.com/ptitzler/elyra_pipelines and runs in local environments and on Kubeflow Pipelines. The implementation is using a fairly rudimentary approach. The main drawback is that each cell needs to be annotated with the cell magic - there isn't a way (that I know of) to terminate a notebook early without raising an error - which would break the pipeline execution logic.

[Priyanka4pc]

How can this be executed if notebook B and C are supposed to generate output files based on the condition? Is there a way to not get error if specified output files are not generated?

[akchinSTC]

See this related open issue : #408