Is Marimo really reproducible?

[JonasLoos]

Describe the bug

In the project description and docs it's claimed several times that Marimo is reproducible and has "no hidden state".

However, I noticed, that python objects can be changed by repeated and automatic execution, so they could contain content that might not be part of the notebook anymore.

Is this behavior intended or could this be fixed? Did I oversee or misinterpret anything?

Nontheless, great work on this nice project!

Environment

{
  "marimo": "0.2.2",
  "OS": "Linux",
  "OS Version": "5.15.133.1-microsoft-standard-WSL2",
  "Processor": "x86_64",
  "Python Version": "3.10.6",
  "Binaries": {
    "Chrome": "--",
    "Node": "v20.11.0"
  },
  "Requirements": {
    "black": "24.1.1",
    "click": "8.1.7",
    "jedi": "0.19.1",
    "pymdown-extensions": "10.7",
    "starlette": "0.37.0",
    "tomlkit": "0.12.3",
    "typing_extensions": "4.8.0",
    "uvicorn": "0.27.0.post1"
  }
}

Code to reproduce

Example code for repeated cell execution changing the output. First execute all cells and then repeatedly execute cell 2 and see the output of cell 3 change.

import marimo

__generated_with = "0.2.2"
app = marimo.App()


@app.cell
def __():
    # create some object
    mylist = []
    return mylist,


@app.cell
def __():
    # have some value that you change and then rerun this cell
    value = 6
    return value,


@app.cell
def __(mylist, value):
    # modify the object in a cell using that value
    value
    mylist.append(1)
    print(mylist)
    return

Example code for deleted content affecting cell output. First execute all cells and then delete cell 2.

import marimo

__generated_with = "0.2.2"
app = marimo.App()


@app.cell
def __():
    # create some object
    mylist = []
    return mylist,


@app.cell
def __():
    value = "Is this reproducible?"
    return value,


@app.cell
def __(mylist, value):
    # modify the object in a cell using that value
    try:
        mylist.append(value)
    except:
        pass
    print(mylist)

[akshayka]

However, I noticed, that python objects can be changed by repeated and automatic execution, so they could contain content that might not be part of the notebook anymore.

That's correct -- this can happen.

Part of the contract the user must adhere to get reproducibility for free in marimo is to not mutate objects in cells other than the one they are defined in. This is covered in some of our guides/best practices documentation.

Tracking mutations is a huge rabbit hole, and has correctness cliffs as well as performance cliffs that are very difficult to reason about. It would require instrumenting Python code at runtime (performance cliff), and is impossible to get 100% correct (correctness cliffs). For these reasons marimo only uses static analysis in constructing its graph.

marimo's execution order is reproducible in the same sense as Pluto.jl; Pluto.jl also has package reproducibility, which we currently do not have but may in the future. Hope that helps clarify things.

[akshayka]

Converting this to a discussion, in case you'd like to chat more.

[JonasLoos]

Thanks for the clarification and explanation!