update docs (#602)

* update docs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update

* update docs

* bump version

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

.pre-commit-config.yaml

@@ -11,6 +11,7 @@ repos:
       - id: check-json
       - id: check-merge-conflict
         args: ['--assume-in-merge']
+        exclude: ^(docs/)
       - id: check-toml
       - id: check-yaml
       - id: debug-statements

docs/source/_api/cli.rst

@@ -1,4 +1,4 @@
-CLI
+cli
 ===
 .. automodule:: zntrack.cli
     :members:

docs/source/_api/core.rst

@@ -1,4 +1,7 @@
-Core
+core
 ====
-.. automodule:: zntrack.core
+.. automodule:: zntrack.core.node
+    :members:
+
+.. automodule:: zntrack.core.nodify
     :members:

docs/source/_api/exceptions.rst

@@ -0,0 +1,5 @@
+exceptions
+==========
+
+.. automodule:: zntrack.exceptions
+    :members:

docs/source/_api/fields.rst

@@ -0,0 +1,14 @@
+Fields
+==========
+
+DVC Fields
+----------
+
+.. automodule:: zntrack.fields.dvc
+    :members:
+
+Zn Fields
+---------
+
+.. automodule:: zntrack.fields.zn
+    :members:

docs/source/_api/index.rst

@@ -7,9 +7,7 @@ API Documentation
     :maxdepth: 2
 
     core
+    project
     cli
-    utils
-    dvc
-    zn
-    meta
-    metadata
+    exceptions
+    fields

docs/source/_api/project.rst

@@ -0,0 +1,5 @@
+Project
+=======
+
+.. automodule:: zntrack.project
+    :members:

docs/source/_get-started/api_classes.rst

@@ -13,12 +13,13 @@ Compared to the function-based approach, the class-based approach has several ad
 - The class state can be serialized more easily.
 - Parameters, inputs, and outputs are handled as class attributes, simplifying the creation of new Nodes.
 
-To define a custom Node, simply inherit from the Node class. Parameters and outputs can be defined in two ways. The ``from zntrack import zn`` module enables seamless integration with your existing workflows, allowing Python objects to be automatically serialized.
+To define a custom Node, simply inherit from the Node class. Parameters and outputs can be defined in two ways.
+The ``from zntrack import zn`` module enables seamless integration with your existing workflows, allowing Python objects to be automatically serialized.
 
 
 ..  code-block:: python
 
-    from zntrack import Node, zn
+    from zntrack import Node, zn, Project
 
     class SumValues(Node):
         """Node to compute the sum of two parameters."""
@@ -32,14 +33,16 @@ To define a custom Node, simply inherit from the Node class. Parameters and outp
             self.result = self.a + self.b
 
     if __name__ == "__main__":
-        SumValues(a=1, b=2).write_graph()
+        with Project() as project:
+            node = SumValues(a=1, b=2)
+        project.run(repro=False)
 
 We define our parameter using ``zn.params()`` and define the respective output using ``zn.outs()``.
 
 Gather results
 --------------
 
-Using ``<Node>.write_graph()`` will add the Node as a stage to the DVC Graph.
+Using ``project.run(repro=False)`` will add the Node as a stage to the DVC Graph.
 To access the results, we need to run the first graph.
 
 
@@ -52,9 +55,15 @@ It is not only possible to access the results, but also the parameters and input
 
 ..  code-block:: python
 
-    sum_values = SumValues.load()
-    print(sum_values.result)  # will print 3
-    print(sum_values.a)  # will print 1
+    node.load()
+    print(node.result)  # will print 3
+    print(node.a)  # will print 1
+
+If you don't have access to the ``node`` object, you can also load the Node based on it's name:
+
+..  code-block:: python
+
+    node = SumValues.from_rev(name="SumValues")
 
 Explanation
 -----------
@@ -63,7 +72,7 @@ The same files as in the previous ``@nodify`` example are created.
 The main difference is the ``outs`` of our Node ``SumValues``.
 Using the `zntrack.zn` module will store results in the Node Working Directory (``nwd``),
 It is typically set as ``nodes/<nodename>``.
-The ``outs.json`` file is used, when calling ``SumValues.load()`` to gather the results.
+The ``outs.json`` file is used, when calling ``node.load()`` to gather the results.
 
 
 ..  code-block:: yaml

docs/source/_get-started/api_functions.rst

@@ -25,7 +25,7 @@ We specify two arguments:
 - ``params``: The parameters of the Node. A dictionary of values used as parameters.
 
 The ``write_text`` function receives these arguments as a ``NodeConfig`` object.
-To put the Node on the graph, we need to run it via ``write_text()``.
+To put the Node on the graph, we need to run it inside the ``zntrack.Project``.
 
 
 ..  caution::
@@ -34,14 +34,7 @@ To put the Node on the graph, we need to run it via ``write_text()``.
 
 Gather results
 --------------
-Running ``write_text()`` alone will not call the function itself.
-Instead, it will create a Node on the DVC graph.
-To access the results, we need to run the graph using:
-
-..  code-block:: bash
-
-    dvc repro
-
+To access the results, we need to run the graph using ``project.run()`` or ``dvc repro``.
 This will run the Node and save the results to disk.
 The file ``outs.txt`` is created with the content ``Lorem Ipsum``.
 At this stage, running the graph again using ``dvc repro`` won't have an affect.

docs/source/_get-started/dvc.rst

@@ -6,7 +6,7 @@ Data Version Control
 ====================
 
 Data Version Control (DVC_) is a fundamental building block of the ZnTrack package.
-To learn more about DVC, please refer to the DVC documentation <https://dvc.org/doc>_.
+To learn more about DVC, please refer to the `DVC documentation <https://dvc.org/doc>`_.
 DVC_ is responsible for keeping track of all the files and loading results from the cache if they are already available.
 
 Why not just DVC?

docs/source/_get-started/index.rst

@@ -12,6 +12,7 @@ ZnTrack is designed to build workflows, track parameters and keep track of the a
     installation
     workflows
     dvc
+    project
     api_functions
     api_classes
     making_connections

docs/source/_get-started/making_connections.rst

@@ -12,8 +12,10 @@ We can connect these Nodes as follows:
 
 .. code-block:: python
 
-    generate_data = GenerateData(**kwargs)
-    process_data = ProcessData(data=load_data)
+    with zntrack.Project() as project:
+        generate_data = GenerateData(**kwargs)
+        process_data = ProcessData(data=load_data)
+    project.run()
 
 Now, the ``process_data.data`` attribute will be the loaded instance of ``GenerateData``, when running ``dvc repro``.
 The following connection has been established:
@@ -22,23 +24,15 @@ The following connection has been established:
     :alt: mermaid diagram
 
 In some cases it is useful to connect Node attributes instead of Nodes.
-In the above example the Node ``ProcessData`` has to know the correct attributes of ``GenerateData`` to e.g. access the ``data``.
-Therefore, one can also connect attributes of Nodes.
-This is done by appending ``@`` and the attribute name to the Node.
-With this, any attribute of any Node can be connected.
+This can be achieved in the same way.
 
 .. code-block:: python
 
-    generate_data = GenerateData(**kwargs)
-    process_data = ProcessData(data=generate_data @ "data")
+    with zntrack.Project() as project:
+        generate_data = GenerateData(**kwargs)
+        process_data = ProcessData(data=generate_data.data)
+    project.run()
 
 .. tip::
-    In a Future release of ZnTrack it will be possible to connect Nodes directly inside a Context Manager.
-    The current API will still remain but it can be worth looking for updates of the ZnTrack package.
-    Check out `ZnFlow <https://github.com/zincware/znflow>`_ for a preview and further information.
-
-    .. code-block:: python
-
-        with zntrack.DiGraph() as graph:
-            generate_data = GenerateData(**kwargs)
-            process_data = ProcessData(data=generate_data.data)
+    You can also pass ``list`` or ``dict`` of Nodes or Node attributes to other Nodes.
+    This allows to easily build sophisticated pipelines with ZnTrack and DVC.

docs/source/_get-started/project.rst

@@ -0,0 +1,82 @@
+.. _userdoc-project:
+
+Project
+=======
+You can run ZnTrack nodes in three different ways.
+Let's assume the following two nodes:
+
+..  code-block:: python
+
+    import zntrack
+
+    class AddOne(zntrack.Node):
+        parameter: int = zntrack.zn.params()
+        outputs: int = zntrack.zn.outputs()
+
+        def run(self):
+            self.outputs = self.parameter + 1
+
+    class SubtractOne(zntrack.Node):
+        parameter: int = zntrack.zn.deps()
+        outputs: int = zntrack.zn.outputs()
+
+        def run(self):
+            self.outputs = self.parameter - 1
+
+Direct Use
+----------
+ZnTrack Nodes are designed to be used as regular Python classes.
+This allows for easy debugging and testing.
+Typically, you would use :ref:`target to eager graph` or :ref:`target to dvc graph` for production.
+
+..  code-block:: python
+
+    node1 = AddOne(parameter=1)
+    node1.run()
+
+    node2 = SubtractOne(parameter=node1.outputs)
+    node2.run()
+
+    assert node1.outputs == 2
+    assert node2.outputs == 1
+
+.. _target to eager graph:
+
+Eager graph
+-----------
+You can use ZnTrack without DVC.
+This closely resembles the `ZnFlow <https://github.com/zincware/znflow>`_ package.
+ZnTrack is built on top of it.
+The general API would look as follows:
+
+..  code-block:: python
+
+    with zntrack.Project() as project:
+        node1 = AddOne(parameter=1)
+        node2 = SubtractOne(parameter=node1.outputs)
+
+    project.run(eager=True)
+    assert node1.outputs == 2
+    assert node2.outputs == 1
+
+.. _target to dvc graph:
+
+DVC
+---
+The main purpose of ZnTrack is to use it with DVC.
+If you use ``project.run()`` it will serialize the Nodes inside the project and run them with DVC.
+You can use ``project.run(repro=False)`` to only build the graph and execute it later.
+
+..  code-block:: python
+
+    with zntrack.Project() as project:
+        node1 = AddOne(parameter=1)
+        node2 = SubtractOne(parameter=node1.outputs)
+
+    project.run()
+
+    node1.load()
+    node2.load()
+
+    assert node1.outputs == 2
+    assert node2.outputs == 1

docs/source/index.rst

@@ -34,32 +34,38 @@ Class based Node
 ----------------
 .. code-block:: python
 
-   from zntrack import Node, zn
+   import zntrack
 
-   class AddNumbers(Node):
-      number1 = zn.params()
-      number2 = zn.params()
+   class AddNumbers(zntrack.Node):
+      number1 = zntrack.zn.params()
+      number2 = zntrack.zn.params()
 
-      result = zn.outs()
+      result = zntrack.zn.outs()
 
       def run(self):
          self.result = self.number1 + self.number2
 
-   AddNumbers(number1=10, number2=20).write_graph()
+   with zntrack.Project() as project:
+      node = AddNumbers(number1=10, number2=20)
+
+   project.run()
 
 
 Function based Node
 -------------------
  .. code-block:: python
 
-   from zntrack import nodify, NodeConfig
+   import zntrack
 
-   @nodify(outs="number.txt", params={"number1": 10, "number2": 20})
-   def add_numbers(cfg: NodeConfig):
+   @zntrack.nodify(outs="number.txt", params={"number1": 10, "number2": 20})
+   def add_numbers(cfg: zntrack.NodeConfig):
       with open(cfg.outs) as file:
             file.write(str(cfg.params.number1 + cfg.params.number2))
 
-   add_numbers()
+   with zntrack.Project() as project:
+      node = add_numbers()
+
+   project.run()
 
 
 .. toctree::