Read CSV, output is columns the csv

[kevinlinxc]

Here is what I imagine for a node:
You double click it to choose a .CSV file

The outputs update dynamically to be Pandas columns named using the headers of the CSV file, so I could pipe those columns to new functions.

Does something like this seem possible?

[KennedyRichard]

You double click it to choose a .CSV file

As a start, you can convert pandas.read_csv directly into a node. This way you'll have all the power of pandas.read_csv without any additional effort. Remember, Nodezator can convert any Python callable into a node.

Here's a format I suggested to a user on discord some months ago, and the one I recommend most for this case:

### third-party import
from pandas import read_csv

main_callable = read_csv

third_party_import_text = 'from pandas import read_csv'

def _read_csv(
      filepath_or_buffer: 'filepath' = 'dummy_value',
      **kwargs,
    ):
    pass

signature_callable = _read_csv

This way you can choose the .CSV files in your system by clicking the path preview widget in the filepath_or_buffer parameter of the resulting node and will be able to use any other additional parameter from pandas.read_csv by using keyword arguments (the **kwargs).

Alternativelly, if you think it is more convenient for your use-case, you can expose specific parameters that you think you may want to use more often. Below, in addition to the filepath_or_buffer parameter, note that I also expose the verbose and skip_blank_lines parameters as well.

### third-party import
from pandas import read_csv

main_callable = read_csv

third_party_import_text = 'from pandas import read_csv'

def _read_csv(
      filepath_or_buffer : 'filepath' = 'dummy_filepath',
      *,
      verbose : bool = False,
      skip_blank_lines: bool =True,
      **kwargs,
    ):
    pass

signature_callable = _read_csv

In the future, nodes in Nodezator will probably be able to hide unused sockets like can be done with Blender nodes:

Mode	Appearance
full display
collapsed

Just for completeness, since pandas.read_csv is compatible with inspect.signature (which Nodezator uses internally to retrieve metadata about callables and use the metadata to create nodes), there's still this format below, but I don't think it is convenient because it doesn't allow you to define custom widgets as you desire:

### third-party import
from pandas import read_csv

main_callable = read_csv

third_party_import_text = 'from pandas import read_csv'

All of this is explained here, just in case you want to know more or understand how/why this works.

[KennedyRichard]

As for the second part of your question:

The outputs update dynamically to be Pandas columns named using the headers of the CSV file, so I could pipe those columns to new functions.

Every time your hit F12 your graph will execute, so your read_csv node will automatically load your file into a DataFrame and from there you can manipulate the DataFrame whenever you like, either using default nodes already provided by Nodezator from Python built-ins or standard library callables, or using your own custom additional nodes.

As for displaying the DataFrame resulting from your manipulation, there are number of options each with their own trade-offs, so it is up to you which one fits your needs best.

1. you could simply create a node from the Pandas.DataFrame.to_csv function, pass your DataFrame to it and display the resulting .CSV string with this node to visualize the text (you can copy the node to your node pack and use it as you wish, everything is in the public domain).
2. If you want a more complex visualization, then you'll have to create your own viewer node, as described here and here; it would require a lot more work, but you'd be able to customize it as much as you wanted;
3. Another alternative is to simply use an external app, like your browser. That is, make it so the manipulated DataFrame is saved somewhere, for instance, using a node created from Pandas.DataFrame.to_html, the open your browser to visualize the file, reloading the page manually every time you execute the graph to see the resulting changes.

The usage of the browser suggested above can be further automated by creating a custom node that saves the DataFrame as html in a temporary file (for instance, using the tempfile module) and open it automatically on your browser (using webbrowser.open), deleting the temporary file after the browser opens.

Another alternative that will only be available in the future is another kind of viewer node called persistent node which will be implemented in the future. The main difference from viewer nodes is that the visuals generated from persistent nodes will still be visible in the graph after it executes. It was suggested/discussed here and is already listed among the requested features (#13). This would require a similar amount of work as working with regular viewer nodes though. Either way you'd have to create your own visualization or use one other people created.

---

Approaching the second part of your question from a different perspective, maybe you want to know whether the graph can execute automatically every time you change a value in the graph (like viewer nodes in Blender for instance)?

For now it is not possible, but it has already been suggested (#26) and listed among the requested features (#13).

[kevinlinxc]

Hi @KennedyRichard , thank you for your comprehensive replies. I think I may not have explained what I wanted in the 2nd part properly though.

After having a dataframe, I want the node to have the columns exposed as sockets. However, these columns will not always have the same name.

Imagine if this is my dataframe:

data = {
  "calories": [420, 380, 390],
  "duration": [50, 40, 45]
}

df = pd.DataFrame(data)

Then, I would want my node to have output sockets "calories" and "duration".

But if my data was different, like

data = {
  "a": [420, 380, 390],
  "b": [50, 40, 45]
}

, then I would want to have "a" and "b" be the name of the output sockets. It seems to me like we have to define the output socket names in the function signature, which would be static and not dynamic, so I am wondering if there is a dynamic way to do this. Basically, once I load the csv into a dataframe and hit f12, there would be new socket names.

[kevinlinxc]

Another way of phrasing my question is like this:

"Can I choose the names and quantity of output sockets inside the function itself, rather than in the signature?"

[KennedyRichard]

I regret to say as of now there's yet no way to define the outputs of nodes dynamically.

Such idea sounds very useful and exciting, though.

A similar idea, one of being able to edit the name of an output socket from a node instance in the app was also suggested by a user who also happened to be using pandas.read_csv in a given example (see the item 2 listed in the message linked from discord).

If you think this is essential for your workflow or at least very desirable you can suggest this as a feature by creating a discussion in the Feature Request category or by commenting on the feature request list discussion #13. If you do this, of course I'll help you personally, though it will take some time though, since I have a lot on my plate.

Let me know if you need anything else, and feel free to keep the questions coming. I intend to make the improvements you suggested to the manual (#36) this week.