Confusion on installing node packages

[arthexis]

Hello, I was surprised that no default nodes were included in the default install:

The documentation only says this about how to install the node packages, but it isn't clear enough exactly what I should do to install them, and this may confuse other users:

Thanks, I love the idea behind this project!

[KennedyRichard]

Mr. Guillen,

Thank you for taking the time to give us feedback regarding Nodezator's shortcomings, it is much appreciated.

About your difficulty in finding info on how to load nodes

I'm sorry that you had to experience such inconvenience.

Your experience made it clear that we need to make some crucial resources more easy to find. For instance, we have a section in the online manual dedicated to explaining how to load nodes, and another one explaining how nodes are defined and organized into packs so they can be loaded, but it is my fault that none of those resources were shown in the form to load nodes inside the app, nor were they linked in the section of the README file that you showed us.

As you pointed out, this ends up demanding extra effort to look for the information somewhere else. The entire online manual, including the chapters linked are available in-app as well, in the menubar "Help" > "Read manual", so there's even less excuses not to make this information more discoverable.

I'll make sure such resources are very easy to find, both in the form inside the app used to load nodes and in the README file. It should not take much time, but It'll have to wait a few days or a week until I can squeeze it in between my current tasks. I'll get back to you on that when I'm finished.

About Nodezator not including node packs

Nodezator indeed doesn't include node packs in its installation. However, it comes with a lot of nodes by default already, as one can see by creating a new file and right-clicking the canvas (a popup menu appears showing a lot of different nodes that can be used).

When I created Nodezator, I wanted it to be an app that didn't force people to use specific nodes. Instead, I wanted to give people maximum freedom to create their own nodes and only load the nodes (packs) they wanted for each file.

While this gives people a lot of freedom, I do understand that this may cause Nodezator to appear a bit bare at first.

However, it is not as though this is set in stone. I'm not opposed to include a node pack or two with the default installation. Even so, wanting to include them presents its own challenges, all related to creating and choosing the right node packs for the job:

1. Nodezator was released less than a year ago and multi-purpose node editors aren't so popular with developers yet, so it has a small community (though thankfully it is growing). Many people still hadn't time to develop node packs they are willing to share and some are happy to use them in their own personal workflow without having to worry about publishing and maintaining them.

2. Apps like Nodezator make it easy to define new nodes, but designing them in the first place is the real challenge. That is, designing modular functions that go well together for a particular purpose still takes some time, effort and experimentation with real problems. Even the node packs available for download in https://gallery.nodezator.com, despite their simplicity, took some time and effort to think about real problems, which kind of operations would help solve them, and translating this all into functions to use as nodes; despite being fun work, it still takes time; and even so, those packs are far from complete; they serve as very neat examples and are useful on their own to solve some specific problems for which they were designed, but they are only, as is shown by their names, example packs.

3. Choosing the node packs themselves is also a challenge: Nodezator is a multi-purpose node editor, so any node packs for any field can be created/chosen: image editing, data visualization, machine learning, text processing, you name it; based on what criteria will we choose? I mean, we could vote, but even so, won't we be tempted to add additional packs, and maybe still want to add more in the future?

Please, don't feel discouraged by the challenges I listed, they are only honest concerns, nothing more. Additionally, as I said, I'm not opposed myself to the idea of adding one or more node packs to the default installation. The prospect is quite interesting actually.

My point is just that it is too soon to think of this. I believe it is better to wait a bit more for more people to start using Nodezator, grow used to it and develop their own node packs naturally, and once we have a good number of node packs available we may start to think about picking the most used ones or design our own custom-made packs to include in Nodezator.

If after what I explained you still think we could try something soon, let me know and we can consult with the rest of the community about it, either here on github or on the discord server.

Turning this issue into a discussion

I'll also take the liberty of converting this issue into a discussion, but don't worry about it, it doesn't mean your request is any less important. It is just that in Nodezator and other Indie Python repos we use issues for things that crash the app or otherwise prevent the user from doing something that is supposed to be available. When such an issue appears, any other work is paused and all attention is given to that issue so that it can be fixed ASAP.

This measure is taken for the benefit of the users, including yourself: by doing things this way, whenever you have a real issue, it won't compete for space with other less urgent matters. Suggestions of improvements and feedback about bad experiences like this post of yours are as important as an issue, but they don't have the same urgency, which is why we turn them into discussions.

---

Thank you again, Mr. Guillen, for taking the time to give us feedback. Sorry for any inconvenience you may have had while using Nodezator. My goal is to work as hard as healthily possible to ensure Nodezator meets the needs of our users.

Let me know if you need anything else.

Peace.

[KennedyRichard]

Last but not least, thank you for your kind words in the top message:

Thanks, I love the idea behind this project!

[arthexis]

Hi Kennedy,

Thanks for your response, I responded via email with some additional thoughts, but I had an observation as I was going over the docs you shared:

I think this could be made more clear if there was a concrete example of the pip command, since I found it hard to determine what are the correct package names of the examples in the gallery. You could make the gallery more useful by showing the explicit command needed to install each one, with a copy to clipboard link.

Hope the suggestions help. Thanks a lot for the attention!

[KennedyRichard]

Thank you again for yet another pertinent feedback.

You are absolutely right. The text in question is to keen to explain what must be done but lacks when it comes to demonstrating it directly. Yet another example where the user has to look further for additional info when that info should be easily available where it is need.

I'll make sure to fix this as well when I tackle the other problems mentioned before.

About your email, could you please confirm whether you sent it to the right address ([email protected]) and perhaps send it again? I just checked my email and unfortunately there's no message from you there.

Peace.

[KennedyRichard]

Just stopping by to say I already started to make the improvements. They should be ready and pushed by Thursday or even earlier.

Thank you for your patience.

Peace.

[arthexis]

Thanks, I was just thinking about this about 20 minutes ago, believe it or not. I had this idea: how hard would you think it'd be to implement something that notifies the user if a connection they are making is somehow illegal (ie, like if you get a mypy typing warning) ?

…

On Sat, Apr 8, 2023 at 1:49 PM Kennedy Richard ***@***.***> wrote: Just stopping by to say I already started to make the improvements. They should be ready and pushed by Thursday or even earlier. Thank you for your patience. Peace. — Reply to this email directly, view it on GitHub <#45 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAF6RFKTGEHBWQ2MXWO6UQTXAG6MBANCNFSM6AAAAAAWJ6MQK4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[KennedyRichard]

Hello, I'm off from any kind of work on Sundays, but I'm here to acknowledge receipt of your message. I'll answer asap, ETA for reply is a day or two.

Have a nice weekend!

[KennedyRichard]

Mr. Guillen,

Thank you for your patience, here's my reply.

I had this idea: how hard would you think it'd be to implement something
that notifies the user if a connection they are making is somehow illegal
(ie, like if you get a mypy typing warning) ?

The core of the feature would be actually very easy/straightforward to implement.

Whenever a user connects a pair of sockets, before the connection is actually established, Nodezator already triggers a series of checks to determine whether the connection is allowed or not. For instance, if you try to connect an input socket to another input socket from another node, the connection never happens.

Doing what you described is just a matter of adding another check. Something like "only allow these 02 sockets to connect if their annotations are compatible" or something like that.

However, the whole feature would require 02 extra factors to be considered (and extra measures to be taken).

First, since by default Python itself doesn't do that, I don't think it is appropriate for this to be a default behaviour for Nodezator either. Rather, I think this would be better as a preference that users can turn on or off, either in general on the user preferences or for specific .ndz files. For instance, you might have a .ndz file were you work with your own nodes containing carefully typed parameters/outputs where you want to feature to be on and then you might have to load another .ndz file to work with nodes from someone else (probably for a different purpose) and that someone didn't care to properly type their parameters, so maybe (I'm only speculating), leaving the feature on would causing some connections to be unintentionally prohibited.

Second, when we implement this feature, we need to also implement some kind of logic to deal with the existing .ndz files in the wild that already contains connections that the feature would consider prohibited/invalid? We could just make it so such kind of connections are automatically undone when the feature is on. However, the user might want a different behaviour in some cases, for instance, that instead of undoing the connections, the user might want to change the annotation on the parameter or the output or both (imagine a .ndz file with a lot of nodes and their connections). Because of this, I also think it would be more appropriate to add this feature only after the "conflict resolution mode" feature is added. This feature is one we already plan to implement to solve conflicts that appear in the .ndz file as result of changes introduced by the user. This conflict resolution mode feature is already listed in the list of planned/requested features (#13) and is also a bit discussed in the discussion #42.

Of course, none of what I said is final/set in stone. It needs further discussion and adjustments when it is time to implement it.

Nonetheless, everything considered, and even despite the extra work needed, I think your idea qualifies as a neat feature. Do you want me to add it to the list of planned/requested features? Please, let me know. If you do, your idea have my approval and I'll add it to the list right away.

Of course, in the future, when it is time to implement it, all features are subject to discussion and approval by the community, but that's a matter for the future, and, to be honest, I think your idea consists of a fairly desirable behavior that most if not all will approve.

Peace

[OlegAlexander]

Hi @KennedyRichard and @arthexis, I'm a huge believer in the benefits of static typing, and I'm in favor of having the option to enable static typing of connections in Nodezator. All node editors I know take this approach. However, I agree with @KennedyRichard that because we're dealing with Python, this shouldn't be the default. I also think that beyond checking connections for basic types like int and str, it may be more difficult to check connections for generic types like List[T] or Callable[[Optional[str]], bool]. Of course, I could be wrong--maybe it's relatively easy to check if the types of the connections match, no matter the complexity of the function type signature. If "static typing" mode is enabled, and the user imports nodes that don't have function type signatures, then I think the connection should be allowed, and the types should be assumed to be Any.

[KennedyRichard]

Hello, thank you for you feedback, Mr. Alexander.

The inspect module actually provides a class to parameters and return value that are not annotated (inspect._empty), but your suggestion of using Any seems to be the way to go and the current practice, so it will probably be our choice.

I still have much to think about the feature and also plan to research the topic to become acquainted with it and be able to work on it.

In principle, I'm thinking of having two modes: strict and loose. In strict mode, only exact matches would be allowed. In loose mode, compatible types will also be able to connect (for instance, a connection between int and numbers.Real would be allowed, since int is a subclass). However, isn't typing everywhere else actually implemented using this more loose behaviour already? Maybe there shouldn't be a strict mode after all?

Another thing we'll have to discuss/decide upon: what to do about illegal connections? That is, should they be impossible to make or should Nodezator just issue a warning when they are made? Should we add an additional configuration for this, so users can choose? I'm actually inclined to have the additional configuration, so users have the final say about which behaviour they desire.

Moreover, it would not be right/ethical for someone like me without any background/research/practice in typing to be making any propositions on my own regarding the feature. Because of this, in addition to feedback from users (specially those familiar with typing), I'll also be requesting input from people with research and/or practice in typing, specially within the Python ecosystem. I actually have a couple of names in mind already, but any suggestions on this are welcome and appreciated as well.

Peace.

[arthexis]

Hi Mr. Richard, Sorry for the late reply, work was quite busy the last few weeks. I am glad you would consider this a good feature, indeed I would love for it to be implemented. At the very least you can count me in for help with testing once it comes to life. Thanks for your time and attention! Semper Ascendens

…

On Tue, Apr 11, 2023 at 1:52 PM Kennedy Richard ***@***.***> wrote: Mr. Guillen, Thank you for your patience, here's my reply. I had this idea: how hard would you think it'd be to implement something that notifies the user if a connection they are making is somehow illegal (ie, like if you get a mypy typing warning) ? The core of the feature would be actually very easy/straightforward to implement. Whenever a user connects a pair of sockets, before the connection is actually established, Nodezator already triggers a series of checks to determine whether the connection is allowed or not. For instance, if you try to connect an input socket to another input socket from another node, the connection never happens. Doing what you described is just a matter of adding another check. Something like "only allow these 02 sockets to connect if their annotations are compatible" or something like that. However, the whole feature would require 02 extra factors to be considered (and extra measures to be taken). First, since by default Python itself doesn't do that, I don't think it is appropriate for this to be a default behaviour for Nodezator either. Rather, I think this would be better as a preference that users can turn on or off, either in general on the user preferences or for specific .ndz files. For instance, you might have a .ndz file were you work with your own nodes containing carefully typed parameters/outputs where you want to feature to be on and then you might have to load another .ndz file to work with nodes from someone else (probably for a different purpose) and that someone didn't care to properly type their parameters, so maybe (I'm only speculating), leaving the feature on would causing some connections to be unintentionally prohibited. Second, when we implement this feature, we need to also implement some kind of logic to deal with the existing .ndz files in the wild that already contains connections that the feature would consider prohibited/invalid? We could just make it so such kind of connections are automatically undone when the feature is on. However, the user might want a different behaviour in some cases, for instance, that instead of undoing the connections, the user might want to change the annotation on the parameter or the output or both (imagine a .ndz file with a lot of nodes and their connections). Because of this, I also think it would be more appropriate to add this feature only after the "conflict resolution mode" feature is added. This feature is one we already plan to implement to solve conflicts that appear in the .ndz file as result of changes introduced by the user. This conflict resolution mode feature is already listed in the list of planned/requested features (#13 <#13>) and is also a bit discussed in the discussion #42 <#42>. Of course, none of what I said is final/set in stone. It needs further discussion and adjustments when it is time to implement it. Nonetheless, everything considered, and even despite the extra work needed, I think your idea qualifies as a neat feature. Do you want me to add it to the list of planned/requested features? Please, let me know. If you do, your idea have my approval and I'll add it to the list right away. Of course, in the future, when it is time to implement it, all features are subject to discussion and approval by the community, but that's a matter for the future, and, to be honest, I think your idea consists of a fairly desirable behavior that most if not all will approve. Peace — Reply to this email directly, view it on GitHub <#45 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAF6RFOF4BOSJ6VAJ7NA5RTXAWZBLANCNFSM6AAAAAAWJ6MQK4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[KennedyRichard]

No need to apologize. I encourage people to do things at their own pace always, whenever possible. No judgement.

That's decided, then: I'll list the feature for implementation.

Coincidentally, I was just reading last week about how the Maya app also restricts connections to attributes that have the same type. It seems that, if well designed, a typing system may indeed be invaluable to the strength and maintainability of a system. I've been leisurely studying Rust on my own, more as a way to complement my Python programming than to use it per se. I already planned to learn C or C++ for that purpose but the high praise that Rust has been getting recently added to the complaints about C/C++ made me feel impelled to give Rust a try first.

Rust typing feature inspires more confidence than what I've been seeing in Python, but that is just speculation on my part. I don't know enough about typing neither in Rust nor in Python to provide an accurate assessment. Additionally, to my knowledge, Python typing facilities are still under active development and constant change. Back when I started learning Python in 2010 there was no encouragement to annotate code, even though I believe it was already possible to annotate functions due to PEP 3107. At least the learning resources I used back then, mostly books, didn't even mention the possibility.

I'm eager to learn more about those topics, though, since I want to experience for myself the actual benefits of systems with well-designed typing. I look forward to implementing this feature in Nodezator and the discussions and learning that will precede it.

[KennedyRichard]

In other news...

🥳 🥳 🥳 I finally finished the improvements in the manual and other key spots in Nodezator based on your feedback in this discussion. 🥳 🥳 🥳

Thank you for your patience.

There are still spots that can be improved but I think the major pain points were already taken care of.

That is:

1. the chapter about loading nodes was revised and now have more accurate/direct/clear steps to loading nodes; for instance, the user is presented with an actual command to install an example node pack if desired (pip install mynodepack), in order to try additional nodes; a link to download another example node pack is also provided;
2. the form where nodes are loaded now has a button that redirects the user directly to the aforementioned chapter inside the app itself;
3. the README file on the repo was updated as well to explain better how nodes are created or downloaded and then loaded in the app;

A bit of reading is still required, since the process has some inherent complexity to it, but the user is now better guided through the entire process.

The changes were already pushed to the repository and uploaded to PyPI as the new 1.3.13 version. To access them, just upgrade your Nodezator installation with

pip install --upgrade nodezator

Or download the source again if you prefer to use Nodezator as a standalone app.

The new version also includes some other unrelated changes.

The online version of the manual was already updated as well.

Please, let me know if you have anything to add. Or if there's something you still think must be changed. Otherwise, I'll be closing this discussion following your confirmation.

Peace.

[KennedyRichard]

Since there's nothing else to be added, I'll close this discussion. Thank you again for the feedback and let me know whenever you run into any other issues.

Peace.