add DefaultGraphStack

[alexpantyukhin]

No description provided.

[migueldeicaza]

While I do like the principle, you just ran into the core problem that comes with this.

It is still cumbersome to use, because only a handful of operators can be overloaded, but the operations on the TFGraph are still ugly to invoke as an instance is still required.

One possible option (but I am not sure that I want to do that yet) is to create a "Graph" API, and make all the methods in that class static, allowing us to do "using static Tensorflow.Graph" and that could be a place where we could bring something like this.

[alexpantyukhin]

In the python implementation Graph object is created by DefaultStack when there is no instance yet. Maybe we need something like this? Also I didn't see explicit createing Graph object in usage of python impl.

[cesarsouza]

Hi @migueldeicaza,

Just to let you know, I am doing something similar to the option you have suggested within Keras#, if you would like to see how this approach would look.

In most of Keras# I static import a class containing a static property K that allows me to use the graph operations through it. I suppose that in the case of TF#, this could have been the DefaultGraphStack in this PR.

But then for operator overloading to work, I've created a Tensor class that wraps TFOutput and keeps a reference to which graph this TFOutput actually belongs to, and wrote the operator overloading methods such that they would use a graph from whichever argument it would be available.

I am not completely sure if this would work for all scenarios though.

Regards,
Cesar

[migueldeicaza]

I see what you are doing, it is just not clear to me that doing K.Operation is much better than g.Operation -- here I am assuming that the user would have done var g = new TFGraph. Perhaps the benefit is that there is a notion of a global, and users do not need to pass this TFGraph around.

If I end up going this way, it would likely take the form of the generator producing an additional set of methods that are proxies to the existing TFGraph methods.

I like where you are going with KerasSharp on operators. Let me ponder a bit how that would look like.

[cesarsouza]

Hi Miguel,

it is just not clear to me that doing K.Operation is much better than g.Operation

Well, I am not completely sure myself either. I like TFS current approach because it is much more explicit, and you have less to wonder about what is going on under the hood when programming against it. Took me a while to figure out where all the magic from Keras+TF was coming from when I was using it from Python.

In Keras# it makes sense because I am using it to support the different backends (and because I wanted to porsue that idea of having a line-by-line port), but I am not sure it is as important in TFS. Actually, I think that the main issue in TFS is that most users are coming from Python and are looking for something that can be instantly recognizable given their past experience. And for this, I think there is something super simple that could be done: I suppose that users would feel instantly at home if the TFGraph variables were called "tf" in the examples currently in TFS, because they would instantly recognize that all tf.* operations from Python would be available from this scaring TFGraph class they have just read about. And I am supposing here that this would have almost the same benefit as having the notion of a global, since users would see things like tf.MatMul and immediately know how to use them.

Regarding the operators: I have more interesting examples in the learning algorithms if you would like to see how it looks like in practice (I forgot to link in my previous comment). I need to say that it is not completely possible to get rid of the explicit references to TFGraph because sometimes you need to pass more parameters to the operation (i.e. an explicit name).

The main problem with this approach is that I do not really know how feasible it is to make every TFOutput/TFTensor carry a reference to the TFGraph that created them. So far I haven't found any fundamental problem with this, but I guess I am just scratching the surface of TF's APIs at this point.

Regards,
Cesar

[alexpantyukhin]

Should we close this PR?

[migueldeicaza]

I would like to keep this open, as I am still pondering what to do about this.

I do not have a good answer, but I would like to keep this as a reference as I think about this. One option I am considering is introducing a static class "EasyGraph" that would surface a bunch of methods that would allow a scenario like this to work.

Easygraph would have to replicate every bound method in TFGraph.