Exploring options to update torch dependency in phycv used by ezsynth

[wladradchenko]

Hello,

I've been exploring the ezsynth project and noticed that it relies on the phycv library, which has its Torch dependency pinned to version 1.13. This poses some limitations especially now that Torch version 2.x has been released. Using the latest versions often brings about performance improvements, new features, and bug fixes which can be beneficial for users and developers alike.

I understand that phycv also depends on kornia, which is pinned to Torch 1.13. This creates a cascading dependency limitation.

Given the importance of staying up-to-date and the potential benefits of using the latest version of Torch, I wanted to open a discussion on possible ways to address this:

Understanding the Core Dependency: From what I gathered, a primary reason for using phycv is edge detection. Could you provide some insight into the specific functionalities or features from phycv that are crucial for ezsynth?

Alternative Libraries: There are several other edge detection implementations and libraries that might serve as alternatives or replacements. For instance:

From controlnet_aux, there are detectors like OpenposeDetector, PidiNetDetector, HEDdetector, NormalBaeDetector, and so on. Are there potential opportunities here to replace or augment the functionalities provided by phycv?

[Trentonom0r3]

Hello,

I've been exploring the ezsynth project and noticed that it relies on the phycv library, which has its Torch dependency pinned to version 1.13. This poses some limitations especially now that Torch version 2.x has been released. Using the latest versions often brings about performance improvements, new features, and bug fixes which can be beneficial for users and developers alike.

I understand that phycv also depends on kornia, which is pinned to Torch 1.13. This creates a cascading dependency limitation.

Given the importance of staying up-to-date and the potential benefits of using the latest version of Torch, I wanted to open a discussion on possible ways to address this:

Understanding the Core Dependency: From what I gathered, a primary reason for using phycv is edge detection. Could you provide some insight into the specific functionalities or features from phycv that are crucial for ezsynth?

Alternative Libraries: There are several other edge detection implementations and libraries that might serve as alternatives or replacements. For instance:

From controlnet_aux, there are detectors like OpenposeDetector, PidiNetDetector, HEDdetector, NormalBaeDetector, and so on. Are there potential opportunities here to replace or augment the functionalities provided by phycv?

Sure thing!

I don't necessarily have any particular attachment to phycv -- I come from a mechanical engineering background, so it intrigued me moreso than standard edge-detection libraries, and I found great benefit in output quality using directional edge detection as opposed to gaussian edge detection.

The two crucial features of phycv would be the PAGE and PST edge detection.

As I'm sure you've noticed, edge-detection is contained within the EdgeDetector class, so any further changes should be relatively straightforward.

I like the idea of directional edge detection as it gives ebsynth a bit more to go off of when stylizing, and often times is more robust to detailed changes.

I think there's certainly opportunity to replace or augment these functionalities.

I've been working on optimizing the library for parallel processing recently and managed to get the guide-truth sequence creation down to within ~180 seconds, from ~400+ seconds.

I hadn't even considered the dependency limitations across various libraries, so thank you for bringing this up!
I'm a super novice programmer, so insights like this are incredibly useful and help me to learn and grow!

I guess the main thing regarding edge-detection, is keeping processing time sub 10s, and possibly continuing use of some sort of directional edge detection would be beneficial. Getting the edge- to near real-time would be preferred though.