Fusion: Work with pathmaps

[EmberLightVFX]

Changelog Description

Path maps are a big part of our Fusion workflow. We map the project folder to a path map within Fusion so all loaders and savers point to the path map variable. This way any computer on any OS can open any comp no matter where the project folder is located.

Additional info

In OpenPype's default Fusion render location is within the comps root folder. This means every saver node created will get the path Comp: and the path will continue from there. This is universal for Fusion and will render on any Fusion renderer.
For any other kind of path map the users have to manually enter the path map in Fusions settings for anything different to happen than before this PR.

This feature would fit perfectly with an updated Fusion Profile functionality where the studio could enter in custom path maps per project and they would be added automatically to Fusions prefs when opened through OP.
This would make the whole system super clean and easy to use.

Testing notes:

1. Add a path map in Fusions settings that point towards your project folder. Name the path map your project name
2. Create a new saver
3. Check the path and you'll see the start of the path is now Comp:
4. Render and publish the footage
5. Load the footage back into the comp
6. Check the path of the loader and you'll see the path now starts with what you named your path map in step 1.

[iLLiCiTiT]

I'm quite disapointed about the formatting changes. Like PR is changing formatting of files based on preferences of the developer, so the PR has many lines of changes that are totally irrelevant to the PR. And they will be changed back in a future PRs, because preferences of other developer will be different...

I'm not saying I don't agree with them, but please, at least do them in different PRs as it's uber confusing to see 50+lines changes when only 7 of them are functional changes.

[EmberLightVFX]

@iLLiCiTiT Thanks for the note. I'm running black to fix most problems Hound gave me before. It also do some reformatting.
I would love instruction on what formatter and rules to use for future PRs so I won't push unneeded changes.

[jakubjezek001]

Altought I agree with @iLLiCiTiT (And I am the one who would change it to my preferences, which were the original, since they are easier to read from my perspective ;), this PR works as advertised and can be merged.

[BigRoy]

I'm not too sure about this PR since it seems to be doing the opposite to what some things in OpenPype's Houdini integration tries to explicitly remove from the workfile. For some context, Houdini has a $HIP variable referring to the current workfile's folder. Houdini artists frequently load it to load relative files to the workfile, it's a common technique.

However, on publishing workfiles (and for deadline render submissions) and loading publishes this is explicitly avoided because it will break things in certain occassions:

1. Deadline often has a feature for hosts where it can render the scene locally (copies scene locally, then renders). If the scene uses these 'relative path mapping' features then that local scene would not be rendering at the intended location or not loading files from their intended location.
2. If the workfile was to be saved elsewhere the files that were originally a part of the scene now won't load.

With that said, I do think there are valid use cases for using Comp: or other path mapping relative vars - I'm just thinking that loading published content isn't one of them - unless it's always mapped to a project root. For the Savers I can definitely see a use case because saving your scene elsewhere likely means you also want to render elsewhere.

So I'd like to take this moment to discuss the approach we want to take from Ynput/AYON's development standpoint. @mkolar @antirotor @jakubjezek001 how would you like to proceed with this.

---

In essence what the PR tries to do is to create saver paths and resolve loaded paths relative to Comp: or relative to another var path mapping var. As far as I know there's no way to know what Path map vars it might expand too, but say someone would have set up a Fusion Path Map var to a 'project root' or whatever then likely it might make paths relative to that. (I'm not entirely sure, but likely @EmberLightVFX can give some more examples).

Say I have path maps:

• Comp: comp workfile folder
• Project: custom defined path map to root of project
• Resources: custom defined path map to a special folder in project, e.g. {project}/resources
• Stock: a server wide folder contain stock footage

And I have paths like:

/project/asset/work/fusion/comp_v001.comp  # workfile
---
/project/asset/work/fusion/footage/lens_dirt.png
/project/random/folder/art.png
/project/resources/logos/client_logo.png
/stock/explosions/explosion_v001.mov

Then I assume it path maps as:

Comp:footage/lens_dirt.png
Project:random/folder/art.png
Resources:logos/client_logo.png
Stock:explosions/explosion_v001.mov

[EmberLightVFX]

@BigRoy
I haven't used Deadline myself so I can't say how this will work with it, but say that I'm using fuses in my comp. The fuses are mapped by fusions path maps. How does Deadline know what these path maps are? Maybe there will need to be an extra PR to pass path maps to deadline?

Comp: will exist with or without this PR. If you open the path in a saver and re-select the sequence to render the saver will automatically convert the path to Comp:. It's not automatic like after this PR but it could currently happen by accident and break stuff.

As far as I know there's no way to know what Path map vars it might expand too

Outside of Fusion you would have to parse the fusion config (that OP do create) to get the real path. Inside of Fusion you can use comp.ReverseMapPath() to get the correct path.

Your examples are correct, that's how the loaders/savers paths would look like with this PR applied.

As mentioned in the first post, path maps are a big part of the Fusion workflow to allow files to be located independently on different OSes and still work. But a conversation on what approach we want to take from Ynput/AYON's development standpoint is welcome if needed.

[BigRoy]

I tried quickly setting custom path maps within Fusion but it doesn't seem to allow it for me within Fusion. Likely because the Master Prefs override it and they are 'locked'.

Could you explain how you are setting e.g a project-specific path map. Did you change the masterprefs? If so, what did you change them to? How could I e.g. have OpenPype set up a project path map that maps to the project's root?

[EmberLightVFX]

I tried quickly setting custom path maps within Fusion but it doesn't seem to allow it for me within Fusion. Likely because the Master Prefs override it and they are 'locked'.

Could you explain how you are setting e.g a project-specific path map. Did you change the masterprefs? If so, what did you change them to? How could I e.g. have OpenPype set up a project path map that maps to the project's root?

Currently I'm opening Fusion outside of OP, set the path maps as needed and then in OP's studio settings I have Copy profile on first launch enabled so the created profile will have the path maps.
It's not a good way to do it but it's the current way to do it.
Adding a feature where you as a studio could have a profile per project will solve this cumbersome way, but that's for another PR.

[BigRoy]

Currently I'm opening Fusion outside of OP, set the path maps as needed and then in OP's studio settings I have Copy profile on first launch enabled so the created profile will have the path maps.
It's not a good way to do it but it's the current way to do it.
Adding a feature where you as a studio could have a profile per project will solve this cumbersome way, but that's for another PR.

Did you set that path map up in a way that it's still dynamic per project, e.g. mapping it to an env var?

[EmberLightVFX]

Did you set that path map up in a way that it's still dynamic per project, e.g. mapping it to an env var?

Yes. This is how an example look like:

After I have launched it through OP with Copy profile on first launch (and a custom profile path to not overwrite other projects profiles) I can remove those paths in the non-OP fusion.

[BigRoy]

Did you set that path map up in a way that it's still dynamic per project, e.g. mapping it to an env var?

Yes. This is how an example look like:

After I have launched it through OP with Copy profile on first launch (and a custom profile path to not overwrite other projects profiles) I can remove those paths in the non-OP fusion.

I'm not seeing any dynamic mapping - you're just explicitly mapping it to the project folder so would likely need to do this each time you switch project? I was thinking maybe you mapped it to something like $(AVALON_ROOT)/$(AVALON_PROJECT). That should be an environment variable notation that Fusion supports - however it'd mean that the relevant project + root paths are available as environment variables. Anyway, thanks for the info.

[EmberLightVFX]

I'm not seeing any dynamic mapping - you're just explicitly mapping it to the project folder so would likely need to do this each time you switch project? I was thinking maybe you mapped it to something like $(AVALON_ROOT)/$(AVALON_PROJECT). That should be an environment variable notation that Fusion supports - however it'd mean that the relevant project + root paths are available as environment variables. Anyway, thanks for the info.

That would be the dream! But I think that would need to be part of another PR to dynamically generate/modify the profile instead of simply copy the local profile.

[BigRoy]

Overall the functionality works - there's only one code change that should be done.

Admittedly setting up the path mapping itself currently is a bit quirky (even though doable for a tech artist). I suppose that could be an issue for another PR because this functionality doesn't seem to break anything.

[BigRoy]

Works for me.

[BigRoy]

Look how neat this PR changes turned out - lovely work!

[Minkiu]

Hven't tested but code looks good.

[iLLiCiTiT]

Looks like evebody is happy. Merging...