Submit along Extra Info with jobs

[BigRoy]

Changelog Description

Submit along Extra Info with jobs.

Additional info

Currently this PR hardcodes some values to pass along in a plug-in but the idea is that this showcases that any plug-in could "add" this data for the job submissions to get them. So that a studio could either implement their own addon to provide the data they want to add or we set-up a plug-in from which you can define some dynamic extra info values from studio/project settings.

Testing notes:

1. Deadline farm submissions should now come with extra job info set.

[MustafaJafar]

The recent PR #49 introduced the following two fields. Is that an alternative for this PR?

[BigRoy]

The recent PR #49 introduced the following two fields. Is that an alternative for this PR?

@MustafaJafar nice. And good question. I suppose it could if the values themselves can be formatted using the context and instance data so that you can pass along additional custom data from the actual instance.

Also, I'm not entirely sure how trivial it is to manage ExtraInfoKeyValue from just those settings while still nicely mixing them with the dynamic nature of setting those during publishing. Because for the submission itself they need to turn into indexed keys, e.g.:

EnvironmentKeyValue0=AYON_APP_NAME=houdini/20-5-370-FX
EnvironmentKeyValue1=AYON_BUNDLE_NAME=2024.11.25-02
EnvironmentKeyValue2=AYON_DEFAULT_SETTINGS_VARIANT=production

Which may be hard to "mix" with dynamic key values defined during publishing.

So I'm not super confident the raw dict nature suits our needs here. But potentially dedicated "Key Value" profiles one could specify with the dynamic formatting for values could very well be how settings for this PR could look.

[MustafaJafar]

suppose it could if the values themselves can be formatted using the context and instance data so that you can pass along additional custom data from the actual instance.

At what point of time of the publishing? I think it won't be an issue since the job info collector works at pyblish.api.CollectorOrder + 0.420

Also, I'm not entirely sure how trivial it is to manage ExtraInfoKeyValue from just those settings while still nicely mixing them with the dynamic nature of setting those during publishing.

so, the order of these extra info keys is important.

For information, this is how adding additional jobinfo is done, which doesn't support formatting each item with info or context data.

ayon-deadline/client/ayon_deadline/plugins/publish/global/collect_jobinfo.py

Lines 75 to 76 in 1df52ae

	for key, value in json.loads(additional_job_info).items():
	setattr(job_info, key, value)

[kalisp]

Calling it directly job_info seems weird, I would prefer job_extra_info which is more descriptive.

[BigRoy]

Totally agree!

[kalisp]

I don't really see purpose of this collector, I can understand reason for Extra Info values though.
But I would prefer them to be implemented as Additional JobInfo data (we could actually change it from json to list of key values tuples).

Or should be ExtraInfo be really dynamic from instance values and it is expected that specific collectors will be created for separate DCCs and collector in this PR is just a "skeleton/example"?

[BigRoy]

Or should be ExtraInfo be really dynamic from instance values and it is expected that specific collectors will be created for separate DCCs and collector in this PR is just a "skeleton/example"?

They do use dynamic values as can be seen in the code. In our case, I'm even collecting what "tools" were active at the running time of the submission. See here.

I created the PR mostly to discuss what make sense as standardization here - and whether it'd even be possible.

For context - this is then what it can look like in Deadline Monitor:

I could actually even filter the Tools to only that application (because now also those listed for Maya are shown for example).

Anyway, having this data there technically also allows me to filter in Deadline to those things.