Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix/add tmp dir for deadline submission #21

Open
wants to merge 9 commits into
base: develop
Choose a base branch
from

Conversation

moonyuet
Copy link
Member

Changelog Description

Add temp directory for deadline submission so that it would not error out during deadline submission.
Continuity of #17
*Support different system platform

Additional info

n/a

Testing notes:

  1. Launch Blender
  2. Create Render Instance
  3. Publish
  4. If validate render output for deadline errors out, make sure you perform repair action
  5. Publish
  6. It should be rendered successfully.

@@ -347,8 +347,9 @@ def prepare_rendering(asset_group):

# Clear the render filepath, so that the output is handled only by the
# output node in the compositor.
bpy.context.scene.render.filepath = ""

bpy.context.scene.render.filepath = (
Copy link
Member

@kalisp kalisp Aug 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This BigRoy question is still valid #17 (comment)
(eg. why the "\")

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, the empty path for bpy.context.scene.render.filepath is working for some cases. So guess we need to investigate deadline submission issue(e.g. some permission denied eror etc.)
I get inspiration from this website: https://blender.stackexchange.com/questions/119851/render-stops-after-1-frame-render-error-permission-denied-cannot-save-0001

@64qam 64qam added the type: bug Something isn't working label Oct 15, 2024
@moonyuet
Copy link
Member Author

@BigRoy, we gonna dismiss the use of this output setting as we discussed in the meeting, so are we gonna close this PR or?

@BigRoy
Copy link
Contributor

BigRoy commented Dec 10, 2024

@BigRoy, we gonna dismiss the use of this output setting as we discussed in the meeting, so are we gonna close this PR or?

I'd say that setting it to tmp at least isn't the correct fix. We will need to find better ways. Unfortunately it seems that Blender ALWAYS renders to the Output destination set there - and when e.g. /tmp\ is set it just means that on Windows it renders to C:/tmp (not that this is NOT %TEMP%). Preferably we're able to completely disable the output and just rely on the compositor outputs instead - but I haven't found a way in Blender to do so.

So, the only remaining alternative is putting the files somewhere that makes sense in production. Like, somehow enforce it to the correct temp dir locally like %TEMP% or store them inside the project somewhere as {work_dir}/renders/tmp or alike.

@moonyuet moonyuet requested review from kalisp and BigRoy December 11, 2024 08:09
Copy link
Member

@LiborBatek LiborBatek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It correctly kicks in the validator when output dir left blank (empty) as seen below:

Screenshot 2024-12-11 111851

but once hitting Repair it triggers error and do not perform repair action:

Traceback (most recent call last):
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\dependency_packages\ayon_2406251801_windows.zip\dependencies\pyblish\plugin.py", line 528, in __explicit_process
    runner(*args)
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\core_1.0.11\ayon_core\pipeline\publish\publish_plugins.py", line 350, in process
    plugin.repair(instance)
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\blender_0.2.6+dev\ayon_blender\plugins\publish\validate_deadline_publish.py", line 83, in repair
    prepare_rendering(container)
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\blender_0.2.6+dev\ayon_blender\api\render_lib.py", line 350, in prepare_rendering
    bpy.context.scene.render.filepath = temp_render_dir
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: bpy_struct: item.attr = val: RenderSettings.filepath doesn't support None from string types

@moonyuet moonyuet requested a review from LiborBatek December 11, 2024 10:29
Copy link
Member

@LiborBatek LiborBatek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok so now the Repair action works without any errors!

There is one caveat with temp folder path which had been set like this:

Screenshot 2024-12-11 134242

C:\projects\Blender\Shots\sh010\work\lighting\renders\tmp which leads to a staging renders been in incorrect place sitting next to that tmp folder as follows:

Screenshot 2024-12-11 134618

I guess its because we miss the last slash in the path so it probably should look more like this instead:

C:\projects\Blender\Shots\sh010\work\lighting\renders\tmp\ but thats just me guessing.

Maybe also including host name within the folder path could be preferable so it would end up below lighting\renders\blender\tmp\ again not sure if possible.

@moonyuet moonyuet requested a review from LiborBatek December 11, 2024 13:40
Copy link
Member

@LiborBatek LiborBatek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now all behaves correctly!

LGTM

side note: it seems that by filling up the output directory, Blender produces redundant rendered frames in that tmp folder and those arent cleaned up after publish job been finnished. Thats something we should think about in future and how to deal with it... as its a kind of blender specific side product outside AYON templating system...

@moonyuet moonyuet requested a review from LiborBatek December 11, 2024 15:23
@@ -82,5 +82,4 @@ def repair(cls, instance):
container = instance.data["transientData"]["instance_node"]
prepare_rendering(container)
bpy.ops.wm.save_as_mainfile(filepath=bpy.data.filepath)
bpy.context.scene.render.filepath = "/tmp/"
cls.log.debug("Reset the render output folder...")
cls.log.debug("Reset the render output folder...")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logged message is now in a very odd place.

Suggested change
cls.log.debug("Reset the render output folder...")

@BigRoy
Copy link
Contributor

BigRoy commented Dec 11, 2024

Note that the changes in 9b0bba3 do not make it cross-compatible on the farm OR in some case even cross-compatible between machines. It is totally valid to override TMPDIR environment variable per machine if e.g. it has some storage location you want to set it to instead... which may be a location not available on the other workstations running on the farm.

Basically this HARDCODES the path on publish to the temp directory location for that particular machine.

@moonyuet
Copy link
Member Author

Note that the changes in 9b0bba3 do not make it cross-compatible on the farm OR in some case even cross-compatible between machines. It is totally valid to override TMPDIR environment variable per machine if e.g. it has some storage location you want to set it to instead... which may be a location not available on the other workstations running on the farm.

Basically this HARDCODES the path on publish to the temp directory location for that particular machine.

Guess it would be much safer for us to revert the change and store the path into the workdir/renders instead? And we can think about how to clean up that tmp directory lasted in workdir later?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants