Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add depth-of-field (Blurring rendering options) #1662

Merged
merged 12 commits into from
Oct 12, 2024

Conversation

perminder-17
Copy link
Contributor

issue : #347
Signed-off by : [email protected]
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

Signed-off-by: Perminder <[email protected]>
Signed-off-by: Perminder <[email protected]>
@perminder-17
Copy link
Contributor Author

perminder-17 commented Apr 11, 2024

Hi @ghutchis code is functioning correctly, but there are a couple of concerns I'd like your help with. Also I am thinking to add visual tests when the all the requirements will be done. Here's the other PR OpenChemistry/avogadroapp#492

  1. The code currently uses 64 samples for blurring, following the number of samples from Speck. However, this reduces frame rates a bit. I'm considering reducing the number of samples, but I'm concerned about compromising the quality of the blur. Do you think decreasing the samples would be advisable?
  2. When we zoom in or out, the blur changes, which differs from the behavior in Speck. While this deviation could mimic real-world effects, such as objects appearing blurry when they're too close or too far. Considering potential future features like VR, this behavior might actually enhance realism. What's your take on this? Should we maintain or adjust the blur behavior?
  3. Your feedback is invaluable, and I'm committed to optimizing and tidying up the code. I'd greatly appreciate your input on any minor cleanups or optimizations you think are necessary.
  4. I was thinking about adding blur and fog effects to a new file, but I believe combining all rendering options into one would be better.
  5. I'll include the ability to adjust the strength of blur, allowing users to control how blurry they want things to be. Will be setting some other uniforms for that.

Screenshot from 2024-04-10 06-10-04

@avo-bot
Copy link

avo-bot commented Apr 11, 2024

This pull request has been mentioned on Avogadro Discussion. There might be relevant details there:

https://discuss.avogadro.cc/t/add-depth-of-field-fog-or-blurring-rendering-options/5332/9

Signed-off-by: Perminder <[email protected]>
@perminder-17
Copy link
Contributor Author

perminder-17 commented Apr 11, 2024

Have a look, In the above commit I changed as you told("does not blur when camera is too close")

BEFORE (without BLUR)

Screenshot from 2024-04-11 21-50-41

AFTER-BLUR (THE INTENSITY CAN BE INCREASED)

Screenshot from 2024-04-11 21-50-28

BEFORE (without blur)

Screenshot from 2024-04-11 22-10-23

AFTER-BLUR

Screenshot from 2024-04-11 22-10-10

Copy link

@vinayakjeet vinayakjeet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of Shader Code Enhancements

The proposed changes to the shader code, including enhanced randomness in the rand function, dynamic sample adjustment, and smoother weight transitions using smoothstep, significantly improve performance and visual quality. These updates are well-aligned with best practices for rendering optimizations.

@ghutchis , could you please review these enhancements and provide your feedback?


float total = 1.0;
vec4 color = texture2D(inRGBTex, texCoord);
// number of samples can be optimized.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

int numSamples = int(clamp(blurAmt * 64.0, 16.0, 64.0));
// dynamically reduce number of samples based on blur amount

for (int i = 0; i < numSamples; i++) {
// Remaining loop code of blur application...
}
@ghutchis your reviews for the changes proposed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the review. I'll definitely make time to address the suggestions. However, regarding this issue, the loop index can't be compared with a non-constant expression. Therefore, adding numSamples to the for loop won't function since its value isn't constant. Could also leads to error. Are you sure about this one??

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, and I appreciate your careful consideration. While GLSL 1.2 does support dynamic looping, the behavior can indeed vary based on the GPU and driver support. Sticking with your current approach to ensure compatibility and stability is wise. Thank you for pointing this out, @guthichs, and your reviews and help are greatly appreciated!

@@ -66,6 +68,47 @@ float lerp(float a, float b, float f)
return a + f * (b - a);
}

float rand(vec2 co){
return fract(sin(dot(co.xy ,vec2(12.9898,78.233))) * 43758.5453);
}
Copy link

@vinayakjeet vinayakjeet Apr 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

float rand(vec2 co) {
return fract(sin(dot(co.xy ,vec2(12.9898,78.233)) + cos(dot(co.xy, vec2(93.9898,50.233)))) * 43758.5453);
}
//more complex pattern produced which helps in reducing the noise. @ghutchis your insights ?

vec2 offset = (vec2(cos(angle), sin(angle)) * radius * 0.05 * blurAmt) / pixelScale;
float z = depthToZ(texture2D(inDepthTex, texCoord + offset).x);
float sampleBlur = calcBlur(z, pixelScale);
float weight = float((z >= origZ) || (sampleBlur >= blurAmt * radius + 0.));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

float weight = 1.0 - smoothstep(0.0, 1.0, abs(z - origZ) / blurAmt);
// using smoothstep to soften weight transitions
This modification uses smoothstep to create a more gradual transition in weights applied to sampled colors, potentially reducing artifacts like banding. Reviews @ghutchis

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the smoothstep() would be nice here, since it's also easier to read.

Signed-off-by: Perminder <[email protected]>
Signed-off-by: Perminder <[email protected]>
@perminder-17
Copy link
Contributor Author

simplescreenrecorder-2024-06-21_01.48.58.mp4
simplescreenrecorder-2024-06-21_02.08.17.mp4
simplescreenrecorder-2024-06-21_01.57.25.1.mp4

Hi, I just have pushed the recent changes, above is the result I uploaded to show how depth-of-field is working for now. If any contributors or users of avogadro software have any suggestion, I would be more than happy to consider those changes into my code. :)

@avo-bot
Copy link

avo-bot commented Sep 8, 2024

This pull request has been mentioned on Avogadro Discussion. There might be relevant details there:

https://discuss.avogadro.cc/t/september-live-updates/5571/1

@ghutchis ghutchis linked an issue Sep 11, 2024 that may be closed by this pull request
Copy link
Contributor

Here are the build results
Avogadro2.AppImage
Win64.exe
Artifacts will only be retained for 90 days.

Copy link
Member

@ghutchis ghutchis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some minor cleanups, but it otherwise looks good.

avogadro/rendering/solid_first_fs.glsl Show resolved Hide resolved
vec2 offset = (vec2(cos(angle), sin(angle)) * radius * 0.05 * blurAmt) / pixelScale;
float z = depthToZ(texture2D(inDepthTex, texCoord + offset).x);
float sampleBlur = calcBlur(z, pixelScale);
float weight = float((z >= origZ) || (sampleBlur >= blurAmt * radius + 0.));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the smoothstep() would be nice here, since it's also easier to read.

avogadro/rendering/solid_first_fs.glsl Outdated Show resolved Hide resolved
avogadro/rendering/solidpipeline.cpp Outdated Show resolved Hide resolved
Copy link
Contributor

Here are the build results
Avogadro2.AppImage
Win64.exe
macOS.dmg
Artifacts will only be retained for 90 days.

@perminder-17
Copy link
Contributor Author

I separated the fog shader into a different file to make the process faster because the Ambient Occlusion (AO) operation was running a bit slow. Thanks to @ghutchis for the helpful insights!

simplescreenrecorder-2024-09-26_04.27.26.mp4

Signed-off-by: Perminder Singh <[email protected]>
@ghutchis
Copy link
Member

Sorry, I guess I wasn't clear. I think the fog should be in both shaders:

  • AO + DOF + fog (i.e., good GPU shader)
  • just fog (i.e., faster rendering)

In other words, if AO or DOF or Edge detection is enabled, the "full" shader is used. If all three are off, then the faster fog-only shader is used for speed.

Does that make sense?

So my suggestion is to put back the fog into the main shader. That's the default. Then we can change which shader is used in the C++, e.g.

if (ao_enabled || dof_enabled || edge_enabled)
   // use full shader
else
   // fast fog-only shader

@perminder-17
Copy link
Contributor Author

Sorry, I guess I wasn't clear. I think the fog should be in both shaders:

Ahh..I am sorry. I thought we need to separate the fog shaders, Sorry for the misunderstanding. I will fix that now.

@perminder-17
Copy link
Contributor Author

Okay, I think I have worked on the suggestion @ghutchis . Let me know if you have any other suggestions?

Copy link
Contributor

Here are the build results
Avogadro2.AppImage
Win64.exe
Artifacts will only be retained for 90 days.

Copy link
Contributor

Here are the build results
Avogadro2.AppImage
Win64.exe
macOS.dmg
Artifacts will only be retained for 90 days.

@ghutchis
Copy link
Member

I think the effect may need some tweaking (e.g., strength, position) but the code looks good. Thanks for all the work!

@ghutchis ghutchis merged commit 818c059 into OpenChemistry:master Oct 12, 2024
19 of 21 checks passed
Copy link
Contributor

Here are the build results
Avogadro2.AppImage
macOS.dmg
Win64.exe
Artifacts will only be retained for 90 days.

@perminder-17
Copy link
Contributor Author

perminder-17 commented Oct 12, 2024

I think the effect may need some tweaking (e.g., strength, position) but the code looks good. Thanks for all the work!

Yes, it could be improved. The problem was when you zoom-out zoom-in the change in the projection matrix is not consistent or linear. That's what makes it a problem. :"). I could probably open a new PR in improving it shortly. Also, I think we could probably merge this PR OpenChemistry/avogadroapp#492

@ghutchis
Copy link
Member

It's a great effect. Exactly as you say, it's the change when zooming. Tweaking the strength and projection matrix are secondary now that the code is already merged.

I needed to confirm that OpenChemistry/avogadroapp#492 built cleanly after the merge. I'll also tweak the text a bit in the labels for avogadroapp.

@perminder-17 perminder-17 deleted the depth-blur branch October 12, 2024 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add depth-of-field fog or blurring rendering options
4 participants