Implemented bicubic downscaling

[ruanformigoni]

This PR implements a bicubic filter to remedy issue #692.

Update 10-mar-2023

Nier Automata (2160p)

Current gamescope result:

Proposal:

Need for Speed ProStreet (2160p)

Current gamescope result:

Proposal:

GTA IV (4320p)

Current gamescope result:

Proposal:

Dark Souls II (4320p)

Current gamescope result:

Proposal:

[vic-bay]

Downsampling from 2160p to 1080p looks okay, but everything lower looks weird.
Here is comparison between 1440p and 1080p with FXAA. 1440p looks a bit aliased and pixelated. In my opinion forced FSR downsampling still looks better.

[EndlesslyFlowering]

you should probably look into Catmull-Rom (Bicubic with b=0 and c=0.5, rather than than b=0.33 and c=0.33) for downscaling rather than using normal bicubic. it's sharper and preserves edges the same as the default.

[rKsanu2MMYvypWePtQWM]

I can confirm that resolutions that aren't double the monitor resolution look weird.

[ruanformigoni]

Hey @EndlesslyFlowering , I've included the Catmull-Rom algorithm, thanks for bringing it to my attention. Thanks for the heads up @rKsanu2MMYvypWePtQWM, I've fixed this in the latest commits.

Here's the ranges of are shown below, the numbers represent the resolution multiplier from the monitor resolution and the render resolution, e.g., render at 2160p and display in 1080p == 2160/1080 == 2.

• Range of [1,2) uses FSR Bilinear
• Range of [2,3) uses Catmull-Rom with a mean of 4 samples
• Range of [3,4) uses Catmull-Rom with a mean of 9 samples
• Range of (4,inf) uses Catmull-Rom with a mean of 16 samples

You can toggle the downscaling effect with super + K

[DisplayTalk]

Definitely making really great progress. I think to cap it all off you should add the ability for user to adjust the bicubic parameters of B and C. Catmull-rom has ringing, which increases perceived sharpness but I want to be able to adjust it.

Examples of different bicubic setups with the 2 parameters:

B-Spline(blur only): B1 C0

Hermite(blocking only): B0 C0

Mitchell(balanced minor amounts of blocking, aliasing, and ringing): B0.33 C0.33

Catmull-Rom(balance of ringing and blocking): B0 C0.5

Explanation of different scalers:
https://artoriuz.github.io/blog/mpv_upscaling.html
https://en.wikipedia.org/wiki/Mitchell%E2%80%93Netravali_filters

Image showing the effects of different bicubic parameters on scaled quality:
https://artoriuz.github.io/blog/images/mpv_upscaling/mitchell_survey.png

Also, if it's not too far out of scope, these scalers(especially the ringy ones like catmull-rom or lanzcos) work better when you convert gamma to sigmoid light before scaling, and back to the native gamma afterwards.

Pic showing that with upscaling:
https://artoriuz.github.io/blog/images/mpv_upscaling/kanao/kanao_comparison.png

This applies also to the downscaling(your PR can be used as both an upscaler or downscaler, if it still works like the first iteration)

[ruanformigoni]

@Displaysguy thanks a ton for the explanation and resources! I'll study how to implement this properly and look into the possible method for argument passing you mentioned here.

[DisplayTalk]

Another tidbit, this one is specifically on sigmoidization:

"You may decrease halos and increase perceptual sharpness by increasing the sigmoidal contrast (up to 11.5, say). Higher contrasts are especially recommended with greyscale images (even “false RGB greyscale” that have three proportional color channels). The downside of sigmoidization is that it sometimes produces “color bleed” artefacts that look a bit like cheap flexographic (”gummidruck”) printing or chromatic aberration. In addition, sigmoidization’s “fattening” of extreme light and dark values may not work for your image content. If such artefacts are obvious, push the contrast value down from 7.5 (to 5, for example, or even lower). Setting the contrast to 0 is equivalent to enlarging through linear RGB. (Robidoux, 2012)"

From this(and the MPV scaling article) we know that sigmoidization can lower both light and dark overshoot(overshoot=ringing) with ringy scalers on both upscaling and downscaling. It's always superior to linearization because linearization only accounts for light overshoot and not dark overshoot.

However, it is not without downside. It has negative effects on the image and in certain situations(even where using it makes sense, which is only when using ringy scaling configurations) might not even be a net benefit. Because of this I wouldn't make adding sigmoidization a priority, and if you do add it make sure it is user-adustable and can be disabled(and is not enabled by default).

https://guide.encode.moe/encoding/resampling.html (ctrl-f sigmoid to find the quote)

[specfreq]

I've had a chance to test this out a bit. Here's some test screenshots.

[ruanformigoni]

Hi @specfreq , could you also include a screenshot with the downsampling algorithm disabled?

[specfreq]

@ruanformigoni Here's some new test screenshots.
One thing I'm doing is changing the desktop resolution to match my target scaled resolution (1024x768 in this case) and forcing on vsync through the OS.

Launch options: gamescope -D -f -W 1024 -H 768 -w 4096 -h 3072 -- %command%

Let me know if I don't have it set up correctly.

[ruanformigoni]

Thanks for the followup @specfreq. I did the same as you (change the desktop resolution and downscale) and got way better results, here is a comparison. Did you set the in-game render resolution to 4096x3072 in the resolution settings?

[specfreq]

The resolution is set to 4096x3072

Here's my git pull:

[ruanformigoni]

Are you able to toggle it with super + k?

[specfreq]

I am not able to toggle it with super + k. I'm pretty new to using Git, should I just delete it and reinstall?

[ruanformigoni]

That should be fine, make sure that lines 176-179 of the file src/sdlwindow.cpp are these:

case KEY_K:
	g_wantedUpscaleFilter = (g_wantedUpscaleFilter == GamescopeUpscaleFilter::BICUBIC) ?
		GamescopeUpscaleFilter::LINEAR : GamescopeUpscaleFilter::BICUBIC;
	break;

In your launch command gamescope -D -f -W 1024 -H 768 -w 4096 -h 3072 -- %command%, do could you point it directly to where it is compiled? E.g.: ~/Repositories/gamescope/build/gamescope -D -f -W 1024 -H 768 -w 4096 -h 3072 -- %command%.

Edit: You can also try with other keys, maybe change KEY_K to KEY_P , recompile, and try with super + p. By your screenshots the bicubic filter is not enabled.

[specfreq]

I just rebuilt it and it is working now.

[specfreq]

Here's a test video:
https://youtu.be/mbMsGK45p-U

And high quality:
https://youtu.be/6RVYbmluInk

I'd love to be able to use this with the integer scaling.

[Filoppi]

I'm not too sure about it but, this branch seems unreachable?
You pass in f (x) as a fract of a pixel coordinates, thus it will never be beyond 0-1?

The original implementation, which I reimplemented here
https://github.com/dolphin-emu/dolphin/pull/11999/files#diff-368d3b7ce4f1890976b9f3b3d7f59df06cc906721d2dd882552a8503b3a40d7bR240
actually passed in fract*m (or n) here, effectively using all branches as it spanned from -1 to 2.
Thought to be honest I haven't 100% understood what the algorithm does so maybe my implementation is all wrong.

[ruanformigoni]

Thanks for spotting this, I've updated the code and now it calculates the weights as described here.

[rKsanu2MMYvypWePtQWM]

any news? would love to see this finally merged...

[Filoppi]

I highly suggest to take a look at this more advanced downsampling method:
dolphin-emu/dolphin@ca93a51
It's heavier, but it looks damn fantastic when scaling from a relatively higher resolution. On older games, this would be perfect and very temporally stable.

[misyltoad]

I don't understand why you are still running the whole RCAS pass, surely we don't want sharpening after doing downscaling?

What's the rationale behind doing FSR for bilinear at certain ranges? Does it look better? I have my doubts there.

I think it also makes sense to have a g_wantedDownscaleFilter and g_downscaleFilter. Shouldn't affect upscaling at all. I would be happy to default downscaling method.

Then do like:

				bool needsUpScaling = frameInfo.layers[0].scale.x < 0.999f && frameInfo.layers[0].scale.y < 0.999f;
                bool needsDownScaling = frameInfo.layers[0].scale.x > 1.001f && frameInfo.layers[0].scale.y > 1.001f;
                frameInfo.useBICUBICLayer0 = g_downscaleFilter == GamescopeDownscaleFilter::BICUBIC && needsDownScaling;
				frameInfo.useFSRLayer0 = g_upscaleFilter == GamescopeUpscaleFilter::FSR && needsUpScaling;
				frameInfo.useNISLayer0 = g_upscaleFilter == GamescopeUpscaleFilter::NIS && needsUpScaling;

[ruanformigoni]

Ok, I managed to fix the black screen issue @misyltoad, and also rebased the project to have no conflicts with master. I have modified how the arguments work to fit more what gamescope is doing right now with the --filter argument; so the usage now is --filter=bicubic or -F bicubic, with arguments it looks like --filter=bicubic:0.3,0.3 or -F bicubic:0.3,0.3. My previous comment was updated with the full command line examples.

[matte-schwartz]

@ruanformigoni nice work on this!

something I've found while testing:

• I'm getting quite a few [gamescope] [Error] xdg_backend: Compositor released us but we were not acquired. Oh no. logs when using gamescope -F bicubic -W 2560 -H 1080 -w 5120 -h 2160 -- vkcube on Plasma 6 Wayland with the gamescope Wayland backend. this does not happen with the upscaling filters in gamescope, and it also does not seem to matter if Gamescope WSI is in use or not. I'm not seeing any obvious causes with gamescopectl log_xdg_backend debug but I haven't looked too far into it yet.

I will try to test some with the DRM backend sometime later today.

[ruanformigoni]

Hi @matte-schwartz , thanks! I do not think this is related to the filter, since I also have this issue with the current implementation of FSR:

[matte-schwartz]

Interesting, was not seeing the same on my rig with FSR although I was not testing in-game quite yet... just with vkcube. Could have just been a desktop compositor issue or something on my end I suppose. In general it's not a huge deal when that error appears but figured I'd make a note of it anyway.

[gnusenpai]

I've found a strange bug. Enabling the bicubic filter destroys the gamma in some games, lowering contrast and making the colors washed out. It seems to affect all the Unreal Engine games I've tried.

EDIT: It appears that this is related to 10-bit formats. Disabling Gamescope's WSI layer restores the gamma, but then you don't have 10-bit color of course.

[arch1t3cht]

Image showing the effects of different bicubic parameters on scaled quality:
https://artoriuz.github.io/blog/images/mpv_upscaling/mitchell_survey.png

(The image link is dead but it's still available in the wayback machine, or e.g. at https://upload.wikimedia.org/wikipedia/commons/thumb/6/6f/Mitchell-Netravali_artifacts.svg/1280px-Mitchell-Netravali_artifacts.svg.png )

Note that this graph (and the survey it's based on) is for upscaling. When downscaling, the parameters can have very different effects (depending on how the kernel is scaled with the downscaling ratio).

[Artoriuz]

Image showing the effects of different bicubic parameters on scaled quality: https://artoriuz.github.io/blog/images/mpv_upscaling/mitchell_survey.png

I realise I'm replying to a very old message, but I just wanted to point out that the information previously contained in the linked page has been somewhat moved to this other page. The content itself is different, but it covers the same topics.

[LethalManBoob]

lets get this merged asap pls, linux gamers deserve hq super sampling :<

[MCPO-Spartan-117]

The help output needs to be updated, there isn't a Wayland keybind for this and Nvidia seems to have issues with this for whatever reason as @gnusenpai said.

[Sp3ci3s8472]

I build the project and installed the binaries to test it on my 7900XTX, sadly it does not seem to run. The version I had before from my package manager (pacman) does seem to work (gamescope-3.15.14-1-x86_64). Not sure what additional details you might need but I still have X11 instead of Wayland.
The command I used is:
gamescope -h 2160 -w 5160 -H 1440 -W 3440 -r 120 -- %command%

When I run it on my terminal without any extra parameters:
[gamescope] [Info] console: gamescope version 6649fd8 (gcc 14.2.1) No CAP_SYS_NICE, falling back to regular-priority compute and threads. Performance will be affected. [gamescope] [Info] vulkan: selecting physical device 'AMD Radeon RX 7900 XTX (RADV NAVI31)': queue family 1 (general queue family 0) [gamescope] [Info] vulkan: physical device supports DRM format modifiers [gamescope] [Info] wlserver: [backend/headless/backend.c:67] Creating headless backend [gamescope] [Info] vulkan: supported DRM formats for sampling usage: [gamescope] [Info] vulkan: Creating Gamescope nested swapchain with format 44 and colorspace 0 gamescope: types/wlr_linux_dmabuf_v1.c:532: feedback_compile: Assertion table_len > 0' failed.
Aborted (core dumped)
`

[matte-schwartz]

@Sp3ci3s8472 that's unrelated, you need #1548

[Sp3ci3s8472]

@Sp3ci3s8472 that's unrelated, you need #1548

Thanks that indeed solved it!

[doonv]

@Sp3ci3s8472 that's unrelated, you need #1548

How do I apply that PR's changes when building this one with docker?

[doonv]

How do I apply that PR's changes when building this one with docker?

I found a scuffed way to do this, copy this PR's src/rendervulkan.cpp, modify it with #1548 patches, upload that to a pastebin website, then add a curl command in the Dockerfile so it downloads the modified file and puts it where its supposed to be. Cursed, but it works.

[Boulotaur2024]

Please merge this.
I'm planning to use it for upscaling actually.
I'd love to have some gaussian upscaling to make things a little blurry in-game.
I like my games this way while most people want super sharp stuff, I know.
But not every one loves the "oil-painting look" of FSR, you know...

[Reonu]

Very excited about this and hoping it gets merged soon

[rKsanu2MMYvypWePtQWM]

Me too...

…

On 17 December 2024 01:13:01 EET, Reonu ***@***.***> wrote: Very excited about this and hoping it gets merged soon -- Reply to this email directly or view it on GitHub: #740 (comment) You are receiving this because you were mentioned. Message ID: ***@***.***>

[Eckoa]

Please merge this. I'm planning to use it for upscaling actually. I'd love to have some gaussian upscaling to make things a little blurry in-game. I like my games this way while most people want super sharp stuff, I know. But not every one loves the "oil-painting look" of FSR, you know...

This is actually what im waiting for also, Bicubic does a much better job of preserving shapes in the image even if the image is softer vs FSR/NIS which both tend to round corners or have some nasty ringing