Author Topic: GLSL Post-Processing  (Read 12494 times)

Michael Lobko-Lobanovsky

  • Administrator
  • *****
  • Posts: 1481
GLSL Post-Processing
« on: December 14, 2016, 02:24:18 pm »
If we draw an OpenGL scene first into a texture (in fact, a Frame Buffer Object a.k.a. FBO) rather than directly into the OpenGL canvas and only then output this texture as a skin for an OpenGL window sized quad, then we can post-process this texture to add some visual effects to it to make the scene look more expressive.

Some most common modern post-processing effects are bloom a.k.a. halo, bokeh a.k.a. depth of field, motion blur, full-screen anti-aliasing, light shafts a.k.a. "God rays", and vignetting.

Some of these effects are easier to implement than the others because they wouldn't require manipulations with the real scene geometry. Our Wavefront Object scenes contain millions of polies, and re-rendering them all over again to trace a bunch of rays for every pixel in every poly would be an enormous task even for a modern high-end GPU.

Luckily some effects can be faked with sufficiently convincing results. Below you will find an example of GLSL application that uses a sample texture (we could've used our own FBO to the same effect) to mimic light shaft ray tracing and vignetting. The fake effect may be used to add simulated light shafts to sunny outdoor scenes, high-contrast interiors like in Windows 10's stock wallpaper, or car headlights.

The shader runs in a continuous render mode at approx. 25FPS but even so it wouldn't consume more than 20% of your GPU even at full screen. CPU is typically less than .5%.

Application controls:
  • Spacebar -- pause/resume animation
  • LMB down and drag -- manually control "sun" position (regardless of animation)
  • Alt+Enter -- toggle full screen mode
  • Escape -- quit application.
Feel free to experiment with the fragment shader parameters but avoid using too many samples. The shader uses a lot of texture lookups and may overload your GPU if the number of samples is too high.
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1983
    • zapsolution
Re: GLSL Post-Processing
« Reply #1 on: December 14, 2016, 03:52:39 pm »
Mike

Thank you for this very instructive GLSL example !

...
Patrice
(Always working with the latest Windows version available...)

Michael Lobko-Lobanovsky

  • Administrator
  • *****
  • Posts: 1481
Re: GLSL Post-Processing: FXAA
« Reply #2 on: December 21, 2016, 05:51:22 am »
Multi-Sample Anti-Aliasing (MSAA) that both ObjReader and Objector are currently using to anti-alias their renders has two restrictions:
  • it affects geometric edges only but not the borders between differently colored portions of textured planes nor the visible edges of alpha-textured objects; and
  • it is ineffective entirely if OpenGL uses any other render target than the on-screen window.
Deficiency #1 above can be partially compensated for by heavy anisotropic correction of textures, and that's what we are using now too. But 16/32x MSAA and 16x aniso combined are a very, very heavy burden for our rendering pipelines indeed.

Furthermore, since we will have to use Frame Buffer Objects (FBOs, in fact GPU memory texture pixel array buffers) to be able to apply post-processing effects like in my earlier post at least in our Programmable Pipeline (PPL), we need to find an alternative to MSAA.

Fast ApproXimate Anti-Aliasing (FXAA) is an algorithm that resolves deficiencies #2 and #1 (in its plain- and alpha-textured aspects) entirely, and is at the same time equivalent to approx. 4x MSAA in what regards geometric edges. It is also very lightweight and bogglingly fast: it adds no more than 1% GPU cost to medium-sized window renders, and no more than 2%, in full screen.

Mathematically, FXAA is a very tight, subpixel-perfect, luma-corrected, rhombus-shaped kernel blur filter applied to exery pixel in the final FBO that would texturize the viewport-sized quad to be rendered into the on-screen window.

Being essentially a blur filter, FXAA does add some insignificant "soapiness" to the overall look of the scene. It is also known to yield artifacts in places where the scene may contain some distant objects textured as posters or signboards displaying text. Yet I hear that both drawbacks can be efficiently eliminated by applying yet another Sobel edge detection/sharpening filter to FXAA corrected pixels. It will remove both "soapiness" and text texturing artifacts I mentioned. I do have some fast solutions of such image sharpen filter shaders for use in streaming video rendering, but they are written in DirectX HLSL rather than GLSL, so this will have to wait till I translate them for use with this FXAA shader.

FXAA capability is included with nVidia drivers' Control Center as a 3D rendering option but i) it is a system-wide setting, ii) it can't be enabled programmaticaly for a specific OpenGL application through the Control Center API, and iii) it's a nVidia-only solution.

In the meantime, consider my test case application appended below. Left-click and drag the white splitter bar across the viewport to see how FXAA anti-aliases the jaggy picture that emulates our future post-processing FBO. It performs so well that, even when the window is sent full screen and the jaggies get exaggerated to nearly twice the size they can ever get in reality, the effect is still stunning. The app renders continuously at approx. 30FPS just to show off how low the FXAA cost really is.

« Last Edit: December 21, 2016, 10:45:04 am by Michael Lobko-Lobanovsky »
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1983
    • zapsolution
Re: GLSL Post-Processing
« Reply #3 on: December 21, 2016, 09:12:38 pm »
Very impressive work my friend, and the resulting antialias looks great !!!

I am currently making my {slow} way with Media Foundation (the documentaion is very sparse), and most examples are OOP based, something very indigest to me.  ::)

...
Patrice
(Always working with the latest Windows version available...)

Michael Lobko-Lobanovsky

  • Administrator
  • *****
  • Posts: 1481
Re: GLSL Post-Processing: FXAA Update
« Reply #4 on: December 22, 2016, 02:28:30 am »
Thanks, Patrice!


Heh, Sobel unsharp mask works and does preserve anti-aliasing but is currently too strong. ;)

Need to fiddle some more with the parameters and probably the size of filter kernel too. The smallest Sobel kernel is 3x3 pxs large, so it would add at least 16 extra texture lookups (8 for blur and another 8 for unsharp mask), so an estimated overall cost of a joint FXAA/Unsharp Mask shader will be around 4% GPU, for medium sized, and 9% GPU, for full screen renders. But in any case, it is expected to be significantly less than the current 16x MSAA+16x aniso mixture.

Stay tuned! :)

[UPD] I built a very good and fast 1.5 pxs tight Sharpen Edge shader that uses only 8 texture lookups and is as lightweight as this FXAA. I.e. the sum of the two would be +2% medium size and +4% full screen. It can also selectively blur non-edge texture areas, if needed, and even denoise (despecle) them -- all in one pass.

But now I'm not satisfied with the quality of this FXAA in the first place. This solution is very basic and accepts default settings only. It works very fine for closeups but starts to yield artifacts at midrange distances.

nVidia's FXAA is better than 8x MSAA!!!  In fact, it must be as good as 16x MSAA -- and only at a fraction of its GPU cost.

Believe it or not but I've found nVidia's shader code. It's HLSL and it's terribly versatile; it's intended for use on PCs and game consoles with a zillion of various settings and digrees of quality. I need time to analyze it, isolate PC-only stuff, and translate it to GLSL.
« Last Edit: December 22, 2016, 08:14:08 am by Michael Lobko-Lobanovsky »
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Michael Lobko-Lobanovsky

  • Administrator
  • *****
  • Posts: 1481
Re: GLSL Post-Processing: FXAA Update
« Reply #5 on: December 22, 2016, 08:08:14 pm »
I got NVFXAA.fs working, Patrice.

It works visibly better than nVidia Control Center's own system-wide FXAA option, to say nothing of my initial FXAA.fs shader (which I also fixed). In fact, it works exactly as fine as GeForce.com's link above -- with this original 1920x1080 non-AA image that can be drag-and-dropped from their another FXAA/NO AA comparison page, and it works real fast even at its highest quality setting. It also looks pretty nicely with various 1:1 full-screen no-AA screenshots of your models I took in my Objector. 8)

I'm now going to experiment with adding that Sharpen Edge filter to it to see if this can improve the results even more.

What is your full screen resolution? Is it 1920x1080 pxs?

Keep staying tuned! :)


(It's a very heavy fine-detail render to AA, Patrice. No MSAA, no aniso, no mipmaps -- just to make everything as bad as it only can be... :) )
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Michael Lobko-Lobanovsky

  • Administrator
  • *****
  • Posts: 1481
Re: GLSL Post-Processing: FXAA Update
« Reply #6 on: December 24, 2016, 07:06:18 am »
As it turned out, there's no need to restore sharpness artificially with an extra filter. The shader has a native parameter to control the amount of area blur/dither (similar to alpha-to-coverage; useful for scenes with lots of grass and foliage in view) independently of edge jaggie smoothness. :)

I'm currently preparing a full-scale demo to test FXAA with a bunch of jagged screenies taken in a number of real games at their low-quality settings. Will the forum allow me a ~75MB upload in my next submission?
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Michael Lobko-Lobanovsky

  • Administrator
  • *****
  • Posts: 1481
Re: GLSL Post-Processing: FXAA Demo App
« Reply #7 on: December 24, 2016, 11:37:12 pm »
Feeling myself Pere Noele tonight... 8)

Here comes an extensive demo of FXAA applied to low-quality, jagged screenshots taken in a few real 3D game environments.

The instructions are in the README file and some relevant documentation has also been added to the zip.

Enjoy and have a Merry Christmas week!

Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Michael Lobko-Lobanovsky

  • Administrator
  • *****
  • Posts: 1481
Re: GLSL Post-Processing
« Reply #8 on: December 25, 2016, 02:13:57 pm »
Hehe, thank you, Patrice!

I love such pieces of code. Stay assured it will be scrutinized to the fullest. :)

Your code will help me add audio support to my GLSL shaders. Yes, it can be done. Consider bass.dll's 512 sample long FFT frequency buffer (1024 samples also seem feasible on modern CPUs/GPUs) and a matching 512 sample long volume buffer for one mono channel (or four buffers for stereo) as two (or four) scan lines of a 512x2 (or 512x4) px large texture that you can bind to a spare texture unit on the FBSL/PB/VB/C/C++/whatever side of your code, and then poll it via a corresponding sampler2D uniform, on the GLSL fragment shader side of it, to modulate color and position data of pixels/texels that you're rendering onto your final on-screen textured quad.

That's how OpenGL/DirectX video plugins (a.k.a. "reactive visualizers") are usually implemented. :)
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1983
    • zapsolution
Re: GLSL Post-Processing
« Reply #9 on: December 25, 2016, 02:45:12 pm »
What Windows OS are you currently using?

I would like to check my WIP ZMF player on another computer than mine...

>>That's how OpenGL/DirectX video plugins (a.k.a. "reactive visualizers") are usually implemented
So far, all the BassBox OpenGL visual plugins i have written are working on ALL the Windows flavor, and even with poor video card.

...
« Last Edit: December 25, 2016, 02:47:10 pm by Patrice Terrier »
Patrice
(Always working with the latest Windows version available...)

Michael Lobko-Lobanovsky

  • Administrator
  • *****
  • Posts: 1481
Re: GLSL Post-Processing
« Reply #10 on: December 25, 2016, 03:20:03 pm »
What Windows OS are you currently using?
  • x86 XP Pro Sp3 / GTX 550Ti
  • x86 Vista Ult Sp2 / GTX 550Ti
  • x64 Win 7 Ult Sp1 / GTX 550Ti
  • x86 Win 7 Ult Sp1 / on-board Intel SiS Mirage
  • x64 Win 7 Ult Sp1 / ATi Radeon R7 240
  • x64 Win 10 Anniversary / geForce 8600 GTS
Quote
I would like to check my WIP ZMF player on another computer than mine...

Any time; just send me the executables appropriate for each of the above.

Quote
>>That's how OpenGL/DirectX video plugins (a.k.a. "reactive visualizers") are usually implemented
So far, all the BassBox OpenGL visual plugins i have written are working on ALL the Windows flavor, and even with poor video card

Modern on-board Intel HD Graphics claims OpenGL 3.0/3.1 compatibility, however their support of OpenGL specification is poor and incomplete. Its GLSL in fact lacks many extensions that were part of OpenGL 2.0. Yet some basic shaders can still be run on it too.

This Julia will definitely run on any nVidia-less (low-end Intel HD Graphics) laptop.


P.S. I think you'll be able to check your compatibility with Intel HD Graphics even on your own laptop, by using your nVidia Control Center Rendering Options like this:



This is for Lenovo but it also works for Acer.


P.P.S.
.... OpenGL visual plugins i have written are working on ALL the Windows flavor, and even with poor video card.


With all my respect to your prior artwork, using legacy immediate mode OpenGL rather than GLSL shaders, you will probably need half a minute to calc, anti-alias and draw just one frame of such a disco light visualizer, let alone sync it with your streaming audio, Patrice: ;)
« Last Edit: December 25, 2016, 04:14:29 pm by Michael Lobko-Lobanovsky »
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1983
    • zapsolution
Re: GLSL Post-Processing
« Reply #11 on: December 25, 2016, 04:38:44 pm »
Currently ALL my visual plugins have been written in PB, and most of them 10 years ago !.
Today i would write them in C++ 64-bit, using the same setting than in ObjReader :)
Patrice
(Always working with the latest Windows version available...)