Realtime Global Illumination implementation progress

[mafiesto4]

Hello! Some time ago I started working on real-time Global Illumination in Flax. It’s very challenging task but required to reach higher visual quality for games in Flax. I’ll use this thread to give you updates with the progress of implementation, as well as, descripting the algorithm steps.

Implementation happens on separate gi branch on Github. Flax Roadmap task: https://trello.com/c/57rwlKmf/1-realtime-gi

Goals:

• Provide real-time Global Illumination for desktop and console games
• Target DX11, DX12 and Vulkan - no RTX required!
• Achieve nice scalability for older GPUs but allow for super high-quality GI if needed
• Ability to use new GI on all Flax Showcase scenes without any changes in content
• Support materials with all properties: color, roughness, emissive, etc.
• Limit to ~4ms at 1080p on GTX 1070 on Medium quality

[mafiesto4]

In order to implement real-time GI we need a way of gathering diffuse light in the scene to bounce it on objects. There are several great techniques used in games such as light probes, screen-space tracing with denoising or voxel GI. Some of them utilize rendering scene to the cubemap, integrating it into irradiance and using as light probe (irradiance volumes-based techniques). Others use custom scene representation as voxels or SDF to perform tracing. Nowadays, with hardware raytracing (RTX) it’s easier to gather diffuse light by shooting bunch of rays across the scene.

One of our goals is to provide GI on mid-price GPUs so I decided to use Sign Distant Fields (SDF) as a scene representation in low-resolution to perform cheap, software raytracing. Here you can see a small a Global Sign Distant Field generated for the whole scene in real-time (4 cascades, 19 world units precision = 19cm).

It’s being rasterized in real-time (limits to 1ms per frame) from all meshes on a scene (engine has new utilities to generate SDF for imported meshes - also at runtime for procedural games). Building such Global SDF required some tricks as it uses 130 MB of GPU memory (256x256x256 volume resolution).

With the Global SDF baked we can easily perform whole-scene tracing in shader:

GlobalSDFTrace trace;
trace.Init(ViewWorldPos, viewRay, ViewNearPlane, ViewFarPlane);
GlobalSDFHit hit = RayTraceGlobalSDF(GlobalSDF, GlobalSDFTex, GlobalSDFMip, trace);
if (!hit.IsHit())
	return float4(float3(0.4, 0.4, 1) * saturate(hit.StepsCount / 80.0), 1);
return float4(hit.HitNormal * 0.5f + 0.5f, 1);

[mafiesto4]

Global SDF support has been added to materials and particles as well, which allows to sample distance and direction to the nearest surface geometry at any point in the world.

it can be used to render foam or splashes in water materials, or to add procedural dirt in building corners. Below you can see the plane material being darker nearby intersecting geometry by using a single node in material graph.

Particles can collide with whole-scene SDF (GPU-only) and implement more complex VFX systems such as worms, bees flying though the scene . For example, birds particle system can avoid nearby surfaces and fly though the scene.

[mafiesto4]

Global SDF provides a good way of ray-tracing though the scene, but how to get color of the hit surface? Voxel GI can sample color of the voxel, RTX can evaluate hit material color but how to do it for SDF? We could store material properties in he Global SDF volume but this would grow it’s size significantly and result in color bleeding artifacts (eg. single voxel wall would bleed color between sides).

To overcome this I introduced Global Surface Atlas rendering. It’s a big 2D texture atlas (2k/4k res) in GBuffer format (depth + emissive + color+ roughness + normal + metallic + ao). It stores scene object projections from all sides (up to 6 projection per-object).

Here you can see debug view of those projection in the scene-view. Note that objects are captured from all sides by rendering them into atlas.

Dynamic objects are recaptured every few frames, while static every few seconds. Each object projection (called tile) has resolution calculated based on its size in the world and distance to camera - larger objects use more atlas space. Atlas supports tiles refitting (eg. when object gets closer to view) and can perform automatic defragmentation.

With this Atlas we can sample it per-pixel to get low-resolution geometry properties.

[mafiesto4]

Global Surface Atlas pass after capturing material properties can iterate over all lights to evaluate direct lighting for each atlas texel. This works just like for normal deferred rendering in screen-space but it’s done in atlas texture-space. This outputs a single atlas texture with direct lighting and it will be a main input to the future Global Illumination and Reflections algorithms.

Additionally, since we have access to full-scene tracing and this rendering is performed in low-res we can use Global SDF to implement fast dynamic shadows (no need for shadow maps).

Here you can see a single dynamic point light in the Library scene (books on shelves are culled from atlas because they are too small ).

[mafiesto4]

Progress update on software raytracing implementation: it’s ready to start working on actual Global Illumination algorithm. Here you can see Desert Ruins scene as a reference.

Here is the same viewport with Global SDF (for all static meshes and terrain). With cache for static geometry the performance impact is very small (<0.1ms). Affects performance only for dynamic parts of the map and for moving camera.

Here is Global Surface Atlas view with per-pixel software raytracing (direct lighting on the left, material properties on the right). Performance is pretty good: ~1ms per-frame to update Global Surface Atlas and 1.1ms for 1 ray/pixel at 1080p (on RTX 3070).

[mafiesto4]

Dynamic Diffuse Global Illumination implementation progress update. Here are irradiance probes with automatic placement around camera. We use Global SDF and Global Surface Atlas for software raytracing. It uses not only color but also depth information around the probe to reduce light leaking when sampling probes gid. With Global SDF we can quickly sample distance and direction to the nearby geometry to disable probes that are too far the geometry and relocate the ones that are inside or too close geometry. This improves both quality and performance.

Here is automatic probes relocation in action:

[mafiesto4]

GI work in progress. Base algorithm works, now it’s time to work on quality and stability. Then will be performance pass. Newly added skylight (you can use HDRi as skybox to light whole scene without any lights!) and infinite bounces (previous frame GI is used in Global Surface Atlas).

[mafiesto4]

Here is a little longer showcase of dynamic global illumination in action:

As you can see it’s getting pretty nice. There are some minor artifacts and we gonna need to add cascades to cover larger area of the map but the base algorithm works. Since we’re using irradiance probes at ~1m resolution it doesn’t capture so much small indirect shadowing or details but still has way low amount of leaking compared to the traditional light probes (DDGI algorithm is very good).

[mafiesto4]

Realtime Global Illumination is already on a master branch and is ready to be tested by devs. Now, I’m working on sample Demo scene for the upcoming Flax 1.4 update. Here you can see the variable time of the day that affects the lighting in the scene:

[mafiesto4]

Showcase of some screenshots from the upcoming Tech Demo in Flax:

[Spectrix]

This is incredible work! Thank you! Will this demo be released for developers to look at? Or will it remain closed?

[mafiesto4]

We will see…

[SpaceWithoutDistance]

This looks so danged good. I’ve always liked the lighting in Flax. It’s not just technical, it’s aesthetically good.

Tons of 3D game engines have the sun, and they have clouds, and they have skies, but you can tell when someone thought beyond just maths when it comes to the sun in the sky. When the sun shines on you in Flax, it feels good. It’s one of two engines that I will stop and watch the sun set while play testing.

[reanimate]

That’s looks really good, any chance we can get a build demo version of that?

[mafiesto4]

Tech Demo 2022 - Flax Engine

[alphayash]

This looks really good to be honest...

[lsscpp]

Any chance that Flax will feature some lighter kind of global illumination, such as VoxelProbeGI or similar?
I found everything great in this engine, apart its portability. I’m not taking into account lightmapping, also because it requires RTX card to be baked

Wider context

I've been working on a research project (a detailed interior) both in unreal engine and in unigine, latest versions. While epic gives the most eyecatching result, it poorly failed to keep a decent bitrate on a 10xx rtx nvidia card. Unigine a bit inferior visuals, but still quite pleasing, performed very smoothly. I really liked how easy it was to setup unigine GI. Seeing something similar in Flax would be 'killa'

[reanimate]

Having this option would be great especially if we want to support lower spec devices. Maybe precomputed DDGI or any other Precomputed Radiance Transfer ideally support sky visibility and streamable volume and lighting scenario , The last part would be great for a game that use Time Of Day but have a static scenes

[mafiesto4]

Pre-baking of DDGI volumes and other cool features are planned on a roadmap: Trello (some of them are planned to be done for Flax 1.8).