This approach was originally developed by Alex Sanikov for Path of Exile 2.
Of course in PoE2 it's used in full 3d.
The benefit of this approach is that you get global illumination with a constant cost for the entire scene and because it doesn't use any temporal acculation, it has zero latency as well.
There is no traditional "lighting" added to these two effects. The light on the nearby surfaces is indirect light from the GI solution that you get for free by just spawning the particles. That means all effects just naturally emit light with no extra work from artists.
On my GPU (which is admittedly a 4090) the GI solution runs in 0.8ms for a 4k scene in "High" quality. This is exactly what it will always cost, no matter what scene you choose.
One caveat is doing screen space means anything offscreen doesn’t contribute to lighting. For a fixed perspective dungeon or 2d game it works great, but a light source behind a column or in another room will cast no light on the scene
There are multiple ways to solve this issue. The most naive would be rendering to a larger texture and cropping, but with radiance cascades you can do better- you could only render what's necessary based on per-cascade interval lengths. Depending on needed accuracy, you could calculate it similar to ambient light, only calculating it for the largest cascade- wouldn't be perfect, but could feel pretty natural.
From what I understand, PoE2 has a fixed-perspective camera, so radiance is calculated in screenspace (2D) as an optimization (and not using a 3D texture). It would be interesting to see how the performance of this technique scales to full 3D, as that would be significantly more expensive / require a lower-resolution voxelization of the scene.
The source data is indeed from screen space, but this generates a full 3d data structure by ray marching the depth buffer. Its not using the 2D algorithm. In PoE2s case, it doesn’t need to go further since light sources are not usually going to be behind the camera.
It looks fine, but I have yet to see any examples scaled up to an outdoor scene where you can see for miles(I don't even know if this is possible?).
Also the article this post links says this is diffuse only.. kinda not so impressive as specular is also very important.
I assume this means they are using a diffuse model that is view direction independent so Lambert.. which is a rather crummy diffuse model.. the better diffuse models are view dependent..
There's ambient occlusion that computes light intensity with high spatial resolution, but completely handwaves the direction the light is coming from. OTOH there are environment maps that are rendered from a single location, so they have no spatial resolution, but have precise light intensity for every angle. Cascade Radiance observes that these two techniques are two extremes of spatial vs angular resolution trade-off, and it's possible to render any spatial vs angular trade-off in between.
Getting information about light from all angles at all points would cost (all sample points × all angles), but Radiance Cascades computes and combines (very few sample points × all angles) + (some sample points × some angles) + (all sample points × very few angles), which works out to be much cheaper, and is still sufficient to render shadows accurately if the light sources are not too small.
Path tracing techniques usually focus on finding the most useful rays to trace, to focus only rays that hit a light (importance sampling).
RC is different, at least in 2D and screen-space 3D. It brute-force traces fixed sets of rays in regular grids, regardless of what is in the scene. There is no attempt to be clever about picking the best locations and best rays. It just traces the exact same set of rays every frame.
Full 3D RC is still too expensive beyond voxels with Minecraft's chunkiness. There's SPWI RC that is more like other real-time raytracing techniques: traces rays in the 3D world, but not exhaustively, only from positions visible on screen (known as Froxels and Surfels elsewhere).
Penumbra hypothesis is an observation that hard shadows require high resolution to avoid looking pixelated, but soft shadows can be approximated with bilinear interpolation of low-res data.
RC adjusts its sampling resolution to be the worst resolution it can get away with, so that edges of soft shadows that are going from dark to light are all done by interpolation of just two samples.
IIUC basically you have a quad/oct-tree of probes throughout the area of screen space (or volume of view frustum?). The fine level uses faster measurements, and the broad level uses more intensive measurements. The number of levels/fineness determines resolution.
I guess for comparison:
- Radiance cascades: complexity based on resolution + view volume; can have leakage and other artifacts
- Ray tracing: complexity based on number of light sources, screen resolution, and noise reduction; has noise
- RTX: ??
- Radiosity: complexity based on surface area of scene
Also not sure, but I guess ray tracing + radiosity are harder to do in GPU?
No octree/quadtree. It's just a stacked grid of textures, halving resolution (or otherwise reducing) each level. Low resolution layers capture many rays (lowest resolution say 4096 rays) and longer distances at lower spatial resolution. High resolution layers capture fewer rays (lowest say 4 rays) for shorter distances at high spatial resolution. When you merge them all together, you get cheap, soft shadows and a globally illuminated scene. In the post I wrote, I calculated it's similar in cost to firing 16 rays using a classic path tracer. But in theory should look similar to 4096 rays (or whatever the highest cascade layer does) but softer shadows.
Depending on your approach the geometry of the scene is completely irrelevant. (Fixed step / DDA truly, JFA + DF has some dependence due to circle marching, but largely independent)
This is true for quite a few graphic algorithms today. Certain terms in the equations used are pre-computed, and you can generalize all of them in a sense to be “probes”, “queries”, “function inversions”, “look up tables” or whatever. The only real way to know what can be computed on the fly vs what can be stored is to go ahead, try to implement it, think really fucking hard, and then see what you can do in real time. That’s basically how the field has advanced the formulas used into something that can run in real time, or can run in screen space complexity rather than scene complexity, and so on.
An equation given to you will be one of 2 forms usually; the raw equation that may come from optics, in which case expect an ass load of calculus and diff equations with arcane terms not seen outside of Maxwell’s equations. Or, in the likely case, the presenter is a PhD ubermensch nerd who has already pulled the terms out, rewritten them, is presenting them, and you need to be really paying close attention to every single word during their talk to figure it out. It’s at your discretion to determine which of the two forms is easier for you to deal with.
Am I correct in thinking that this approach effectively makes real time GI a solved problem?
Can someone who's more knowledgeable than myself offer any explanation on why it took so long to discover a method like this? I remember hearing about rt-GI from over 25 years ago.
So on my webcam there is a cover but it doesn't fully cover the webcam. So this technology would be able to infer something from the radiance of the light seeping in around the edges?
This is a rendering technique designed for real-time graphics, and it's not applicable to that kind of image analysis. It does what has already been possible with ray tracing, but using an approximation that makes it suitable for real-time graphics.
However, the technique has been used to speed up astrophysics calculations:
Of course in PoE2 it's used in full 3d.
The benefit of this approach is that you get global illumination with a constant cost for the entire scene and because it doesn't use any temporal acculation, it has zero latency as well.
This means you can rely on it as the lighting for fast effects. For example: https://www.youtube.com/watch?v=p1SvodIDz6E
There is no traditional "lighting" added to these two effects. The light on the nearby surfaces is indirect light from the GI solution that you get for free by just spawning the particles. That means all effects just naturally emit light with no extra work from artists.
On my GPU (which is admittedly a 4090) the GI solution runs in 0.8ms for a 4k scene in "High" quality. This is exactly what it will always cost, no matter what scene you choose.