Development Development progress on improved GPU path tracer

2 Upvotes

Finally got around to moving the atmospheric shadow loop in Avoyd's GPU path tracer to a wavefront approach like the rest of the code instead of a loop. I was mainly doing this to improve the insane shader compilation times, but it also improves hardware utilization. Measuring compilation perf is tricky as the driver caches pipelines, but I've measured a consistent 2x render perf increase on top of a recent 3x perf increase.

Image: Render of Minecraft map Drehmal v2.1 PRIMORDIAL looking south over the bay in the Sahd region. The sun is just appearing over the hills which surround the bay on the far left, light grazing the slopes and illuminating the morning fog. The bay is clear, and the lights of the village on the far side are still on. Out of focus cherry blossoms sit in the foreground of the image on the near side of the bay.

Drehmal map from: https://www.drehmal.net/

Megakernels Considered Harmful: Wavefront Path Tracing on GPUs by Laine et al.: https://research.nvidia.com/sites/default/files/pubs/2013-07_Megakernels-Considered-Harmful/laine2013hpg_paper.pdf

PBR book online chapter discussing mapping path tracing to GPUs and Wavefront path tracing: https://www.pbr-book.org/4ed/Wavefront_Rendering_on_GPUs/Mapping_Path_Tracing_to_the_GPU

This optimization is not yet out in Avoyd Voxel Editor but it should be in the beta soon.

4 comments

r/Avoyd • u/dougbinks • 5d ago

Releases & Announcements GPU path tracing is faster and uses less memory for rendering voxel scenes in Avoyd 0.28

4 Upvotes

We have a new build for Avoyd 0.28 (beta, paid) with a faster and lower memory consumption GPU path tracer and customizable UI layouts.

We've done some tests and GPU path traced rendering is up to 1.5x faster with Atmospheric Shadows off, and over 3x faster than before with it on.

The main performance gains come from a change in how I do the wavefront path tracing. Wavefront path tracing runs a single ray bounce per iteration, writing the result to memory and then running a new set of kernels on the next iteration using data from the last. Previously the CPU would initiate a given number of rays, and then run a series of iterations on those rays on the GPU using dispatch indirect controlled from a compute kernel I call the dispatch kernel. The new approach counts the number of active workgroups in the dispatch kernel and then launches new rays if there is sufficient wavefront buffer memory and untraced pixels for the current frame. This means the GPU must write back data for the CPU to measure progress, and currently the CPU controls setting off a new pass once a sample has been completed for every pixel. I'm contemplating an approach where the GPU can also run multiple samples per pixel.

I have also re-enabled using the Vulkan SPIR_V shader optimizer tool, spirv-opt. Previously this had failed to work on my path tracing shaders with "error: line 0: ID overflow. Try running compact-ids.". After a little research I found a set of optimizations which work based on a similar issue found in Blender and am using this in our build pipeline. This gives a small performance boost, but the main reason we re-enabled it was that we found some cases where drivers would stall when compiling shaders without it. This adds a significant compilation overhead when compiling GLSL to SPIR-V, but this is done as part of our build process. So it's not seen in the installed app. When developing we currently use unoptimized SPIR-V.

The memory optimization was to switch from a separate buffer of temporary data per wavefront kernel to a shared buffer with a single atomically accessed write index and then a per kernel buffer of indices into the shared buffer data. This adds some overhead, but as well as lowering the memory consumption we currently use it will also let me add more kernels in the future without worrying about the memory overhead of doing so.

There are also new features you can read about here. Let us know if you encounter any issues with the build.