Interactive Monte Carlo Denoising using Affinity of Neural Features
SIGGRAPH 2021
Mustafa Işık Krishna Mullia Matthew Fisher Jonathan Eisenmann Michaël Gharbi
Our denoiser targets images from interactive path-tracers with low sample per pixel (spp) budget (a). It runs at interactive rates and produces temporally-stable, high-quality results (c), with better details and fewer artifacts than state-of-the-art off-line denoisers (b), and significantly outperforms interactive denoisers (not shown). Timings are recorded using PyTorch implementations on a GeForce RTX 2080 Ti graphics card. See Table 1 for our optimized implementation that runs interactively. The reference (d) was rendered with a much higher computational budget. The resolution is 1024 × 1024.
Abstract
High-quality denoising of Monte Carlo low-sample renderings remains a critical challenge for practical interactive ray tracing. We present a new learning-based denoiser that achieves state-of-the-art quality and runs at interactive rates. Our model processes individual path-traced samples with a lightweight neural network to extract per-pixel feature vectors. The rest of our pipeline operates in pixel space. We define a novel pairwise affinity over the features in a pixel neighborhood, from which we assemble dilated spatial kernels to filter the noisy radiance. Our denoiser is temporally stable thanks to two mechanisms. First, we keep a running average of the noisy radiance and intermediate features, using a per-pixel recursive filter with learned weights. Second, we use a small temporal kernel based on the pairwise affinity between features of consecutive frames. Our experiments show our new affinities lead to higher quality outputs than techniques with comparable computational costs, and better high-frequency details than kernel-predicting approaches. Our model matches or outperfoms state-of-the-art offline denoisers in the low-sample count regime (2–8 samples per pixel), and runs at interactive frame rates at 1080p resolution.
Bibtex
@article{isik2021anf,
title = {Interactive Monte Carlo Denoising using Affinity of Neural Features},
author = {I\c{s}{\i}k, Mustafa and Mullia, Krishna and Fisher, Matthew and Eisenmann, Jonathan
and Gharbi, Micha\"{e}l},
journal = {ACM Transactions on Graphics (TOG)},
volume = {40},
number = {4},
articleno = {37},
pages = {1--13},
year = {2021},
publisher = {ACM New York, NY, USA},
doi = {10.1145/3450626.3459793},
url = {https://doi.org/10.1145/3450626.3459793},
}