Lecture
Image synthesis involves computing a large number of integrals. It is impossible to evaluate them analytically, since the integrand functions are not given analytically. Consequently, numerical integration methods must be applied.
The Monte Carlo path tracing algorithm [Ritschel et al 2011] is a physically correct, high-quality synthesis algorithm. However, its significant drawback is slow convergence. Synthesis can be accelerated by tracing a small number of rays and then removing the remaining noise by filtering the image. This work proposes an algorithm that outperforms comparable methods in filtering quality and makes it possible to substantially reduce image synthesis time.
However, the use of quadrature formulas is not efficient. Their convergence rate depends on the dimensionality of the space and drops rapidly as the dimensionality of the integration domain increases. The dimensionality of the integrals in computer graphics is fairly high (2 for primary illumination, 4 for secondary illumination from light reflected once from a scene surface, and so on). Therefore other methods are required. Such methods are the Monte Carlo and quasi-Monte Carlo methods.
Let us explain how Monte Carlo methods work from the outside. Rays are cast from the visible surface. As they are traced, paths are formed along which light may propagate. At each point where a path intersects a surface, the primary illumination is computed. This value is then used to compute the contribution to the secondary illumination of the point from which the path was cast. Next, a decision is made whether to continue the path. This can be done using simply a random variable, or the path can be terminated when the contribution to the secondary illumination of the path's origin becomes very small.
Now the integral must be computed. To evaluate it, it is represented as the mathematical expectation of some random variable.
The direction can be chosen not as the value of a uniformly distributed random variable, but as the value of a random variable whose probability density is proportional to the BRDF. Algorithms for choosing random directions that adapt to the lighting conditions can also be applied.
When using quasi-Monte Carlo methods, the formulas do not change. Simply, the random choice of direction is replaced by a quasi-random one. One of the quasi-random (deterministic) sequences is used. This makes it possible to increase the convergence rate.
Monte Carlo and quasi-Monte Carlo methods share the drawback of being too slow. For example, to obtain a rendering with a more or less reasonable value of primary illumination for point light sources, approximately N=400 is required. Computing secondary illumination requires N=1600. With such parameters, interactivity is out of the question.
If a small sample is used in the Monte Carlo method (for example, spending only 30 rays on computing secondary illumination), noise arises due to the random nature of the method.
Irradiance caching [Pharr and Humphreys 2004] is one way of speeding up the computation of the illumination integral over a set of points. The method is designed for
working with the diffuse component of illumination. The secondary diffuse illumination component in an image changes smoothly, so it can be computed by some method at only a very small number of points, and interpolated elsewhere. The work [Kontkanen et al 2006] considers a method for accelerating synthesis by Monte Carlo path tracing or photon mapping by combining the irradiance caching algorithm [Pharr and Humphreys 2004] with filtering. The method significantly reduces the number of samples needed to build the cache by filtering at the stage of sampling values from the cache. For photon maps, filtering is applied at the final gathering stage. Filter weights are chosen individually for each cache point, depending on the complexity of the geometry at that point. The algorithm makes it possible to synthesize an image 5 times faster than with irradiance caching alone. The method avoids the artifacts characteristic of irradiance caching, but has artifacts caused by blurring, and complicates the operation of sampling from the irradiance cache.
The algorithm [Suykens and Willems 2000] accelerates image synthesis by the progressive Monte Carlo ray tracing method by filtering the result of each iteration. The authors propose using a filter with an Epanechnikov kernel [Silverman 1986], whose size varies depending on the density of the Monte Carlo sample distribution according to an empirical law. Taking into account only the sample distribution, rather than the features of the scene, the algorithm preserves edges in images poorly.
The authors of [Gastal and Oliveira 2012] propose a multidimensional filter for images, applicable to processing arbitrary signals. The filter considered makes it possible to take into account a large number of dimensions besides color that contain additional information. While showing a high-quality result comparable to bilateral filtering [Kornprobst and Tumblin 2009], the method has linear complexity and a high processing speed.
[Dammertz et al 2010] proposes an algorithm for filtering illumination in screen space in real time. The paper proposes an improvement of the bilateral filter using the A-Trous wavelet transform [Holschneider et al 1989], adding an additional weighting function to the transform to determine edges in the image. Additional pixel information is used to compute the function: normal direction and world-space coordinates. The additional weight in the wavelet transform is determined based on the proximity of pixels in terms of normal and coordinates. The method preserves image edges well, but the filtering suffers from artifacts in the form of colored spots on uniform surfaces. A GPU implementation makes it possible to process 0.25-megapixel images at a speed of approximately 4 frames per second.
The article [Hachisuka et al 2008] considers a method for the adaptive distribution of Monte Carlo samples over an image. Each ray is treated as a point in a multidimensional space (for example, taking into account lens coordinates, time), thereby defining a multidimensional function in the image space. The multidimensional function is interpolated anisotropically, and positions for subsequent samples are chosen closer to sharp changes in the multidimensional function, which correspond to those areas of the image where more samples are needed: edges, soft shadows, areas outside the camera's focus. MSE minimization. The work [Rousselle et al 2011] describes an iterative method for computing Monte Carlo samples and synthesizing images. At each iteration, the method chooses the most optimal distribution of a certain fixed number of Monte Carlo samples in the image space, minimizing the error according to the MSE metric [Lehmann and Casella 1998]. Since it is impossible to compute the MSE without a reference image, an approximation is used [Silverman 1986]. The image is then reconstructed from the computed samples: the computed image is blurred with a predefined set of filters (Gaussian blur with different kernel sizes), and then for each pixel the value that minimizes the error according to the MSE is chosen. The error estimate from the current iteration is taken into account in the next iteration when choosing the distribution. Use of non-local filtering. The iterative method for computing and filtering Monte Carlo samples [Rousselle et al 2012] substantially accelerates the Monte Carlo ray tracing process and outperforms the method [Rousselle et al 2011] in quality. Each iteration is divided into three steps. In the first step, a fixed number of rays is computed, and the result is written to a buffer. Then, in the second step, non-local filtering of the buffer is performed using the nonlocal means denoising method [Buades et al 2005]. The third step computes the per-pixel residual, and the residual values are then used in the first step of the next iteration to distribute rays across the image space. For the first iteration, a uniform distribution is used. The method's performance is limited by the computationally complex non-local filtering algorithm; synthesizing 1 megapixel takes 8-16 seconds with GPU acceleration.
proposes a guided filter that processes an image guided by additional information about each pixel. With a well-chosen source of additional information, the guided filter outperforms bilateral filtering in quality. [Bauszat et al 2011] considers the application of guided filtering in Monte Carlo ray tracing. In the method considered, illumination is split into primary and secondary. Secondary illumination is processed using a guided filter, where the additional information used consists of a normal map, a depth map, and an additional color buffer obtained by computing rays passing through the pixel center. The proposed approach demonstrates better results, both in quality and in time, compared with cross-bilateral filtering [Kornprobst and Tumblin 2009] and A-Trous filtering [Dammertz et al 2010].
Treating the Monte Carlo ray tracing process as a Poisson process, the authors [Doidge and Jones 2013] propose an efficient way of filtering samples. For each pixel of the image, all samples are clustered by ray depth, and, based on the assumption of a Poisson process, it can be assumed that the distribution of samples in the cluster of one pixel can be compared with the distribution in the clusters of neighboring pixels, which allows the filter to reuse illumination from neighboring pixels. The filter also uses information about normals and depth to detect edges. The algorithm considered makes it possible to obtain an image with a small number of samples per pixel that surpasses cross-bilateral filtering [Kornprobst and Tumblin 2009] in quality. In the article [Sen and Darabi 2012] the authors presented a multidimensional filtering method that takes into account the stochastic dependence between random variables in the Monte Carlo ray tracing method and the values in the resulting sample. For each pixel, a vector is considered that includes a set of random variables and additional information about the scene (for example, the distribution of points on the lens). If the noise in the image is caused by the random distribution of points where the ray passes through the lens, then a certain functional dependence exists between the distribution of points on the lens and the noise in the image. Having such information makes it possible to reduce the contribution to the filter kernel of those pixels whose color was affected by noise. The algorithm proposes the use of bilateral filtering [Kornprobst and Tumblin 2009], whose parameters are computed based on the mutual information between the distribution of additional information at a point and the random variables.
[Pharr and Humphreys 2004], [Kontkanen et al 2006] solve the problem of accelerating image synthesis, but have artifacts and require substantial modification of the synthesis algorithm. The works [Hachisuka et al 2008], [Rousselle et al 2011], and [Rousselle et al 2012] describe the use of a more efficient sample distribution in screen space than uniform, but these approaches require modification of the ray tracer. Filtering methods make it possible to manage with smaller changes to the synthesis algorithm. The methods [Suykens and Willems 2000], [Kornprobst and Tumblin 2009], [Dammertz et al 2010] process images without information about the geometry, and, consequently, preserve image edges poorly. Similarly, the approaches [Gastal and Oliveira 2012], [Dammertz et al 2010], [He et al 2010], [Doidge and Jones 2013], [Sen and Darabi 2012] propose using additional scene information to improve filtering quality at edges. The method [Gastal and Oliveira 2012] makes it possible to combine the idea of guided filtering [He et al 2010] for preserving edges with non-local filtering [Buades et al 2005], which improves the processing of uniform regions.
Comments