Denoising camera data - Emerging Trends in Image Processing, Computer Vision, and Pattern Recognition - page 2

1 Introduction
While denoising is an extensively studied task in signal processing research, most denoising
methods are designed and evaluated using readily processed image data, e.g., the well-known
Kodak data set [ 1 ]. The noise model is usually additive white Gaussian noise (AWGN). This
kind of test data does not correspond nowadays to real-world image or video data taken with
a digital camera.
To understand the difference, let us review the color image capturing via a digital camera,
which is the usual way of image capturing nowadays. One pixel captures the light intensity,
thus the sensor data corresponds linearly to the lightness at the pixel position. To capture color
data, a color filter array (CFA) is used, which covers the pixels with a filter layer. Thus the
output of the sensor is a value that represents the light intensity for one color band at one
pixel position. This sensor data cannot be displayed before the following steps are applied: the
white balance, the demosaicking, which leads to a full color image and the color transforma-
tions, which adapt the linear data to displayable monitor data adapted to the monitor gamma
and color space. These steps lead to a noise characteristic that is fundamentally different from
the usually assumed AWGN: through demosaicking it is spatially and chromatically correl-
ated and through the nonlinear color transformations the noise distribution is unknown.
As this noise characteristic cannot be easily incorporated into the denoising methods, we
propose to apply the denoising to the raw CFA data—the mosaicked data linear to the light
intensity with uncorrupted noise characteristics. In the raw data we observe noise with a
known distribution and a signal-dependent variance, which can be precisely estimated based
on measurements [ 2 ]. However, despite the richness of denoising methods, denoising color
image raw data has been less studied until now. Hirakawa extended a wavelet-based meth-
od to CFA data [ 3 ] . Zhang proposed a principle component analysis (PCA) based solution [ 4 ].
The state-of-the-art method in image denoising, BM3D [ 5 ] , was combined with a noise estim-
ation algorithm and used for image denoising and declipping [ 6 ] . The later gives very good
results, which, however, come with a high computational cost. Nonlocal methods inherently
have high memory cost, as the whole image must be present in the internal memory. With
high resolution image sensors this can be a limiting aspect.
In this paper, we therefore propose a local method for raw data image and video denoising,
which builds on a shape-adaptive discrete cosine transform (SA-DCT) [ 7 ] . The method relies
on a homogeneous neighborhood for each pixel, and a thresholding operation, which elimin-
ates the noisy DCT coefficients. The neighborhood estimation prevents from oversmoothing,
which is the prevailing denoising drawback. Foi et al. adapted the method to signal-dependent
noise [ 8 ] , thus it can be easily used for the noise in raw data. However, the method is still not
adapted to linear and CFA data. We, therefore, propose to adapt and extend the method. We
calculate the neighborhood estimation on luminance data and we propose a luminance trans-
formation that can be directly applied to the CFA data. Additionally, we show how to adapt
the shape-adaptive DCT (SA-DCT) to Bayer data, as this is the most usual CFA, and describe
how the real noise characteristics from a digital camera can be obtained and included in the
method. We compare our solution both to Zhang [ 4 , 6 ] and evaluated the visual quality of im-
age and video data. Finally, we discuss the computational cost. While our first results are dis-
cussed in Ref. [ 9 ] , we add new results using real camera data from the ARRI image set [ 10 ] in
this chapter and show how the method can be improved for video sequences.