In the previous topic we saw that a pixel value in the output was set according to a pixel value in the input at the same position and a point processing operation. This principle has many useful applications (as we saw), but it cannot be applied to investigate the relationship between neighboring pixels. For example, if we look at the pixel values in the small area in Fig. 5.1, we can see that a significant change in intensity values occurs in the lower left corner. This could indicate the boundary of an object and by finding the boundary pixels we have found the object.
In this and the next topic we present a number of methods where the neighbor pixels play a role when determining the output value of a pixel. Overall these methods are denoted neighborhood processing and the principle is illustrated in Fig. 5.2. The value of a pixel in the output is determined by the value of the pixel at the same position in the input and the neighbors together with a neighborhood processing operation. We use the same notation as in the previous topic, i.e., f(x, y) is the input image and g(x,y) is the output image.1
The Median Filter
If we look at Fig. 5.3 we can see that it has been infected with some kind of noise (the black and white dots). Let us set out to remove this noise. First of all we zoom in on the image and look closer at particular pixel values. What we can see is that the noise is isolated pixels having a value of either 0 (black) or 255 (white), such noise is denoted salt-and-pepper noise. By isolated we mean that they have a value very different from their neighbors. We need somehow to identify such pixels and replace them by a value which is more similar to the neighbors.
One solution is to replace the noise pixel by the mean value of the neighbors. Say we use the eight nearest neighbors for the noise pixel at position (1,1) in the image patch in Fig. 5.3.
Fig. 5.1 A part of the giraffe-image has been enlarged to show the edge which humans easily perceive.
Fig. 5.2 The principle of neighborhood processing. To calculate a pixel in the output image, a pixel from the input image and its neighbors are processed
This results in the noise pixel being replaced by 181, which is more similar to the neighbors. However, the value still stands out and therefore the median is often used instead. The median value of a group of numbers is found by ordering the numbers in increasing order and picking the middle value. Say we use the eight nearest neighbors for the first pixel infested by noise in Fig. 5.3. The ordering yields
and the middle value is 204, hence the median is 204. The noise pixel is now replaced by 204, which does not stand out.
Fig. 5.3 An image infected with salt-and-pepper noise. The noise is easily recognized in both the image- and the number representations
The next question is how to find the noise pixels in order to know where to perform the median operation. For the particular example we could scan the image pixel-by-pixel and look for isolated values of 0 or 255. When encountered, the median operation could be applied. In general, however, a pixel with a value of say 234 could also be considered noise if it is isolated (stands out from its neighbors). Therefore, the median operation is applied to every single pixel in the image and we call this filtering the image using a median filter. Filtering the image refers to the process of applying a filter (here the median filter) to the entire image. It is important to note that by filtering the image we apply the filter to each and every pixel.
When filtering the image we of course need to decide which operation to apply but we also need to specify the size of the filter. The filter used in Fig. 5.2 is a 3 x 3 filter. Since filters are centered on a particular pixel (the center of the filter) the size of the filter is uneven, i.e., 3, 5, 7, etc. Very often filters have equal spatial dimensions, i.e., 3 x 3,5 x 5, 7 x 7, etc. Sometimes a filter is described by its radius rather than its size. The radius of a 3 x 3 filter is 1, 2 for a 5 x 5 filter, 3 for a 7 x 7 filter etc. The radius/size of a filter controls the number of neighbors included. The more neighbors included, the more strongly the image is filtered. Whether this is desirable or not depends on the application. Note that the larger the size, the more processing power is required by the computer. Applying a filter to an image is done by scanning through the image pixel-by-pixel from the upper left corner toward the lower right corner, as described in the previous topic. Figure 5.4 shows how the image in Fig. 5.3 is being filtered by a 3 x 3 (radius = 1) mean and median filter, respectively. Note the superiority of the median filter.
In terms of programming, the Median filter can be implemented as illustrated below—here exemplified in C-code:
Fig. 5.4 Resulting images of two noise filters. Notice that the mean filter does not remove all the noise and that it blurs the image. The median filter removes all the noise and only blurs the image slightly
where M is the height of the image, N is the width of the image and Radius is the radius of the filter.
What should be noticed both in the figure and in the code is that the output image will be smaller than the input image. The reason is that the filter is defined with a center and a radius, but if the center is a pixel in for example the first row, then no neighbors are defined above. This is known as the border problem, see Fig. 5.5. If it is unacceptable that the output image is reduced in size (for example because it is to be added to the input image) then inspiration can be found in one of the following suggestions2:
Increase the output image After the output image has been generated, the pixel values in the last row (if radius = 1) is duplicated and appended to the image. The same for the first row, first column and last column.
Increase the input image Before the image is filtered the pixel values in the last row (if radius = 1) of the input image is duplicated and appended to the input image. The same for the first row, first column and last column.
Apply special filters at the rim of the image Special filters with special sizes are defined and applied accordingly, see Fig. 5.5.
Fig. 5.5 An illustration of the border problem, which occurs when using neighborhood processing algorithms. If a kernel with a size of 3 x 3 is used, then the border illustrated in f(x, y) cannot be processed. One solution to this is to apply kernels with special sizes on the borders, like the ones showed to the right
Rank Filters
The Median Filter belongs to a group of filters known as Rank Filters. The only difference between them is the value which is picked after the pixels have been sorted:
The minimum value This filter will make the image darker.
The maximum value This filter will make the image brighter.
The difference This filter outputs the difference between the maximum and minimum value and the result is an image where the transitions between light and dark (and opposite) are enhanced. Such a transition is often denoted an edge in an image. More on this in Sect. 5.2.2.
