Expanding the Local Binary Pattern to Multispectral Images Using Total Orderings (Computer Vision,Imaging and Computer Graphics) Part 2

Results

We apply the LMBP operator to the problem of multispectral MR image segmentation problem. MRI has become a very useful medical diagnostic tool for several years, and development around MR imaging in general, and MR brain imaging in particular has recieved a lot of attention from both a methodological and an applicative point of view. One of the great advantages of MRI is the ability to acquire multispectral dataset of a single subject, by scanning with different combinations of pulse sequence parameters, that provide different contrasts between soft tissue allowing them to be fully characterized. Many segmentation methods either use monospectral or multispectral information [2]. Using these techniques, texture in MR images has proved to provide reliable information [13], but up to now only intra band texture information was used. We thus propose to incorporate in these method multispectral texture information stemming from LMBP.

As test data we used simulated MRI-datasets generated with the Internet connected MRI Simulator at the McConnell Brain Imaging Centre in Montreal (www.bic.mni.mcgill.ca/brainweb/). The datasets we used were based on an anatomical model of a normal brain that results from registering and preprocessing 27 scans from the same individual with subsequent semi-automated segmentation. In this dataset the different tissue types were well-defined, both “fuzzy” and “crisp” tissue membership were allocated to each voxel.

Table 1. MR Datasets

dataset no	dataset name	noise	RF
I	n Irf 20	1%	20%
2	nlrf40	I %	40%
3	n3rf20	3%	20%
4	n3rf40	3%	40%
5	n5rf20	5%	20%
6	n5rf40	5%	40%
7	n7rf20	7%	20%
8	n7rf40	7%	40%
9	n9rf20	9%	20%
10	n9rf40	9%	40%

From this tissue labeled brain volume the MR simulation algorithm, using discrete-event simulation of the pulse sequences based on the Bloch equations, predicted signal intensities and image contrast in a way that is equivalent to data acquired with a real MR-scanner. Both sequence parameters and the effect of partial volume averaging, noise, and intensity non-uniformity were incorporated in the simulation results [5,15].

Ten multispectral (T1-weighted,T2-weighted, Proton density) MR datasets of a central slice (including the main brain tissues, basal ganglia and fine to coarse details), with variations of the parameters ”noise” and "intensity non-uniformity (RF)” were chosen (table 1), the slice thickness being equal to 1mm. This selection covers the whole range of the parameter values available in BrainWeb so that the comparability with real data can be considered as sufficient to test the robustness of the method at varying image qualities.

For obtaining the true volumes of brain tissues and background the corresponding voxels were counted in the ground truth image provided by BrainWeb.

We performed three types of analysis for each dataset, first transformed by a subspace analysis method (namely the PCA). More precisely, we characterized pixels with several types of feature vectors:

-    either the vector of the first principal components, or a vector composed of the first principal components, the LMBP and the contrast operators (Interest of LMBP and Multispectral Contrast in the segmentation process, section 3.1).

-    either a vector composed of the first principal components, the LMBP and the contrast operators, or a vector composed of the first principal components, the multi-spectral and the contrast operators as computed in [29] (section 3.2)

In the following, we present results from dataset n9rf20 (figure 4) and use as LBP parameters R = 1.5,P = 12. All features were normalized by their variance. The neighborhood of each pixel was chosen as a square window of size 9.

Multispectral Texture Information

Figure 5 presents the results of the segmentation of the brain slice in 4 classes: background (BG, light gray), Cerebrospinal fluid (CSF, dark gray), white matter (WM, black) and gray matter (GM, white). S1 is the segmentation obtained with only the two first principal components, S2 with these two components plus the LMBP and the contrast operator values.

Fig. 4. Slice of interest of dataset n9rf20

Fig. 5. Segmentation results

In order to assess the two segmentations, we process the confusion matrix C on the segmented images: given a segmented image S, Cs(i,j) is the percentage of voxels assigned to class i G {BG, CSF, WM, GM} in the ground truth and to class j in S. The closer C to a diagonal matrix, the better the segmentation is. Although segmented images seems to be the same, the confusion matrix reveal some important differences:

Table 2. Number of pixels in each class

	BG	CSF	WM	GM
truth 51 52	19826 19620 19601	2336 3039 3088	8741 9755 9328	9577 8066 8463

Fig. 6. LBMP and Contrast images

The multispectral texture coefficients, and especially the LMBP operator, bring some information on structural properties on tissue edges: interfaces between tissues are more or less expressed depending on the acquisition (e.g. the CSF/(WM+GM) interface is very clear in T2-weighted images, whereas the proton density images better shows the WM/(GM+CSF) interface), and the multispectral structure of edges, seen as local oriented textures, is managed during the clustering process (figure 6). It is particularly visible for the WM/CSF interface between the corpus callosum and the lateral ventricles (see image and C(2,3) coefficient) and for the WM/GM interface near the putamens (see image and C(4, 3) coefficient).

Table 2 shows the number of pixels assigned to each class. Relative errors for S1 (respectively S2) are 0.01 (0.01) for background, 0.31 (0.31) for CSF, 0.11 (0.06) for WM and 0.15 (0.11) for GM. For WM and GM, relative errors lower due to the integration of LBMP operators.

LMBP vs. Another Multispectral Texture Definition

We also compared the LMBP operator with a multispectral LBP already proposed in the litterature [29], that computes LBP as:

where norms also stand in the definition of U. As in [29], we used the Euclidean norm, and the LBP parameters were chosen equal to those of LMBP. Note that this method is a particular case of LMBP, with

Fig. 7. Comparison of multispectral LBP and Contrast images

Fig. 8. Comparison of segmentation results

Figure 7 shows LMBP and Contrast images for both methods, and figure 8 compares the two segmentations. Results were always much better using LMBP for all the MR volumes described in Table 1, and the difference increased as the noise increased in the image. The functionused    to produce the total ordering in R" indeed tended to increase the LBP value in a noisy neighborhood of a voxel gc, producing the noisy LBP image of figure 7.

Conclusions

We proposed in this article a multispectral version of the classical Local Binary Pattern operator, based on a total ordering on the vectorial space of the data. We demonstrate its efficiency on multispectral MR images of the brain, assessing the results with respect to a ground truth, and comparing segmentation results with those provided by another multispectral LBP approach.

Numerous perspectives are now expected from this preliminary work. First of all, h needs to be better defined to allow the local topology to be preserved: two neighbors in Rn need to stay close when transformed by h. For x and y neighbors in Rn, the solution may be to define h using space filling curves directly on the multispectral image, in order to impose small variations of h(x) — h(y) in areas of interest, and higher variations for example in the background.

The segmentation scheme also needs to be refined. For this study, standard techniques (PCA, Kmeans) were applied, and some work has now to be done to tune subspace analysis and segmentation methods to this specific problem. Finally, this mul-tispectral approach finds natural applications not only in medical imaging, but also in remote sensing imagery. We now intend to tune and apply the LMBP to this domain.