Cascaded Active Contours
The cells in our microscope images show characteristic spatial intensity distributions that cannot adequately be modeled by a Gaussian distribution as underlying Er in equation (10). Rather spatial changes within the intensities need to be appropriately covered. Figure 4 (a) shows that the average intensities within the cells decrease from central nucleus regions towards outer sections of the cells and the background.
Instead of adapting the intensity model for cell regions according to the spatial intensity profiles observed, which is quite difficult as not simply more sophisticated intensity distributions like Gaussian mixtures can be used, we tackle this problem introducing a cascaded segmentation approach. Starting with initial nucleus region contours these are iteratively expanded towards the outer sections of the cells whereas in each of three iteration levels additional fractions of the cell areas are included.
In Figure 5 a diagram of the cascaded segmentation algorithm with its three levels is shown. The initial nucleus contour segmentation is accomplished using a global Otsu’s thresholding followed by morphological closing. For each connected component a cell is hypothesized and its contour yields an initial snake.
Fig. 5. Overview of the cascaded segmentation approach
The main idea of the iterative procedure is to drive the snake further towards the outer sections of a cell in each level given that it converged in the previous level, i.e. reached a stable energy state and does not show any dynamics anymore. Accordingly, to set the snake in motion again, at the beginning of each segmentation level l the interiors of the initial snake contours for this level are dilated by 10 pixels. This dilation is constraint to avoid overlap between neighboring snakes. Subsequently from each expanded snake region of approximately 10 pixel width a new initial average intensity is extracted for each snake cn(s) and used as initial constant μιη for subsequent optimization.
The new initial average intensity tends to be smaller than μ^ ‘ extracted during the previous segmentation level since the average intensity within the expanded region fraction will most presumably be smaller than the average intensity inside the contour. Thus, the snake is driven to evolve towards the outside for integrating the new darker regions into the cell region. Each snake finally converges to a position where its overall energy is minimized according to Eq. (12). This result is subsequently used as contour initialization in the subsequent segmentation level and drives the snake outwards until it finally stops at the border to the background.
Results
In the following we discuss experimental results of our cascaded segmentation approach. To this end we compare automated cell segmentation with manually labeled ground-truth data. Furthermore the assignment of PBs and SGs to individual cells is assessed using automatic and ground-truth cell segmentation.
Image Data
The algorithm was evaluated on 5 sample images obtained using a common experimental setup. The cells stem from the human hepatoma cell line HUH7. Each image consists of three channels, containing fluorescently labeled nuclei, SGs and PBs, respectively. The nuclei are labeled by DAPI. SGs are labeled by immunostaining of TIAR (a protein localized in SGs), while PBs are labeled by immunostaining of DCP1a (decapping enzyme localized in PBs).
Fig. 6. Cropped image section showing the evolution of the snakes starting with initial snakes extracted from nucleus regions (a), and the results of subsequent optimization levels 1 to 3 (b-d)
The five images include a total of 86 cells manually labeled as ground-truth. Initial snakes for the snake segmentation are derived from detected cell nuclei as explained above. It may happen during nuclei detection that nuclei located very close to each other are fused into a single nucleus region. In such cases snakes are missing, i.e. not all ground-truth cells can properly be segmented with the given set of initial snakes. To enable a fair evaluation of the snake segmentation approach and taking into account that the automatic nucleus detection is not part of this contribution, the labeling of ground-truth cells was adapted accordingly for such cells. In detail, if two or three nuclei merged during automatic segmentation also the cells involved were merged in manual ground-truth labeling. After this correction, 77 ground-truth cells are available for evaluation.
Segmentation Results
The cascaded segmentation algorithm was applied with three levels, each with its own parameter settings (Tab. 1). Note that for all images and each of the three levels the same set of parameters was applied.
When optimizing snakes in practice one important issue to ensure accurate and comparable localization properties at each point position along the contour is to keep the distance between subsequent points of the snake, i.e. the length of the snake segments, more or less uniform. In our approach the snake is parameterized with the desired segment length lseg, and every second iteration the segments are checked and in case of large deviations from the optimum segment length points are added or deleted accordingly.
The snake is supposed to stop in a locally optimal energy state, i.e. a position of minimal snake energy. In our setup we use a combination of two stopping criteria to determine if the snake converged. On the one hand we analyze the relative change in the area of snake interior between two iterations and stop optimization if it falls below a certain threshold value ΔA. In addition, a maximum number of admissible iterations Imax is defined mainly to prevent the snake from oscillating between two steady states with difference in area size just above the threshold ΔA.
Table 1. Parameters used during snake segmentation
|
Level: |
1st |
2nd |
3rd |
|
λ in |
0.1 |
||
|
λ out |
25 |
||
|
P |
10000 |
||
|
a |
1.25 |
||
|
β |
0.75 |
1.25 |
|
|
7 |
0.0001 |
0.0002 |
|
|
Iseg |
15 |
5 |
|
|
ZiA |
0.015 |
0.005 |
0.001 |
|
Imax |
100 |
200 |
|
Fig. 7. Cell segmentation result on complete image, final contours are shown in white, initial nuclei contours in gray
In general, during the first two levels the overall shape of the cells is extracted. In level 3 accurate local segmentation and local improvements with regard to low intensity regions get higher priority. Accordingly the weight of the curvature term is increased, the length of the contour segments is decreased and the stopping criteria are adapted.
Prototypical segmentation results of our approach are shown in Figs. 4, 6 (d), and 7. The two image clips in Fig. 4 (b) and (c) show that the overall segmentation result is appropriate . Especially segmentation of cell boundaries in conglomerating sets of cells is of high quality. In outer sections of the cells sometimes fractions of the cell area are still missing. However, as the figures below indicate the missing sections only marginally degrade the quality of the results. This is due to the fact that for our application the main focus is the correct assignment of PBs and SGs to cells, and the distribution of their numbers and localization with respect to the cells and their nuclei.
The main idea of our 3-level approach is to iteratively improve segmentation results. During level one large parts of the cell are still missing, while in latter levels almost the complete cell area is included in the segmentation result. This is exemplary shown in Figure 6 for a clip of one image which also demonstrates the incremental improvement of cell segmentation, which cannot be achieved in one single snake segmentation run. Fig. 7 shows final cell contours as extracted by our new approach for a complete image of the test set emphasizing again the high performance of the algorithm in adequately separating conglomerating cells.
Fig. 8. Average recall (left) and precision (right) values for all 77 cells
Cell segmentation can be regarded a classification task on the pixel level. For optimal results each pixel of a labeled ground-truth cell should be segmented by the algorithm and correctly classified into the correct cell class. These pixels correctly classified as cell pixels are the true positives (TP), while pixels incorrectly classified as cell pixels are false positives (FP). Pixels incorrectly classified as background or belonging to other cells are false negatives (FN). Based on this interpretation for each ground-truth cell, precision and recall can be calculated. The recall TP/ (TP + FN) is defined as the ratio of ground-truth pixels that were actually correctly detected by the segmentation algorithm, while the precision TP/(TP + FP) gives the ratio of detected cell pixels that are actually lying inside ground-truth cells. In Figure 8 box-plots for the recall and precision achieved are presented. For each of the three segmentation levels the values are shown, in each case averaged over all 77 cells.
The median of recall values steadily increases from 0.66 after the first segmentation level to 0.79 and finally 0.91 after the last level. This is due to the fact that beginning with the initial nucleus region in each level the algorithm segments larger fractions of the cells. For the precision the tendency is just conversely. Since the initial cell regions are derived from the nuclei segmented, the contours from the first level predominately lie completely within the surrounding cells, i.e. very few false negatives appear in the segmentation. With evolving snake contours the cell area grows, and consequently the chance to include pixels into the snake region that actually belong to the background or other cells increases. The precision decreases from an initial median of 0.99 to a final value of 0.92. Note that for more than 75% of the cells the value still exceeds 0.84.
In both plots outliers with recall and precision values close to zero can be observed. In part these result from very small cells that become enclosed into the area of significantly larger cells in their neighborhood during segmentation. This results in a significant amount of false positives and consequently small precision. On the other hand one outlier cell with very low recall values relates to a ground-truth cell located near the image boundary where only small fractions of the cell nucleus are visible. Consequently the initial snake contains only very few polygon points and rapidly collapses rather than to segment the cell. Altogether precision and recall indicate that the segmentation misses some fractions of several cells, but barely includes additional non-cell pixels.
Fig. 9. On the left, number of cooccurring PBs in ground-truth and segmented cells after the first segmentation level (cross) and in the final result (circle) are shown, and on the right number of cooccurring SGs in ground-truth and segmented cells after the first segmentation level (cross) and in the final result (circle)
From the biological point of view besides the actual automatic detection of PBs and SGs one very valuable information is the distribution of the particles with regard to the cells. In detail, different biological implications result if the particles are equally spread over all cells of a sample or quite inhomogeneously distributed in the cell population. Therefore we assess the assignment of the PBs and SGs to the different cells compared to ground-truth cell segmentation. In Figure 9 the corresponding scatter plots are shown.
On the abscissa of each plot the number of particles, either PBs or SGs, in a ground-truth cell is shown, on the ordinate the automatically detected number of PBs or SGs in the corresponding cell segmented via snakes is outlined. In case of optimal particle assignment from correctly segmented cell areas the plots should show a perfect bisecting line. In both plots the results after the first iteration are shown as crosses, while the final results are marked by circles. Note that in the setup of the biological experiment the number of SGs in the images is in general significantly smaller than the number of PBs. Nevertheless in both plots it is clearly visible that the cooccurence increases. Regarding SGs the Spearman rank correlation coefficient slightly increases from 0.852 to 0.888.
Also for PBs the majority of data points moves towards the bisecting line. The correlation value decreases from initially 0.891 to 0.853 and finally 0.832. The main reason for this are two collapse events during segmentation leading to wrong cell correspondences. In both cases at the beginning a set of three ground-truth cells is corresponding to exactly three snake contours. In the course of the segmentation, however, one of the snakes collapses and the cell is included in the area of a neighboring snake (cf. Fig. 10). Consequently a single segmented cell corresponds to two ground-truth cells, whereas only one of them is the correct one while the second ground-truth cell gets assigned many false positive detection results. In the plot in Fig. 9, left, the outlier point in the top left corner of the plot is caused by one of these events.
Fig. 10. Example for segmentation failure. While in level 2 (a) three snake regions model the ground-truth cells, in level 3 (b) one snake disappears allowing one of the remaining two to wrongly occupy its region as well. Note that the cells are hard to separate even manually (c).
Conclusions
For automatic analysis of images from fluorescence microscopy two methods are proposed in this work. For the detection of sub-cellular particles we extend an approach based on correlated wavelet coefficients from the a trous wavelet transform. To accommodate particles of varying size we introduce a set of scale intervals and resolve ambiguities in the resulting particle hypotheses by adapting the concept of meaningful events. Secondly, for the segmentation of cell areas with decreasing intensities towards their borders we extend coupled active contours into a cascaded segmentation technique. The core idea of this technique is to incrementally incorporate areas of decreasing intensities into the segmentation result, starting with the nucleus of each cell as initialization. We evaluate theses methods on a set of 5 images with cells from a human hepatoma cell line. Each image is composed of three channels labeling nuclei, SGs, and PBs, respectively. The results clearly show the benefits of the cascaded approach as the overall segmentation quality is high. This is true as well for precision and recall of cell areas as for the assignment of particles to their corresponding cells. Future work will include improvements of nucleus segmentation to avoid or correct fusion of neighboring nuclei. Furthermore, data-depended adaption of the number of levels used in the cascaded approach with an appropriate stopping criterium will be scrutinized.
