Analysis of Ancient Mosaic Images for Dedicated Applications (Digital Imaging) Part 1

Introduction

Mosaic is a form of art to produce decorative images or patterns out of small components. It requires a great deal of time and labor and it is often very expensive. Mosaic is, in return, far more durable in time than painting, so it can be used in places where painting cannot be of practical use, such as floors. Mosaic moreover allows the realization of light effects that are impossible with other media. The success of the mosaic art form through the ages is at the origin of large collections in several museums; among them we can cite the Great Palace Mosaic Museum (Istanbul, Turkey) and the Musee du Bardo (Tunis, Tunisia).

Recently, questions have arisen about the virtual conservation and management of collections and their accessibility to experts such as archeologists or art historians, and even to the larger public. Requirements concern facilities

•    for cataloging collections and distant consultation, and for intelligent retrieval tools (e.g., the extraction of objects with a semantic meaning, such as animal or human, in a complex mosaic scene);

•    for virtual conservation (e.g., restoration of tesserae color), reconstruction (e.g., missing patches), and anastylosis of ancient mosaics.

The aim of this topic is to draw an overview of image processing-based methods dedicated to the specificities of mosaic images for applications to restoration and cataloging of ancient mosaics.

Some Historical Facts about Mosaics

The following description only intends to give some basis information about the history of mosaics.The worldwide web also gives access to very complete lists of bibliographical references on ancient mosaics (e.g., [44]).

The earliest mosaics were installed as floors or pavements, occasionally as walls. Later, ceilings were also decorated. Originally, mosaics were used strictly in an architectural context. Mosaicking smaller, portable items such as panels or portraits is a later development, though there are a few rare instances of items such as icons from the late Byzantine era. The first mosaics that we know of are from what is now the Middle East. Temple columns in ancient Babylon (present-day Iraq) had thousands of small clay cones pressed into wet plaster in decorative and geometric patterns. These date from five thousand years ago. From these humble beginnings, mosaic developed into a major art form. It was the Greeks, in the four centuries BC, who raised the pebble technique to an art form, with precise geometric patterns and detailed scenes of people and animals. By 200 BC, specially manufactured pieces, “tessera,” were being used to give extra detail and range of color to the work. Using small tesserae, sometimes only a few millimeters in size, meant that mosaics could imitate paintings. Many of the mosaics preserved at, for example, Pompeii were the work of Greek artists. The expansion of the Roman Empire took mosaics further afield, although the level of skill and artistry was diluted. Typically Roman subjects were scenes celebrating their gods, domestic themes, and geometric designs.

With the rise of the Byzantine Empire from the 5th century onwards, centered on Byzantium (present-day Istanbul, Turkey), the art form took on new characteristics. These included Eastern influences in style and the use of special glass tesserae called “smalti,” manufactured in northern Italy, made from thick sheets of colored glass. Smalti have a rough surface and contain tiny air bubbles. They are sometimes backed with reflective silver or gold leaf. Whereas Roman mosaics were mostly used as floors, the Byzantines specialized in covering walls and ceilings. The smalti were ungrouted, allowing light to reflect and refract within the glass. Also, they were set at slight angles to the wall, so that they caught the light in different ways. The gold tesserae sparkle as the viewer moves around within the building. Roman images were absorbed into the typical Christian themes of the Byzantine mosaics, although some work is decorative and some incorporates portraits of Emperors and Empresses.

In the west of Europe, the Moors brought Islamic mosaic and tile art into the Iberian peninsula in the 8th century, while elsewhere in the Muslim world, stone, glass, and ceramic were all used in mosaics. In contrast to the figurative representations in Byzantine art, Islamic motifs are mainly geometric. Examples can be seen in Spain at the Great Mosque at Cordoba, the Alhambra Palace in Granada, and Meknes in Morocco (Figure 14.4(b)). In Arabic countries a distinctive decorative style called “zillij” uses purpose-made ceramic shapes that are further worked by hand to allow them to tessellate (fit together perfectly to cover a surface).

In the rest of Europe, mosaic went into decline throughout the Middle Ages, although some tiling patterns in abbeys, for example, used mosaic effects. In the 19th century there was a revival of interest, particularly in the Byzantine style, with buildings such as Westminster Cathedral in London and Sacre-Cœur in Paris. The “Art Nouveau” movement also embraced mosaic art. In Barcelona, Antoni Gaudi worked with Josep Maria Jujol to produce the stunning ceramic mosaics of the Guell Park in the first two decades of the 20th century. These used a technique known as trencadis in which tiles (purpose-made and waste tiles) covered surfaces of buildings. They also incorporated broken crockery and other found objects, a revolutionary idea in formal art and architecture.

Mosaic still continues to interest artists but also crafts people because it is a very accessible,non-elitist form of creativity. The field is rich with new ideas and approaches, and organizations such as the British Association for Modern Mosaic (BAMM) [2] and the Society of American Mosaic Artists (SAMA) [1] exist to promote mosaic. As a recent example, we can cite the development of algorithms for computer-aided generation of mosaic images from a raster image, simulating ancient or modern styles [6].

Mosaics, Images, and Digital Applications

Mosaics are made of colored tiles, called tessera or tessella, usually formed in the shape of a cube of materials separated by a joint of mortar. The earliest known mosaic materials were small cones of clay pressed into wet plaster. Semi-precious stones such as lapis lazuli and onyx, as well as shells and terra cotta were also used. As the art developed, glass, ceramic, and stone tesserae were the most common materials, along with pebbles. Modernly, any small singular component can be used: traditional materials, glass or ceramic cast or cut into tiles, plus plastic, polymer clay (such as Sculpey or Fimo), beads, buttons, bottle caps, pearls, etc. The physical study of materials used in mosaics continues to be an active research field [10,17,38].

Structure Characteristics

Mosaics can be built on different natural ground made of soil or rock, or on top of a previous pavement. The mosaic itself is composed of a variety of foundation or preparatory layers and a layer of tesserae. Also, the mosaic surface exhibits irregular hollows (tesserae) and bumps (mortar) through the scene. Hence, a mosaic can not be considered as a plane surface, as for paintings, but has numerous forms of irregularities, typical of the artwork style:

•    Shape of tesserae. The shapes of tesserae are irregular, from square shapes to polygonal ones.

•    Organization of tesserae. Tesserae are not positioned according to a regular lattice. On the contrary, the smart and judicious choice in the orientation, size, and positioning of tesserae characterize the artwork style and exhibit the “general flow” (guidelines) of the mosaic chosen by the mosaicist.

•    Mortar joint. This positioning makes the joints appear as an irregular network with numerous interconnections throughout the mosaic. Network color intensity, mainly middle gray, is however not uniform through the scene.

•    Color of tesserae. Because of materials used and their oldness, tesserae of ancient mosaics are generally characterized by pastel colors, with low contrast.

Oldness Characteristics

Other artifacts can be associated to the wear, erosion, and oldness of ancient mosaics (some of them are illustrated by images in Figures 14.1 and 14.2):

•    Disaggregated tesserae. Some tesserae can display a loss of cohesion of their surfaces, disintegrated into powder or small grains. Also erosion and patinae deteriorations are commonly encountered in studying ancient mosaics.

•    Missing tessera patches. Most mosaics are not completely preserved and, as a consequence of disaggregation, lacunas generally corrupt mosaic scenes.

•    Color alteration. Alteration of the mosaic surface characterized by a localized change in color (due to fire damage, graffiti, etc.).

Data Acquisition

For later processing and virtual anastylosis, mosaic scenes must first be digitized using a camera. All peculiarities cited above have a strong and complex impact on acquisition and the way mosaic scenes appear in an image with a limited resolution:

FIGURE 14.1

Excerpt of an ancient Greek mosaic scene (from Paphos, Cyprus), showing missing patches and geometric deformation due to slant acquisition [25].

FIGURE 14.2

Three examples of deteriorated tesserae, from [34].

•    Relief shadows. During snapshot acquisition, the relief of tesserae generates shadows on mortar which does not appear nearly uniform in intensity all over the image, as it should.

•    Indoor/outdoor acquisition. Previous item is further accentuated by indoor acquisition with flash. In outdoor acquisition, light is not controlled and one should expect severe contrast variations in acquisitions.

•    Geometric deformation. Another degradation comes from the snapshot acquisition angle, especially for very large mosaic scenes lying on the floor. This point produces perspective (rotation, scaling , and shearing) deformations of tesserae shape in image and the thinning down of the mortar width (until disappearing).

•    Tessera resolution. The resolution of tesserae in an image (i.e., the number of pixels to describe a tessera) should not be too low for later processing. On the other hand, a too high resolution will only be able to capture a partial scene.

As a consequence, very large mosaic scenes have to be recorded using multiple overlapping images that are subsequently co-registered in geometry and color to produce the entire digital scene. An interesting complementary approach to camera acquisition is given by laser scanning. This technique, used recently for ancient mosaics analysis [23,37], can produce a 3D mesh of the mosaic surface and material for a detailed shape analysis.

Hence, dedicated processing methods are required to take into account the specificities of mosaic and the specificities of mosaic images, and to adopt image-processing strategies suited to the tiling organization of tesserae. This way, one can expect better performances than general purpose algorithms. The nature of the artifacts in mosaic images makes segmentation methods based on pixel intensities inefficient (e.g., pixels associated to the mortar interfere with the classification process of tesserae). A strategy better suited to mosaic images is to consider tesserae as indivisible entities with an almost uniform gray-level value.

Section 14.2 gives a brief description of several projects involving image-processing methods for restoration, preservation, and indexation of ancient mosaics. Then we present in detail some fundamental applications of the tessera-oriented point-of-view on mosaic images proposed by the authors, according to the diagram in Figure 14.3. Section 14.3 presents an effective way to extract tesserae, while Section 14.4 exposes two direct applications of this strategy for mosaic image segmentation and coding. Section 14.5 presents an effective way to retrieve the main guidelines of a mosaic which can be helpful for delimitating semantic objects present in a complex scene. Finally, the conclusion in Section 14.6 examines some open issues and suggests future research directions to succeed in providing appropriated tools to museums and experts.

Recent Image-Processing Projects Concerned with Mosaics

A few projects involving computer science and image processing for mosaic conservation, restoration, or cataloging are reported in the literature. Here is a short description of them.

St. Vitus Cathedral Mosaic Restoration

The more accomplished project concerns the restoration of the St. Vitus cathedral mosaics, in Prague (Czech Republic), reported in [49]. The Golden Gate (the southern entrance to the cathedral) is decorated with a unique work of art: a colored, richly gilded mosaic representing the Last Judgement, see Figure 14.4(a). In 1992 the Office of the President of the Czech Republic and the Getty Conservation Institute [3] began to cooperate to restore and conserve the mosaic.

FIGURE 14.3

Organization of topic with the processing pipeline.

FIGURE 14.4

(a) Photo of the “Last Judgement” mosaic in the St. Vitus cathedral (Prague, Czech Republic), from [4].

Having the historical photo of the mosaic from the end of the 19th century and the photos of the current state, authors study the evolution of the mosaic, which was several times reconstructed and conserved. First they tried to restore the historical photograph – remove noise, deblur the image, increase the contrast. Then, they removed the geometrical difference between images by means of the multi-modal registration using mutual information. Finally, they identified mutual differences between the photos, which indicate the changes on the mosaic during the centuries.

Arabo-Moresque and Islamic Mosaic Pattern Classification

Lot of attention has also been paid to Islamic mosaics, the artwork style of which is illustrated in Figure 14.4(b). The particularities of such mosaics come from the periodicity and symmetry of tile patterns.

A. Zarghili et al., [47,48] propose a method to index an Arabo-Moresque decor database which is not based on symmetry. They use a supervised mosaicking technique to capture the whole principal geometric information (connected set of polygonal shapes, called “spine”) of a pattern. The spine is then described by using Fourier shape descriptors [35] to allow retrieving of images even under translation, rotation, and scale. But, according to [14], the method cannot be automatized and does not allow the classification of these patterns according to any criterion.

In [21], the authors propose image-processing techniques to restore mosaic patterns. Image analysis tools are developed to obtain information about design patterns which are used to recover missing motifs or tesserae. One difficulty was to propose methods robust to the discrepancies between equal object shapes (due to manual artwork, oldness, etc.). The symmetry, once recovered, allows virtual reconstruction by inpainting methods and physical restoration of damaged parts of mosaics.

To study Islamic geometrical patterns, and all periodic patterns such as those encountered in textile patterns [41] or wallpapers, several works are based on the symmetry group theory. In [14], the authors first classify patterns into one of the three following categories: (1) pattern generated by translation along one dimension, (2) patterns which contain translational symmetries in two independent directions (refers to the seventeen wallpaper groups), and (3) “rosettes,” which describes patterns that begin at a central point and grow radially outward. For every pattern, authors extract the symmetry group and the fundamental region (i.e., a representative region in the image from which the whole image can be regenerated). Finally, they describe the fundamental region by a simple color histogram and build the feature vector, which is a combination of the symmetry feature and histogram information. The authors show promising experiments for either classification or indexing.

In [15], the authors exploit symmetry and auto-similarity of motifs to design a system to index Arabo-Moresque mosaic images based on the fractal dimension. Mosaics are first segmented automatically using color information. The classification decomposes the initial motif into a set of basic shapes. Contours of those shapes are then characterized by their fractal dimension which gives, according to the authors, a relevant measure of the geometric structure of the tile pattern. Some retrieval performances are also reported.

Roman Mosaics Indexation

Recently, two Content-Based Image Retrieval (CBIR) systems have been proposed to catalog and index Roman mosaic images.

The first one, proposed in [32], details a complete CBIR system which includes (1) object extraction from a complex mosaic scene by using unsupervised statistical segmentation and (2) invariant description of semantic objects using the analytical Fourier-Mellin transform [13,20]. Similarity between querying mosaic and the database is based on an index constructed from the invariant descriptors and an appropriate metric (Euclidean and Hausdorff).

The second CBIR [27,28] is a general system to index and retrieve by the content historic document images. While the object annotation in database images is done manually and off-line, the indexation is done automatically using an extended curvature scale space descriptor [33] suitable for concave and convex shapes. The query/retrieval of pertinent shapes from the database starts with a user drawing query (with a computer mouse or a pen) that is compared to entries in the database using a fuzzy similarity measure. The system integrates an XML Database conforming to the MPEG7 standard, and experiments on large databases provided by the National Library of Tunisia and some Tunisian museums are reported.

FIGURE 14.5

Excerpt of an ancient mosaic showing a wild boar (a) and a zoom on its hind legs (b).

One key point for such systems to be effective is to improve semantic objects extraction. Pixel-based methods, as the one used in [32], require heavy post-processing to detect shapes of interest from the pixels class. Recently, a new viewpoint on mosaic images, based directly on tesserae, has been proposed [8]. Once tesserae extraction is achieved, this strategy facilitates basic steps toward CBIR applications, as described in the following.

Tesserae Extraction

A “natural way” to describe mosaics is to consider tesserae in their own instead of groups of isolated pixels, allowing the development of tessera-oriented processing. Indeed, a mosaic image can be seen as an irregular lattice in which a node is a tessera (Figure 14.5). The grid structure has been super-imposed to the zoomed image in Figure 14.5(b) to illustrate the complexity of the neighborhood system, both in the number of neighbors and in their orientation. To adopt this point of view on mosaic images, a robust method is required to extract tesserae from the network of mortar surrounding them.

A way to consider this problem is to deal with the dual problem (i.e., the extraction of the network of mortar) which is close to recurrent problems encountered in several image-processing applications (i.e., in medical imaging: vascular network segmentation from angiographies [12,46], or in satellite imaging: road extraction in urban scenes [29, 30]).

Several approaches have been proposed. Methods based on contour extraction are widely used and mainly rely on the assumption that the network pixels and neighboring ones have different gray levels in order to compute gradients. But methods based on high-pass filter, such as Harris’s corner detector, highlight pixels belonging to the network, not connected components. Higher-level processings detect lines with varying widths [18,39]. Strategies that track the entire network from a starting point [7,45] are difficult to justify in the mosaic image case due to the high number of interconnections in a mosaic network graph.

FIGURE 14.6

Processing chain involved in tessera extraction, using gray-scale morphology tools.

Numerous methods based on Markov modeling [19,40] or active contours [36,42] have also been proposed. These methods are quite efficient but time consuming. In the case of mosaics, these methods are not suited because of the high density of the network to be extracted in images. In [19], a Markov model is applied on a graph of adjacency crests, detected by a Watershed Transformation applied on a criterion image. This criterion image, computed from the original one, exhibits the potential of each pixel to belong to the network.

A study of tesserae extraction has been conducted in [8]. The solution adopted, and preferred over other experimented strategies, is based on gray-level morphology which is suited to the tiling organization of mosaics. The Watershed Transformation (WT) approach [31] appeared interesting for mosaic images since this method is a good compromise between low-level methods (contour detection) and approaches by energy minimization (Markov model or active contours). But to work well, the WT needs to be computed on a criterion image (processed from the original one) that shows tesserae as catchment basins and the mortar network as watershed crests.

The entire algorithm is sketched in Figure 14.6. The goal is to present to the WT a potential image that exhibits tesserae as catchment basins and the network as a crest crossing the entire image. The pre-processings are: (1) contrast enhancement, based on top-hat and bottom-hat transforms; (2) potential criterion computation to exhibit the network as a crest; and (3) area closing [43] to reduce over-detection by WT.

The extraction quality greatly depends on the way mosaic images have been acquired. It should not be taken too far from the mosaic since individual tiles will not be visible: according to experiments, tesserae should appear with a resolution of not less than 10 x 10 pixels. Parameter h is a threshold used by the area closing operator [43]. Its value has an impact on the number of extracted tiles: a small value gives an over-detection whereas a big one produces an under-detection one. Hence, it should be (roughly) adjusted according to the mean size α of tesserae in the image (in number of pixels), assuming that α is almost constant in a mosaic. Parameter h should be less than to avoid small tesserae to be deleted by the morphological operator. In experiments, value h = α2/2 gives good results.

Figure 14.7 shows the criterion image obtained by applying the method to the wild boar image.

FIGURE 14.7

Criterion image obtained from Figure 14.5. This processing improves the contrat between tessera and the network of mortar.

FIGURE 14.8

Extraction of tesserae from the criterion image in Figure 14.7, without (a) and with (b) area closing operator.

As can be seen in this example, the network appears in dark. However, tesserae are not uniform in texture and show local gray-level crests that should be deleted before WT in order to avoid over-segmentation. Following [19], an area closing [43] is first computed on the criterion image. This processing gives fewer minima while retaining crest locations as illustrated in Figure 14.8. The crest contours now represent correctly the network, which is confirmed by the zoom in Figure 14.9(a). To determine the width of the network (and not only a one-pixel skeleton as done by WT), which varies through the image, a simple threshold is applied on neighboring pixels of crests: a pixel is aggregated to the crest if its gray value is different by no more than 10% of the skeleton mean gray value. The result of applying such a threshold can be observed in Figure 14.9(b).

A second example of tesserae extraction based on gray-level morphological processing is given in Figure 14.10.

FIGURE 14.9

Result of tesserae extraction on the zoom in Figure 14.5(a) in (a), and network/tesserae classification in (b).

FIGURE 14.10

Illustration of the tesserae extraction algorithm on the mosaic image of a deer.

FIGURE 14.11

Segmentation comparison of the mosaic image in Figure 14.5(a) with a pixel-based strategy (a) and the tessera-based strategy presented in Section 14.3 (b) Both pixel- and tile-based segmentations make use of an unsupervised K-means classification algorithm, the first one with three classes, and the second one with only two classes since the mortar is already extracted.