Using Digital 3D Models for Study and Restoration of Cultural Heritage Artifacts (Digital Imaging) Part 3

Classifying and Archiving Carved Faces: The Bayon Digital Archival Project

The Bayon digital archival project [25] is a very complex acquisition experience, which used several technologies [26] to acquire the extremely big (100 m length on each side by 43 m height at the most) architecture of the Bayon temple in Cambodia.

FIGURE 2.14

An example of a rapid prototyping project, developed by CNR-ISTI in collaboration with the company Scienzia Machinale (www.smrobotica.it). Left to right: the original artifact, the 3D model obtained using laser triangulation, the prototyping machine in action, the reproduction during an intermediate stage, and the final results.

Besides the importance of the acquisition campaign, we point out a peculiar use of the 3D data for the work of art historians. Throughout the temple, there are 173 stone faces, carved on the towers. They were classified in three groups (see Figure 2.13) and an automatic method assigned each of the acquired stone faces to one of the groups. In this simple example, the analysis of data helped the experts in classifying a complex set of data, without the need of working on the real environment, which is set in a very difficult to reach area.

Physical Reproduction from the Digital Model

An extremely interesting feature of 3D scanning and modern photogrammetry (and also of standard 3D modeling) is that the digital 3D model is obtained without any direct contact with the artifact. This is extremely important in the CH context, since the manipulation of the originals can easily produce damages. Once we have an accurate digital 3D model, another technology, rapid prototyping, can find many interesting uses in the CH domain. This is another case of a technology inherited from the industrial field. Rapid prototyping devices are able to create accurate reproductions of an object starting from the digital 3D model. The process is usually automatic and can be applied on a variety of materials (chalk, plastic, plastic with metallic coating, stones, etc.). As a major difference from the consolidated reproduction approach (production of rubber molds for the subsequent production of gypsum copies), digital reproduction allows one to obtain copies in any reproduction scale, or of just portions of the object. Like the other technologies in this field, the improvement in hardware and software components led to a rapid decrease in the costs.

An example of a practical application is shown in Figure 2.14. The subject of the work was a marble head of Mecenate (Archaeological Museum, Arezzo, Italy). CNR-ISTI was commissioned by the German Research Ministry for the production of an accurate marble copy, to be used in the context of the German “Mecenate” research project. The scanning was performed on the original object using a laser triangulation scanner, so that an extremely accurate model was produced. The 3D model was the input for driving the carving path of a robotized drilling system, which is able to sculpt a marble block with great accuracy and repeatability. After a final manual intervention for carving finer details and polishing the surface, a very detailed 3D reproduction of the original artifact was obtained.

Other applications, which do not use typical prototyping devices, have also been considered in the context of the reproduction from digital models. For example, laser cutting machines have been used to reproduce ancient astrolabes [27]. Hence, physical reproductions from 3D models can be used in several ways:

•    Temporary or permanent replacement of originals. If an artifacts has to be removed from its original position, it is possible to replace it with an accurate copy. The replacement can be temporary, for example when a museum lends an object to an external exhibition. In the case of severely damaged or fragile objects, it is possible to put them in a more protected environment (museum, controlled environments) by replacing them permanently with a copy. In this way, the visitor can come in contact with the environment in which the artifact was posed (note that from a medium distance, the difference between the original and the replica becomes not perceptible) and at the same time the original artifact can be protected and conserved.

•    Reconstruction and coloring hypotheses. An accurate copy of the object can be very useful if the restorers want to experiment and propose hypotheses about the original shape of missing parts, or about the original colors of polychromatic statues or decorations. While rapid prototyping devices are able to produce colored objects, and obtain also realistic and visually pleasing results [28], the re-coloring is usually made by hand to obtain full accuracy (accurate selection of color tints and layout of color). This peculiar application helps the restorers in their practical work, with the possibility to produce and compare several hypotheses.

•    Wide-scale accurate physical copies. A more commercial application is the possibility to obtain accurate small-scale copies of an artifact, for commercial purposes. This raises several issues about copyright and opportunity, but it is also an interesting option for funding the activities of a museum or a CH governing institution.

Virtual Reconstruction and Reassembly

Virtual Reconstruction

Virtual reconstruction has always been one of the most straightforward CH applications of 3D graphics. Besides the use of pure 3D modeling, which creates geometry starting from images or historical documents, we will focus mainly on some examples of automatic or semi-automatic reconstruction of 3D models from incomplete or peculiar data. A fascinating application is the reconstruction of artifacts which are not existent anymore, using the historical material that can be found, for example, on the web. A nice example is the work aimed at the reconstruction of the Great Buddha of Bamiyan [29], which was destroyed by the Taliban in 2001. This model was automatically obtained from a set of images which were partly found in web repositories. The resulting 3D model was proposed as a starting point for the project to rebuild the huge statues. Figure 2.15 shows a sample set of the images used for reconstruction, a rendering of the niche without the statue and a rendering of the niche with the reconstructed model. This kind of automatic reconstruction, which is based on photogrammetry and stereo-matching techniques, is able to obtain a sufficiently accurate reconstruction of an object which cannot be scanned anymore. A similar but more simple approach is based on the information which can be inferred from plans of the remains of ancient buildings. Semi-automatic image processing techniques are able, for example, to extract the wall lines, so that it is possible to extrude them in order to rebuild the original architecture. Other drawings can be used to add the peculiar shapes of the decorations. A recent and interesting project [30] aimed at combining some of the already mentioned reconstruction techniques with procedural reconstruction methods to obtain a realistic and navigable model of Ancient Rome. A different approach stands in the reconstruction of the original shape of uncomplete or damaged artifacts. As an example, the cranium shape of a human prehistoric race was reconstructed by comparing and integrating the 3D acquisition of the few remains of the original skull and the radiographies of similar races [31]. A final example of virtual reconstructions are all those systems that start from a fragment of an object and try to reconstruct it. An interesting application field is pottery [32-35], where the artifacts are usually obtained by rotation and thus their shape possesses sufficient regularity and symmetry to allow easy reconstruction from fragments (see Figure 2.16). Hence, after scanning one or more fragments of a vase, it is possible to infer its diameter, and produce a digital model of its entire shape.

FIGURE 2.15

Digital reconstruction of the Bamiyan Buddhas. Top: a sample set of the images used for reconstruction. Bottom: the model of the niche without and with the reconstructed model.

FIGURE 2.16

Some images from works by R. Sablatnig (TU Wien) left ro right: an example of profile computed from the digital model of a fragment; profile-based characterization of vases; an example of reconstruction of a vase from profiles.

Virtual Reassembly

Besides the reconstruction of missing geometry, another field of application is the digital reconstruction of disassembled or fragmented artworks. Physical reassembly is a process done manually by archeologists. The adoption of a computer-aided approach can be justified either in case of extreme fragility of the artifact or of complicated manipulation (e.g., the fragments either are too heavy or too numerous to be manipulated easily by an archeologist). Early methods have been proposed for special cases, such as the reassembly of sherds of ancient pottery, where some hypothesize that regularity and symmetry of shape can simplify the reassembly task [33]. A recent result has shown that the generic process can be solved in a robust manner by taking into account also the non-precise and eroded fractures of archeological remains [36]. The joint improvement of 3D scanning and automatic reassembling methods can open new insights in very complex reconstruction problems. In the case of extremely eroded or very lacking sets of fragments, the mathematical approaches can encounter severe issues: an alternative solution is to provide the restorer with a tool which helps in recomposing the original structure. For example, the user can propose a possible recomposition to the system, which would then try to validate it by comparing the adjoining fracture surfaces (i.e., by using a shape-based matching approach). Even the interactive adjoining of the fragments can be aided by applying constraints to the possible relative and absolute movements of the 3D models (as proposed in [37]).

A peculiar example of virtual restoration is the work on the church St. Maria di Cerrate (Lecce, Italy) [38]. Here the problem was to produce a virtual restoration starting from the erroneous results of a real restoration. One of the walls of the nave of the church fell to the ground and was rebuilt using the fallen stones (see Figure 2.17), without taking into account the (apparently unknown) right order of the stone blocks. As a result, those blocks were shuffled and the remains of an old fresco were no longer legible. The problem was solved at the virtual level, first by moving again all stone blocks in the right location (see Figure 2.17(b)) and second by drawing a global virtual restoration, according to the knowledge available on the fresco (see Figure 2.17(c)). This work is presented to the public by means of a prerecorded video and an interactive system.

FIGURE 2.17

Two snapshots from a video produced by F. Gabellone (IBAM-CNR) to show the results of a virtual restoration project concerning a wrongly reconstructed wall with remains of an old fresco in St. Maria di Cerrate (Lecce, Italy); actual status is in (a); results of the virtual recomposition of the fragments in the correct position are presented in (b); final virtual restoration in (c).

FIGURE 2.18

(a) and (b) A physical recomposition of a small subset of the fragments of the Frontone of Luni [37]; at the same time, many sherds still wait to be reassembled. (c) A rendering of the digital model of one of the statues, its colored version according to the current color of the statue, and a painted model reconstructed according to evidence of its possible original colors.

Moreover, creating assemblies with 3D CG or VR technologies helps to better understand the past. Digital 3D models of artifacts can be used as building blocks in interactive CG/VR applications (or to produce passive animations), to give a better understanding of how different materials or components were used in the past to build architectural structures [39] or complex instruments (which could range from the simple compound tool to the big industrial machine).

Virtual Repainting

The availability of accurate 3D models opens interesting capabilities for the dissemination of the original aspect of ancient sculptures or architectures. In many cases we have statues which either completely lost their original painting (this is the case of many archeological masterpieces) or present severe deteriorations. A seminal work has been performed by professor Vinzenz Brinkmann (see [40] for an overview of several different experiences on this topic, including the description of several projects he coordinated on this subject). Brinkmann adopted an approach based on the reproduction of gypsum or marble replicas which were then painted, to give them the look and feel of the original painted artifacts.

With the aid of user-friendly tools, it is possible to easily produce hypotheses about the original color of the statues, based for example on the analysis of the original pigments found on the surface. Since there are usually several possible proposals, the use of digital 3D models avoids the use of physical replicas or 2D drawings. A simple example of 3D model repainting is described in [37]. Figure 2.18 summarizes this project, where we have first a reassembly problem (many sherds have still to be included in the reconstruction; some original historical reassembly hypothesis have still to be validated) and, second, a problem of investigating and repainting the original color from residual traces still visible on the surfaces.

A very interesting practical deployment of the results of virtual repainting can be proposed by adopting digital video-projection rather than repainting a physical replica. Using cheap video projectors it is possible to virtually restore color to the surface of these artworks by repainting the digital 3D model and projecting back the model on the surface of the original artifact [41]. The same approach can be used also to present a different reconstruction hypothesis for the original painting, or to digitally repaint solid copies produced with rapid reproduction technologies. An issue in this type of application is how to register the original (or the physical copy) with the projected digital image. Manual registration is a slow and complicated process, while the same action could be transformed into a semi-automatic process by coupling the video projector with a video acquisition channel and adopting image-based solutions which could iteratively improve the mapping of the rendered image on the original.

Supporting the Restoration Process

Restoration of CH artifacts can be positively affected by the use of accurate digital 3D models. Restoration is nowadays a very complex task, where multidisciplinary skills and knowledge are required. A complex set of investigations usually precedes the restoration of a valuable artwork: visual inspection, chemical analysis, different type of image-based analysis (RGB or colorimetric, UV light reflection, X-ray, etc.), structural analysis, historical/archival search, etc. These analyses might also be repeated to monitor the status of the artwork and the effects of the restoration actions. An emerging issue is how to manage all the resulting multimedia data (text/annotations, historical documents, 2D/3D images, vectorial reliefs, numeric data coming from the analysis, etc.) in a common and integrated framework, making all information accessible to the restoration staff (and, possibly, to experts and ordinary people as well). The final goal is to help the restorer in the selection of the proper restoration procedure by giving full access to the analysis performed, and to assess in an objective manner the results of the restoration (to compare the pre- and post-restoration status of the artwork, to document the restoration process). Since most of the information gathered is directly related to a multitude of spatial locations on the artwork surface, digital 3D models can be an ideal media to index, store, cross-correlate, and obviously visualize all this information. 3D models can also be a valuable instrument in the final assessment phase, supporting the interactive inspection of the multiple digital models (depicting pre- and post-restoration status) to check the eventual variations in shape and/or color.

Moreover, a number of investigations can be performed directly on the digital 3D model by adopting computer-based simulations or computations. This has been done in the past to assess the static and structural status of buildings or sculptures, or to detect risky conditions due to an exaggerated stress of the materials. Deterioration is another effect that can be simulated, to give a preview of the future conditions of the artworks subject to corrosion or deterioration (e.g., the erosion of sculpted stone decorations in our polluted historical towns). Very few works focused on this subject, which involves an accurate simulation of both shape and reflection properties and modification/evolution of the inspected surface [42]. A similar task, with a different goal, is the virtual presentation of the forecasted effects of a restoration action; the goal here is to allow the restorers to show to decision bodies or to the public, before the execution of the restoration, a plausible model of the expected results. Many post-restoration discussions and harsh polemics could be prevented by a preliminary presentation of the planned results and of the visual changes that will be brought to the work of art. Therefore, a future goal for computer-aided restoration technologies would be the possibility to simulate the geometric and appearance effects of degradation or of the inverse restoration on the different materials. This is a highly challenging task, since it’s necessary to couple high-quality geometry acquisition techniques with accurate models of the physical and chemical properties of materials. Being able to simulate how a given metallic artifact would oxidize, or a marble stone degrade/erode under the attack of pollution, acid rain, and other effects, would be a valuable instrument for CH management and conservation. In this context, the ideal goals are both to guess the extent and locations of future damages or to bring back an endangered artwork to its plausible original status.

FIGURE 2.19

Exposure of David’s surface to dust, mist, or other contaminations. This visualization shows, using a false-color ramp, the different classes of exposition produced by the simulation (red: absence of fall, blue: high density of fall), under a maximal angle of random fall of 5 degrees (on the left) and 15 degrees (on the right).

The two following subsections present some results obtained while using digital 3D models and CG tools in the framework of CH restoration.