Abstract
The abilities to increase precision of surgical procedures by tracking real-time motions, accurately measuring the mechanical properties of complex 3D geometries, and tracking deformations of components over time, among many other applications, depend on the availability of robust, high-speed, full-field-of-view, 3D shape measurement systems.
In this paper, we present advances in our development of a high-speed 3D shape measurement system based on fringe projection. The system consists of a high-speed projector, with speeds up to 20,000 frames per second, that is integrated with a CCD camera to provide full-field-of-view information. By using high-speed projection of sinusoidal fringe patterns with varying spatial densities together with temporal phase unwrapping algorithms that we are developing, we are able to compute and display unwrapped phase maps at video rates, which enable the capability to perform absolute shape measurements of components.
We present representative results obtained with our system as we have applied it to art conservation and to biomedical imaging. Results validate system capabilities as a high-speed method of dynamically gathering, analyzing, and displaying shape information.
Keywords: fringe projection, high-speed shape measurements, structured light, temporal phase unwrapping. 1.
INTRODUCTION
This paper focuses on recent advancements in our development of a fringe projection based system for 3D shape measurements. Noninvasive techniques for surface measurements have become paramount for quality analysis in industrial applications, art conservation and restoration, as well as precision aid in medical procedures. Continued development of optical measurement systems enhances the versatility, applicability, and repeatability required by industry. Additionally, integration of 3D measurement techniques with computer aided design (CAD) software and computer aided machining (CAM) equipment provides opportunities for reverse engineering [1].
The critical advantage of the fringe projection optical technique is the ability to provide full field-of-view (FOV) information. Although this technique is promising, limitations in speed and difficulties achieving sinusoidal projection patterns have restricted many systems and their potential applications. For fringe projection, sinusoidal patterns are critical because they minimize discontinuities and errors in the reconstruction algorithms. This project explores the mathematical importance of sinusoidal projections while analyzing their quality via quantification of processed images, which will help in the continued development of our system as a combined high-speed, high resolution measurement device.
3D image reconstruction is achievable through several image unwrapping techniques. Our system uses an optimized Temporal Phase Unwrapping (TPU) algorithm that utilizes varying fringe frequencies to recover shape information in the time domain. This algorithm was chosen based on its robustness and an error analysis showing the optimal projection pattern for TPU. Contrary to other systems, the 3D shape measurement system developed in the CHSLT laboratories has unprecedented versatility to accommodate a variety of applications with the resolution and speed requirements. Hardware systems are integrated into user-friendly software developed in our laboratory [2].
SYSTEM SETUP
The Fringe Projection system consists of two major components, a spatial light modulator (SLM) and a digital charged-couple device (CCD) camera, shown in Fig. 1. The SLM, packaged by Vialux, contains a digital light processing (DLP®) unit from Texas Instruments called DLP Discovery™ [3]. The system uses a Digital Micro-mirror Device (DMD) with a 1080 x 1920 chip resolution [4]. Each of the independent 10.8 x 10.8 ^m2 micro-mirrors are controlled by a duty cycle representing a percentage of time each mirror is in the on-state; thus, the SLM has intensity modulation control, producing the projected sinusoidal fringe pattern. The second component of the system is a Pike F-032 CCD camera with 640 x 480 chip resolution, 7.4 x 7.4 ^m2 pixel size, a maximum frame rate of 208 frames per second at this resolution, and a bit-depth of 14-bit. Depending on the application and required field-of-view (FOV), the CCD camera can be interchangeable.
Fig. 1. Fringe projection setup that we are developing: (a) a schematic of the system shows the spatial light modulator and the CCD camera separated by triangulation angle, 0, both interfaced into a laptop computer; and (b) realization of our system with an art sculpture under examination.
The SLM projects a pattern onto an object that is recorded by a camera separated by an angle via the method of triangulation. System sensitivity increases with larger triangulation angles, but is more susceptible to unresolvable areas caused by shadowing [5, 6].
PRINCIPLE OF FRINGE PROJECTION
Structured light projection is the basis for 3D reconstruction, as patterns will deform to the shape of an object. Inducing a precise phase shift of the projected pattern makes it possible to solve for the interference phase containing object depth information. The camera recorded intensity distribution is represented by:
where the recorded intensity distribution, I, is a function of the brightness, a, the amplitude, or contrast, b, the random phase, A®, the known induced phase shift, a,, and the fringe-locus function, Q , containing shape information for each pixel (x, y).
A least-squares method can be used to solve for Q by minimization of the summation of quadratic errors [7]. In general, the greater number of phase shifted images used to recover the phase results in better resolutions by minimizing random electronic noise and inaccuracies in phase shifting, A®. Using more images for reconstruction increases the measurement and processing time, but could be advantageous to particular applications that are not time critical. The four phase shift algorithm was chosen for the 3D shape measurements because it is nearly insensitive to phase shifting calibration, therefore minimizing electronic noise [8]. The resulting interference phase is an arctangent function described as:
Each of the four phase-shifted images is represented by "I" and a subscript that corresponds to an increasing shift ranging from 0 to 3n/2, in increments of n/2. The resolution of the interference phase is directly related to how closely the projected fringes follow a sinusoidal pattern. Projection of gray-code, or dark and light, fringes produces discontinuities that are typically corrected by digitally smoothing, resulting in increased processing time. Sinusoidal projection can be decomposed into a Fourier series approximation:
Since the Fourier approximation, fx), contains only the summation of continuous cosine and sine approximations, with integration functions, an and bn, discontinuities will appear as high frequency components. A theoretical sinusoidal fringe pattern projection, Fig. 2a, shows a corresponding cross section and power spectrum, in the frequency domain. The center DC component with a frequency component based on the number of fringes is shown. Figure 2b shows a resulting power spectrum of a square wave projection with many other higher frequency components that do not contain any shape information, but can be regarded as noise from the discontinuous square function. The energy density for a sinusoidal projection and square wave with a frequency of one fringe are 694 and 1351, respectively, proving yet again that sinusoidal fringe projection results in better image resolutions.
Fig. 2. Projected fringes: (a) 512 x 512 sinusoidal fringe projection pattern with a sample cross sectional area and power spectrum; and (b) 512 x 512 square projection pattern, cross section and power spectrum.
Gray scale sinusoidal projection is achieved in our system by controlling each of the mirrors, or pixels, in the DMD, Fig. 3, by setting the duty cycle for each mirror appropriately for the desired gray scale. The camera’s exposure time is set to a level corresponding to the maximum time a mirror can be in the on-state to represent a completely light fringe. Over this exposure, the camera will integrate, or average, the light intensity of other pixels and produce an equivalent to a gray scale level [9, 10].
Fig. 3. Device developed by Texas Instruments and used in our fringe projection system: (a) DMD chip; and (b) Enlarged view of micro mirrors enabling sinusoidal projection.
Current developments enable the projector to change bit-depth rapidly from 5 to 14 bits, and an equivalent range of 32 to 16384 gray levels. An approximation method determines the duty cycle to produce the most appropriate gray scale depending on fringe density. Higher bit-depths result in more accurate sinusoidal representations, but slow the acquisition speeds to a few frames per second (fps). Lower bit-depth projections can maintain speeds, as well as process and display information on the order of 200 fps.
ADVANCEMENTS IN TEMPORAL PHASE UNWRAPPING (TPU)
Processing and viewing the information with high-speed and accuracy requires an unwrapped algorithm. By varying fringe densities on an object, the TPU algorithm can be used to determine the fringe order number and, based on this, resolve 2n phase discontinuities. Different from spatial unwrapping techniques, TPU is performed in the time domain for each pixel by using the wrapped phase images calculated for each of the varied fringe densities, shown in Fig. 4. Consequently, pixels are not affected by poor signal to noise ratios in neighboring pixels, as often seen in spatial unwrapping techniques [12].
Fig. 4. Temporal phase unwrapping is executed along the time axis, with increasing fringe frequency.
This hierarchical approach to unwrapping is based on following an increasing fringe frequency starting with no discontinuities in the phase map [13]. Mathematically, the unwrapped phase, u&, of a particular image can be calculated as follows:
Where & represents the wrapped phase and N is the fringe order number that characterizes phase jumps with the addition or subtraction of integer values; thus, images are corrected and have no phase jump discontinuities. An error term, e, is added due to electrical noise and effects of uncertainties in the algorithm. A particular resolution can be reached by determining how the error term propagates through varying numbers of images used to recover the unwrapped phase. Figure 5 shows how the error is minimized as more images are used in the TPU.
Fig. 5. Error propagation as a function of the number of images used in the temporal phase unwrapping algorithm.
Although the error approaches zero, the maximum fringe frequency is limited to the DMD pixel. Thus, higher resolutions can be achieved by using more images in the unwrapping algorithm in situations where time is not a critical factor. The TPU unwrapping sequence can be determined based on the particular application and its requirements.
REPRESENTATIVE RESULTS
Advancements in the system, particularly gray scale projection, were analyzed by taking cross sections of images captured by the camera from the projected patterns of the SLM. For evaluation purposes, a telecentric lens with a limited FOV was used to reduce distortions. The projected pattern, at 128 pixels per fringe, and the corresponding cross section in Fig. 6 shows the ability to successfully project sinusoidal images from the SLM.
Fig. 6. Fringe projection analysis: (a) Fringe pattern captured by the CCD camera; and (b) Plotted cross section of the image validating the capabilities of our system to projection sinusoidal patterns.
The effects of varying SLM bit-depth were explored using a consistent projection density and triangulation angle based on the 14 bit CCD camera using a highly reflective, diffusive surface. Results indicate that the calculated standard deviation in each individual image varied by less than 5% between the 5 and 14 bit projection patterns. In practical applications, this gives an advantage for performing at higher speeds without losing resolution in the sinusoidal projection.
Using the sinusoidal fringe projection, we were able to successfully implement the TPU technique to acquire 3D images, shown in Fig. 7.
Fig. 7. Unwrapped sculpture under examination: (a) 2D unwrapped image; and (b) Actual 3D images reconstructed via our advanced algorithms with two alternate views.
CONCLUSIONS AND FUTURE WORK
Representative results show the potential of the system to combine high speeds with high resolution images by implementing the sinusoidal projections. Similarly, temporal unwrapping proves a valuable technique for 3D imaging. Preliminary results have validated that the resolution of unwrapped images increases as the number of phase images used increases.
Future work will focus on several advancing aspects of the system. The first is the system calibration process, which will be critical in evaluation of system accuracies and measurement repeatability. A packaging system with mechanical components for easy FOV and triangulation angle adjustments will be developed. The objective is to increase the portability and versatility of the system to adapt to a variety of applications.
