Visual Servoing for Beating Heart Surgery - Visual Servoing via Advanced Numerical Methods - page 96

Figure 6.6 shows one heartbeat ahead prediction results for the linear parameter
varying (LPV) and amplitude modulation (AM) methods. The data used to make
the assessment correspond to a pig heart motion in the anterior posterior direction,
acquired during in vivo experiments. During these experiments the pig heart was
restrained using a passive cardiac stabilizer. The prediction errors computed during
5 respiratory cycles are 25 m respectively for the LPV and the AM

methods. These results are satisfactory as they are below the surgical accuracy eval-
uated to 100

μ m and 13 μ

m [3]. We should however keep in mind that the prediction algorithm
is only one of the keys necessary to build an efficient predictive control scheme and
so the obtained accuracy can only give an approximate indication about the final
motion compensation accuracy.

μ

6.4 Robust Real-time Visual Measurement

Usually, when doing visual servoing in vivo , vision is Achilles' heel. Indeed, en-
doscopic images are especially difficult to process reliably. This is mainly due to
specular highlights caused by the reflection of intense endoscopic light (an annu-
lar light source which is around the optical axis at the tip of the endoscope) on the
wet surface of the organs. Furthermore, respiration or heart beating can cause large
displacements between consecutive images thus increasing the tracking difficulty.
Additionally, some organs are very poor in term of landmarks ( e.g. the surface of
the liver which has an uniform texture) and tracking algorithms have difficulties to
“stick” to a specified patch. Finally, in a surgery context, the scene is supposed to
change during time and occlusions can often occur due to the motion of instruments
in front of the endoscope.
To meet the high safety standards required in surgery, the visual feedback must
tend to be perfectly reliable. The easiest way to improve this point is to add artificial
markers to the scene. So, Nakamura and Ginhoux [31, 19] affixed a target directly
onto the epicardium. The main drawback with this approach is the additional time
needed to attach the markers: the use of a robotic system should always simplify
procedures from the surgeon's point of view. So people started to work on mark-
erless techniques. In [34], Ortmaier proposes a region-based tracking technique us-
ing natural landmarks (a set of small patches in the image) with compensation of
illumination variations and removal of specular highlights. Takata worked on a con-
densation algorithm [40]. Noce et al. worked on a texture-based tracking algorithm
[32]. A texture distance is defined and used to track patches in a high-speed video
sequence. The proposed algorithm is compared to the efficient second order mini-
mization (ESM) method proposed by Benhimane and Malis [7] which behaves also
very well.
In the context of stereo endoscopy, Stoyanov et al. [38] propose a feature-based
method that uses a combination of maximally stable extremal regions (MSER) and
traditional gradient-based image features. MSER-landmarks detection is robust to
illumination variations and specular highlights. The heart is a highly deformable
organ. Thus, in [32], the authors use thin-plate splines to parameterize the surface