Database Reference
In-Depth Information
corresponding screens. The user wears active stereo glasses containing targets of
several light refraction markers in a fixed geometry. The location and orientation
of the user's eyes are traced by a six degree of freedom (6DOF) tracking system.
A tracking server calculates each target's position and orientation based on images
captured by tracking cameras distributed on top of the screens. The tracking data is
used to determine the content to be displayed on the screens. The 3D Unity game
engine and visual C# are used to implement the feedback engine, and interface with
the Kinect sensor. Middle VR is used to control the graphic in the CAVE.
The goal of the system is to recognize sequences of movements, by identifying
where certain target postures occur along the time-line of the performance. The
recognition module extracts occurrences (phases) from the beginner's performance
(a phrase is transition between identifiable postures), and then assesses them against
the teachers' (ground truth) movement. Finally, the system visualizes both the
teacher and student's dance sequences (or isolated movements) in a VR setting.
Students participating in this process would receive feedback on the accuracy of
their performance and on specific areas for which their accuracy is poor and thus in
need of attention. This process is illustrated by a VR dance training system, with its
various feedback protocols, as shown in Fig.
11.2
.
Figure
11.3
shows the gesture recognition module that is implemented by
the SSOM-based isolated gesture recognizer. The SSOM is incorporated with
a template matching method (explained in Sect.
11.5
) or a HMM (explained in
Sect.
11.7
) for gesture recognition. The sequence of frames is recorded by a standard
Microsoft Kinect. Each frame contains data explaining the location of joints in 3D
axes [
349
]. In the preprocessing block, a feature extraction module transforms the
3D positions of all joints in the captured frame to a feature vector characterizing the
Fig. 11.1
System architecture
Search WWH ::
Custom Search