Biomedical Engineering Reference
In-Depth Information
Fig. 16.7
A depth image from the Microsoft Kinect clearly shows a human waving at the camera.
Here we show nearer object in warmer colors and more distant colors in cooler colors
pseudo-random array across a large area. The CMOS sensor is able to then read
the depth of all of the pixels at 30 frames per second. It is able to do this because
it is an active pixel sensor, which is comprised of a two-dimensional array of pixel
sensors. Each pixel sensor has a photo detector and an active amplifier. This camera is
used to detect the location of the infrared dots. Following this, depth calculations are
performed in the scene using stereo triangulation, which adds a requirement of two
cameras. The depth measurement requires that corresponding points in one image
need to be found in the second image. Once those corresponding points are found, we
can then find the disparity (the number of pixels between a point in the right image
and the corresponding point in the left image) between the two images. If the images
are rectified (along the same parallel axis), then, once we have the disparity, we can
then use triangulation to calculate the depth of that point in the scene. The depth
data is then interpreted in and used in the system. To visualize the depth information,
a depth image can be generated by assigning a color coding to the data as shown
in Fig.
16.7
.
The Kinect software is able to track users' skeletons by combining the depth
information with knowledge about human body kinematics that was obtained by
gathering and labeling data fromspecial rigs which captured user motions in everyday
life. These images and labels were then used for training amachine learning algorithm
to create probabilities and statistics about the human form and movement. In real
time, when the user steps in front of the Kinect, a 3D surface is generated using
the depth information, creating a point cloud of the user. The Kinect then creates a
starting guess at the user's skeleton and using the kinematic data, the Kinect makes
attempts to determine the different parts of the body. A level of confidence is also
assigned to each guess based on how confident the algorithm is about guessing the
correct parts. Once this is done, the Kinect finds the most probable skeleton (an
