We have already seen how captioning can be exploited for video search, but further, video search engine systems and IP media systems should preserve any captioning that accompanies the ingested source media in order to reach the broadest possible audience. Again, it is important to point out that captioning is not just for the hearing impaired, but can improve comprehension and enable media consumption in a wider range of environments (e.g. meetings).The National Center for Accessible Media at WGBH pioneered television captioning [Robson97] and has recently formed the Internet Captioning Forum with industry leaders. The Distribution Format Exchange Profile (DFXP) is a subset of the Timed Text Authoring format intended to aid in interoperability of existing legacy formats. While its scope is limited, the specification includes enough generality to support a very wide range of existing captioning presentations (perhaps only exclusive of sign language representations) so it is not trivial by any means [TT06].
Conclusion
We have presented many of the practical aspects of digital video that content-based video search engine systems must deal with in order to operate seamlessly on a wide variety of content sources. At the basic level, issues of encoding and container file formats, and DRM systems must be taken into account in the system design. Next, presentation issues such as aspect ratio and transcoding for archival storage and delivery for a range of applications must be considered in the design of user interfaces for search. We also introduced methods for creating networked user interfaces for media replay with thin clients such as media players with dynamically generated playlists or browser plug-ins. Beyond the basic input and output media handling and rendering, systems that operate on the video content must also deal with real-world issues such as subsampled, noisy chrominance, non-square pixels and various temporal sampling rates. While a theoretician might correctly dismiss many of these issues as engineering decisions arising from legacy (or worse, commercially motivated proprietary and incompatible) implementations, some are related to basic principles or physical properties. There are limits to the fractional bits per pixel to which video can be compressed and the signal to noise ratio of imaging devices.
