Metadata is “data about data,” or in this case “data about media.” Global metadata refers to the entire media assets and typically includes a title, author, copyrights, etc. While almost every video application supports global metadata in some form [DC03], for some applications additional sets of metadata may pertain to segments of the media – such as for news content were one segment may contain footage from a third party and copyrights may be more restrictive for that segment.
Content-based retrieval
Content-based retrieval involves the use of metadata that is derived (typically automatically) from the media streams and almost always includes a temporal attribute. Note that a transcript of the dialog is usually considered to be “content” and not metadata although it can be represented concisely in data structures similar to those that represent global metadata such as the description, and a text stream can be used for both consumption and navigation.
Most video searching systems rely on high level attributes accompanying the video files for search. These typically include title, date, genre, and brief description. However the next generation of searching systems goes beyond metadata to search the content of the video. Content indexing not only provides a more accurate description of the media; it supports temporal information so that users can navigate to the desired segments of long- form video material. It is content indexing that enabled Web search engines to excel since codifying and maintaining consistent document metadata is impractical, from both the engineering and social perspectives. Video search systems must leverage existing global metadata, incorporate any manually added detailed metadata or tags, extract additional detailed metadata automatically, and support the on-going addition of viewer supplied metadata such as tags, ratings, comments, and popularity.
Conclusion
The amount of video content on the Web is growing rapidly as new technologies such as Internet protocol television (IPTV) and mobile video are deployed. Video search engines are being developed to enable users to take advantage of these video resources for a wide variety of applications including entertainment, education and communications. However, the task of information extraction from video for retrieval applications is challenging, providing opportunities for innovation.
All video is not created equal; there is a wide range in terms of quality, available metadata and content. We described some of the challenges for video search as related to text search, and introduced the notion that metadata plays a key role in the accuracy and effectiveness of video search. The metadata may accompany that content and be easily ingested for search, and powerful media analysis technologies may be employed to extract additional, detailed metadata for search. Users may be participants in the metadata creation process through tagging and otherwise commenting on the video that they have viewed. Analysis of user activity can lead search engines to make implications about video content and quality or popularity.
