Metadata for Broadcast Television (Video Data Sources and Applications)

Turning now from IP video sources to the broadcast world, we find a different range of metadata standards and systems. These systems have been developed to deliver television over a range of distribution channels such as cable, direct broadcast satellite, or over the air. They support EPG and systems information as well as services such as emergency alerting, closed captioning, and content advisory for parental control (V-Chip) as required by the FCC. We will mention legacy analog standards because elements of these persist even as we have largely moved to digital TV distribution standards.

Electronic Programming Guide (EPG)

Interactive Program Guides (IPG) are an integral part of digital television systems, allowing viewers to choose which programs to watch and, for DVRs, which to schedule for recording in the future. The term Electronic Program Guide (EPG) is often used synonymously with IPG, but the latter implies an end user application for searching and browsing in addition to the program data itself. More recently, terms including Electronic Service Guide (ESG), Electronic Content Guide (ECG) and Electronic Media Guide (EMG) have emerged to indicate that material beyond traditional television channels may be described as well. As the number of available channels increases, the IPG becomes an invaluable tool since users can no longer remember what channel their favorite programs are on, or for that matter, which channel number is assigned to a particular broadcaster. The guide information is also used for masking the channel change delay inherent in most digital TV systems. As the user changes channels, the program title and descriptive information from the EPG can be instantly displayed along with the channel number and name; the video will begin to display a few seconds later.

Program schedule data is typically displayed as a two-dimensional grid with channels (or services, hence the name “ESG”) on one axis and time along the other axis. On the set-top, these interfaces struggle to overcome the relatively low resolution TV display and distant viewing environment by employing scrolling mechanisms so that only a tiny fraction of the available guide data is displayed at any one time. Filtering by program genre such as “sports,” “news,” or “movies” is another means by which users can locate content using the IPG. Text entry for search is cumbersome at best using an infrared remote control, but it is an option and is typically implemented using a displayed keyboard (a.k.a. “soft keyboard” or “soft keypad”) navigated by arrow keys.

IPGs are becoming commonplace on the Web (Yahoo, MSN, TVGuide, etc.) where text entry and screen resolution are less of a problem. However the instant gratification of viewing desired content such as a movie from a comfortable sitting position is absent. These IPGs are useful for programming DVRs to record content, or for determining if there is anything interesting to watch in the evening once the work day is over. As more and more users consume video content on laptops and desktops, the transition from guide browsing to content consumption will be rapid and seamless.

EPG data consists of two main classes of data: (1) video program (content) metadata, and (2) scheduling data. The former answers the question “what is on?” while the latter answers “how do I find it? (in the time/channel space).” This distinction is clear in the context of movies for example. The content metadata is what might be found on the DVD jacket: title, actors, running time, rating, etc. The scheduling information tells when and to what channel viewers must tune to watch the movie. The same content may air several different times, perhaps on different channels. However this notion that the content metadata is fixed and immutable while the scheduling data is ephemeral becomes muddied in the context of news programming. Here the content may be described as “Nightly news: today’s top stories” which obviously implies a temporal dependency. For 24 hour news channels which use a cyclical programming model, the content metadata becomes less valuable than the temporal scheduling data.

These EPG data sources are of great utility for video search systems, and it is up to the application to determine which data components are most valuable. For example, for archiving movies and comedy or drama series, the content metadata is most useful, while for broadcast monitoring systems, the temporal information may be just as important. These systems must support queries such as “what was on channel 13 in the New York market at 7:24pm EST on December 14th, 1994.”

EPG streams are specific to a given geographic location and service distribution method. The geographic location determines to which metropolitan area or “market” the viewer belongs (in the US, these are referred to by the FCC as “defined metropolitan areas”). Each market may have its own local programming and sports blackout rules. Furthermore, viewers may get TV content from a variety of sources including over the air (OTA) analog, cable, DBS or IPTV. Providing up to date information for all of these sources is a Herculean task indeed. Due to the size of the data and the availability of accurate scheduling information, most EPG data is delivered to terminal devices (or perhaps cable head end or IPTV VHO servers) in units representing the content to be broadcast for the next two weeks. Breaking news and programming that runs over its allotted timeslot such as extra inning baseball games are often not described properly by EPG services. However, scheduling changes on the order of days can be handled by most EPG services, for example, the baseball World Series is a best of seven games played over the course of several days, but ends early if one team wins four straight games.

There are many sources of EPG data; in the US, Tribune Media Services provides EPG through Schedules Direct® (formerly “zap2it”). Notably, use of this feed is available at nominal charge for application developers and as a result numerous applications use it. Other major service providers include FYI television, Infomedia (in Europe), and Gemstar International Group Ltd, which now includes TVGuide. The EPG data are typically distributed in an XML representation as indicated by Table 2.7 and mechanisms for XML compression and segmented delivery are employed. To reduce redundancy, these formats use a keyed record schema representation. For example, information about each series is kept in a single place and assigned an identifier, and scheduled instances refer to this global information through the use of the identifier.

Table 2.7. Representative EPG XML formats.

In addition to HTTP, EPG data may be delivered in other ways. For ATSC in the US, the TV Guide On Screen (also known as Guide Plus+ from Gemstar International) service uses the National Datacast Service to provide EPG data for digital television receivers. An eight day dataset is transmitted using the PBS channels.

Extended Data Service (XDS)

In the US, the analog TV transport used for closed captions has been used to encode program information as well. NTSC Field 2 can carry the Extended Data Service (XDS) which repeatedly sends metadata including channel call letters, program title, media duration and current play time. This service also includes the time of day which is used on terminal equipment, and this was originally appreciated by consumers since it was used for setting their VCR clocks. XDS is specified by the EIA-766 standard. Although most PBS stations use XDS, commercial stations have not widely adopted it, and it is therefore not reliable as a metadata source for program data.

Program and System Identifier Protocol (PSIP)

While XDS was originally developed for transmission in analog TV systems, today this information is delivered digitally using the ATSC digital television specification using MPEG transport streams with a newer protocol, PSIP, the Program and System Identifier Protocol [PSIP02, Eyer02]. PSIP defines a set of “tables” for representing data in the MPEG-2 transport stream. The time of day is carried in a System Time Table (STT), and the ratings in a Rating Region Time Table (RTT). A “virtual channel table” is used to convey the list of available channels for guides, but of particular interest for content description are the EIT or Event Information Tables. The ATSC uses the term “Event” to describe an instance of a TV program (note that the term “program” has a different meaning in MPEG parlance). EITs are recommended to be transmitted twice a second. The EIT contains start, duration, title, and optional description, content advisory data, and metadata about the closed caption and audio (not the data itself). Receiver UIs may display only the first 30 characters of the title. Descriptions may be sent using extended text messages (ETM) in extended text tables (ETT) and up to 16 days of program data may be advertised in advance.

The Digital Video Broadcasting (DVB) consortium specifies a broad range of transmission standards for terrestrial, satellite, cable and handheld applications (e.g. DVB-T, DVB-S, DVB-C, and DVB-H). Like the ATSC specifications, DVB uses MPEG-2 transport streams (TS) but the protocol for program metadata differs and is encoded in Program Specific Information (PSI) tables to include service information (DVB-SI). This is used for delivery of EPG information in DVB systems [DVB98].