Introduction
Regardless of the type of display used, its size, or the application in question, there are some concepts that are common to just about any display system. In the most general usage of the phrase, “display system” can be taken to mean any system through which information is conveyed to people through visual means. A topic, a painting, or a sign could all be considered “display systems” in this sense, although admittedly they are not the sort of things with which this topic will primarily be concerned.
But even these simpler examples of “displays” share some basic properties and concepts with the most advanced electronic display now being developed. If we use the above broad definition, and use the convenient term image to refer to any information which is being conveyed visually, then several of these basic concepts should become readily apparent.
Basic components of a display system
Remaining within the broad definition of a display system, again “a system through which information is conveyed to people through visual means,” we could further divide this system into several basic elements. Any practical display system would include, in some form or another, all of the following (Figure 1-1):
1. An image source. This might be a real object, as viewed by an image transducer such as a camera. It might be a computer program (along with the hardware on which that program is running), as in the case of the completely synthetic images seen in many movies. It might even be information which is originally not in viewable form; an example might be spoken words transcribed to text (in which case other factors, such as the choice of font, etc., also could be considered a part of the “image source”).
2. Image processing and rendering. Almost always, the basic information from the source must be processed in some manner before being delivered to the rest of the system, and then must be put into a form which is suitable for the intended display device. An example of basic image processing is the “gamma correction” performed as a basic part of the broadcast television system. The computer graphics (CG) field has given us the term “rendering” to refer to the process of finally changing the image information into the form required by the display; in the case of CG, “drawing” the image in nearly display-ready form into a frame buffer.
3. Image storage, compositing, and transmission. Within many, if not most, systems, there are provisions for the storage of the image information prior to display. This can range from the combination of digital memory and video tape storage common in the television industry, to the purely digital frame buffer of the computer graphics system. The image storage portion of a display system (in this broad sense of the term) may be used simply to delay the delivery of the image information to the viewer, but it may also be an important factor in the image processing and rendering step, and shared by that portion of the system. Between the two, the image storage subsystem is also quite often that point at which information from multiple sources is composited into the final single image which will be seen by the viewer. Lastly, the composited image must be delivered to the viewer via a transmission channel suited to the application in question.
4. The display itself. In the context of this particular discussion, also included within the display subsystem are those portions of the complete system which are used to translate the image information between the format in which it is stored (or in which it has been transmitted so far) and the format used by the display device proper, should such a step be required. For example, computer systems will most commonly store image information in digital form, which often must then be converted into an analog video signal for use by the display.
5. The viewer. It may seem odd to be including the viewer in a discussion of a display system, which might normally be assumed to be simply a collection of electronic equipment or some other inanimate construction. The point here is that the viewer is in reality the single most important factor in determining the performance requirements and other factors that define the rest of the system. It is pointless, for instance, to construct a system which could provide detail or differences in color far beyond the capability of human vision to resolve. Some display systems are even defined completely for the specific needs of a particular class of viewer, as in the case of those which aid the visually impaired. In any case, it is of utmost importance to keep in mind that any display system ultimately relies on, and is limited by, the needs and limitations of human vision. Display systems are a human interface.
The basic notion brought up in that last section is worth repeating for the overall system. It makes little sense to provide capabilities in any one part of the system that cannot be used by the whole. Conversely, a limit in any of these subsystems is a limit on the whole. Most often, unless it is anticipated that a given subsystem will be used with other “blocks” of varying performance, it is best to attempt to achieve a balance between all parts of the complete system. And again, in the case of display systems specifically, it is ultimately the capabilities of the viewer that will define the requirements for the system.
Figure 1-1 The elements of any imaging system include: (1) the image source, in this case a real object as seen by an image capture device (a camera); (2) image processing; (3) image storage and transmission; (4) image display; and (5) last but certainly not least, the viewer. As the entire purpose of an imaging system is to deliver visual information to the viewer, the viewer becomes the most important factor in the entire system, and dictates the required performance of the rest of the chain.
Imaging Concepts
The previous section defined a display system as one that is intended to convey image information to a viewer, but did not say anything about that image information itself. In what form should we expect it to be? How much information will there be, and at what rate must it be conveyed? What do we mean by “image” in the first place?
The most common-sense answer to the last question is that “image” simply means “what people see”. We will look more deeply into just how human vision operates in a later topic, but at this point this simple definition will serve as the basis for examining some concepts from imaging science. Humans see the world in color, and can discern motion, apparently at a fairly high level of resolution, and in three dimensions – how does this ability define that which is captured or produced through electronic or other means for human viewing?
First, while humans do possess visual systems that provide “three-dimensional” information, we should recognize that this comes from having two “image receptors” (eyes), each of which actually only captures a two-dimensional view of the world. We gain information about the third dimension through comparing these two views. So our basic working definition of image might be “a two-dimensional visual representation”, using the word “visual” to hide a multitude of issues regarding the limitations of vision.
More importantly, we draw a distinction between image and reality; the light reflected from an object, and captured by a lens, may form an image of that object on a sheet of paper – but the image is not the real object, and does not contain all of the information about that object. It contains only the information relating to the appearance of the object – “that which we see” – and often only a part of that.
Vector-scan and raster-scan systems; pixels and frames
But how can we represent these images in a form which can be handled by mechanical or electronic equipment? We can answer that by looking at two means through which people have created images – or pictures – for years. The most obvious means of creating a picture is as a child does, through drawing lines which come together to form recognizable representations of shapes, objects, letters, and so forth. We can “teach” a machine to draw in this manner, simply by defining the image space (the “surface” onto which the image will be drawn) properly. We might, for example, create a two-dimensional coordinate system, and then “tell” the machine (through its programming, controls, or whatever) to create the image as follows:
1. Draw a red line from point (2,4) to point (4,4).
2. Draw a red line from point (2,4) to point (2,0).
3. Draw a red line from point (2,2) to point (3,2)… and so forth.
Rather than defining the lines’ starting and ending points, we could also simply define the start point, the direction (relative to some agreed-upon reference), and the length of the line; the results would clearly be the same. This sort of drawing is called a vector representation, and display devices which create images in this manner are said to be vector-scan displays (Figure 1-2). The classic mechanical plotter or analog oscilloscope are excellent examples. For a more current example, we should note that computer graphics systems often use vector representations of lines and objects within an image, even if the final display device does not use this form.
But there’s another way that people have used for centuries to make images. You can make a recognizable picture by placing small dots or tiles of pure color next to each other to form the desired shape or pattern; this is called a mosaic.
Figure 1-2 Vector scanning. The dotted lines represent portions of the scan in which no line is being drawn or illuminated, but rather the drawing device is being repositioned to create the next line.
To get a machine to draw a mosaic, it is again simplest if we define the image surface in terms of a two-dimensional coordinate system. If that is done, then we can instruct the machine as follows:
1. Place a blue tile at (4,3).
2. Place a blue tile at (3,2).
3. Place a green tile at (1,5)…
and so forth until the desired image has been created (Figure 1-3).(We assume in this exercise that all the tiles are the same size, and properly sized so that each one fills the “grid square” to which it is assigned.) But what if we are trying to create this mosaic from a “real” scene? One way to do this would be to project the image of the scene to be created, through a lens as was described above, onto a sheet of paper on which was also drawn a grid of lines using the chosen coordinate system. A person could then simply look at the projected image, determine the color, brightness, etc., at each point on the grid, and use that information to create the mosaic. (We assume we have tiles of any possible color, brightness, and so forth; this implies a very understanding tile supplier, not to mention a rather capable tile-making machine.)
This system of creating images in the manner of a mosaic is actually the basis behind practically all electronic imaging and display systems in use today. Information about the image is taken at a number of regularly spaced sampling points; this information may then be processed, translated, etc., until finally sent to a display which may itself create the desired output through controlling each of a regular array of points or cells making up the display’s “screen”.
Figure 1-3 Creating an image as a mosaic. This forms the basis for the raster-scan display types; however, it is important to note that the picture elements (“pixels”) we will be discussing are actually to be considered as dimensionless point samples, rather than something analogous to square tiles.
The term picture element, most often condensed into the convenient word pixel, has been used to refer both to the individual samples of the original image at each point, and to those individual elements in the display device which make up the final displayed image. It is important not to confuse these two meanings of the word – pixel as a point sample of an image, and pixel as a physical part of a display device – as they have significant differences. The most important of these is that, in the former definition, pixel truly refers to a point sample taken from the original image; it has neither size nor shape, but is only the information regarding the characteristics of the image at a theoretically dimensionless point. This is very different from the pixel of many display devices, which most definitely has a fixed size, shape, and other restrictions. As will be seen later, this distinction is at the heart of many differences in image appearance among different display technologies and imaging techniques.
At this point, we should also realize that there is a more efficient means of conveying the data in such a “mosaic” system than specifying the coordinates for each tile separately. Assuming that each location within the 2-D image space must receive a “tile”, or pixel, it is much easier simply to specify a starting point – the first “tile to be placed”, or the first pixel location to be sampled – and then to proceed through the array in a predefined, regular manner. If the space is defined by a rectangular coordinate system, an obvious method is to proceed by sampling each location in a row, and through each row in turn. This is the basic description of the raster scan technique (Figure 1-4). Almost every display system in use today employs raster scanning; the vast majority of these begin the scan at the upper left corner of the image, and proceed through all the pixels in a given horizontal row and thus through each row in the supposed array of pixels. One pass through the full array is generally referred to as a frame.
It should be noted that the above description, based on a uniform rectangular array of sample points or “pixels”, is a close match to the reality of practical fixed-format display devices such as LCD panels, etc..
Figure 1-4 A raster-scanned image. In this form of display, the “drawing” mechanism follows a regular path across the imaging area, and the picture elements are placed or illuminated as required to build the desired image. Again, the dotted lines represent times during which no “drawing” is performed, but rather the drawing mechanism is being repositioned.
However, the notion of constructing an image through a regular, repeated scanning structure is not necessarily tied to the idea of fixed sampling points. In so-called “purely analog” systems such as the original television implementations, the image is scanned on a continuous basis, without distinct, discrete sampling points as have been assumed above. (There is much more to be said on the distinctions, both real and imagined, between “analog” and “digital” systems; this will be covered in a later topic.) Even in systems which do employ discrete pixels, there is no requirement that these be in a simple rectangular array. Other sampling structures have been proposed, and in many cases may provide certain advantages over the common rectangular structure.
