Databases Reference
In-Depth Information
4.1.2
Examples of Stream Sources
Before proceeding, let us consider some of the ways in which stream data arises
naturally.
Sensor Data
Imagine a temperature sensor bobbing about in the ocean, sending back to a
base station a reading of the surface temperature each hour. The data produced
by this sensor is a stream of real numbers. It is not a very interesting stream,
since the data rate is so low. It would not stress modern technology, and the
entire stream could be kept in main memory, essentially forever.
Now, give the sensor a GPS unit, and let it report surface height instead of
temperature. The surface height varies quite rapidly compared with tempera-
ture, so we might have the sensor send back a reading every tenth of a second.
If it sends a 4-byte real number each time, then it produces 3.5 megabytes per
day. It will still take some time to fill up main memory, let alone a single disk.
But one sensor might not be that interesting. To learn something about
ocean behavior, we might want to deploy a million sensors, each sending back a
stream, at the rate of ten per second. A million sensors isn't very many; there
would be one for every 150 square miles of ocean. Now we have 3.5 terabytes
arriving every day, and we definitely need to think about what can be kept in
working storage and what can only be archived.
Image Data
Satellites often send down to earth streams consisting of many terabytes of
images per day. Surveillance cameras produce images with lower resolution
than satellites, but there can be many of them, each producing a stream of
images at intervals like one second.
London is said to have six million such
cameras, each producing a stream.
Internet and Web Tra
c
A switching node in the middle of the Internet receives streams of IP packets
from many inputs and routes them to its outputs. Normally, the job of the
switch is to transmit data and not to retain it or query it. But there is a
tendency to put more capability into the switch, e.g., the ability to detect
denial-of-service attacks or the ability to reroute packets based on information
about congestion in the network.
Web sites receive streams of various types. For example, Google receives sev-
eral hundred million search queries per day. Yahoo! accepts billions of “clicks”
per day on its various sites. Many interesting things can be learned from these
streams. For example, an increase in queries like “sore throat” enables us to
track the spread of viruses. A sudden increase in the click rate for a link could
