Database Reference
In-Depth Information
organization (or the world). A successful project is one that people use,
enhance, and extend.
Conclusion
Ultimately, the rise of web-scale data collection has been about connecting
“sensor platforms” for a real-time processing framework. Initially, these
sensor platforms were entirely virtual things such as system monitoring
agents or the connections between websites and a processing framework
for the purposes of advertising. With the rise of ubiquitous Internet
connectivity, this has transferred to the physical world to allow collection of
data across a wide range of industries at massive scale.
Once data becomes available in real time, it is inevitable that processing
should be undertaken in real time as well. Otherwise, what would be the
point of real-time collection when bulk collection is probably still less
expensive on a per-observation basis? This brings with it a number of
unique challenges. Using the mix of infrastructural decisions and
computational approaches covered in this topic, these challenges can be
largely overcome to allow for the real-time processing of today's data as well
as the ever-expanding data collection of future systems.
