Databases Reference
In-Depth Information
an ingestion and processing engine that can work at extremely scalable speeds on extremely volatile
sizes of data in a relatively minimal amount of time. Let us look at some examples of data velocity.
Amazon, Facebook, Yahoo, and Google
The business models adopted by Amazon, Facebook, Yahoo, and Google, which became the de-facto
business models for most web-based companies, operate on the fact that by tracking customer clicks
and navigations on the website, you can deliver personalized browsing and shopping experiences. In
this process of clickstreams there are millions of clicks gathered from users at every second, amount-
ing to large volumes of data. This data can be processed, segmented, and modeled to study population
behaviors based on time of day, geography, advertisement effectiveness, click behavior, and guided
navigation response. The result sets of these models can be stored to create a better experience for the
next set of clicks exhibiting similar behaviors. The sheer volume of data that is processed by these
four companies has prompted them to open their technologies to the rest of the world. The velocity of
data produced by user clicks on any website today is a prime example for Big Data velocity.
Sensor data
Another prime example of data velocity comes from a variety of sensors like GPS, tire-pressure
systems, On-Star-vehicle and passenger support services offered by General Motors, based on geo-
spatial and location based intelligence associated with the sensor on the automobile, heating and cool-
ing systems on buildings, smart-meters, mobile devices, biometric systems, technical and scientific
application, and airplane sensors and engines. The data generated from sensor networks can range
from a few gigabytes per second to terabytes per second. For example, a flight from London to New
York generates 650 TB of data from the airplane engine sensors. There is a lot of value in reading this
information during the stream processing and postgathering for statistical modeling purposes.
Mobile networks
The most popular way to share pictures, music, and data today is via mobile devices. The sheer vol-
ume of data that is transmitted by mobile networks provides insights to the providers on the perfor-
mance of their network, the amount of data processed at each tower, the time of day, the associated
geographies, user demographics, location, latencies, and much more. The velocity of data movement
is unpredictable, and sometimes can cause a network to crash.
The data movement and its study have enabled mobile service providers to improve the QoS
(quality of service), and associating this data with social media inputs has enabled insights into com-
petitive intelligence.
Social media
Another Big Data favorite, different social media sites produce and provide data at different velocities
and in multiple formats. While Twitter is fixed at 140 characters, Facebook, YouTube, or Flickr can
have posts of varying sizes from the same user. Not only is the size of the post important, understand-
ing how many times it is forwarded or shared and how much follow-on data it gathers is essential to
