Database Reference
In-Depth Information
Twitter reported that it collected around 500 million tweets per day with
spikes up to around 150,000 tweets per second. That number has surely
grown since then.
This data is collected and disseminated in real time, making it an important
source of information for news outlets and other consumers around the
world. In 2011, Twitter users in New York City received information about
an earthquake outside of Washington, D.C. about 30 seconds before the
tremors struck New York itself.
Combined with other sources like Facebook, Foursquare, and upcoming
communications platforms, this data is extremely large and varied. The data
from applications like web analytics and online advertising, although highly
dimensional, are usually fairly well structured. The dimensions, such as
money spent or physical location, are fairly well understood quantitative
values.
In social media, however, the data is usually highly unstructured, at least
as data analysts understand the term. It is usually some form of “natural
language” data that must be parsed, processed, and somehow understood
by automated systems. This makes social media data incredibly rich, but
incredibly challenging for the real-time data sources to process.
Mobile Data and the Internet of Things
One of the most exciting new sources of data was introduced to the world
in 2007 in the form of Apple's iPhone. Cellular data-enabled computers had
been available for at least a decade, and devices like Blackberries had put
data in the hands of business users, but these devices were still essentially
specialist tools and were managed as such.
The iPhone, Android phones, and other smartphones that followed made
cellular data a consumer technology with the accompanying economies of
scale that goes hand in hand with a massive increase in user base. It also
put a general-purpose computer in the pocket of a large population.
Smartphones have the ability not only to report back to an online service,
but also to communicate with other nearby objects using technologies like
Bluetooth LE.
Technologies like so-called “wearables,” which make it possible to measure
the physical world the same way the virtual world has been measured for
the last two decades, have taken advantage of this new infrastructure. The
