Database Reference
In-Depth Information
The SDA is a result of thousands of iterations by IBM in this area, but as you
know, the journey never ends.
Define Sentiment Targets
It seems pretty obvious when we write it, but you need to tell the SDA what
you are looking for before your analysis can begin. When it comes to text
extraction, you have to define the element to extract and discover from there.
For example, you can provide the SDA a set of brands, products, and services
that you're interested in tracking. You can make things more sophisticated by
also defining product category information and aliases for the product. The
SDA ingests the defined terms of interest and turns it into an AQL dictionary
(see Chapter 8 for more information), which is factored into the feedback
extractors that are executed in the SDA's downstream processing.
Local Analysis
This stage is the first of two text analytics passes against this batch of data. In
this phase, each document is analyzed individually using both the feedback
and profile extractors. Before analyzing the text itself, additional spam filter-
ing is done, and mentions of the brand, products, or services are isolated. At
this point, the feedback and profile extractors are run against the preprocessed
social media messages. The output of the profile extractors is added to the
global user profiles stored in BigInsights. The output of the feedback extrac-
tors is written to BigInsights—in a comma-separated value (CSV) format—for
easy visualization with BigSheets.
Global Analysis
With updated user profiles in place, a second round of analysis is done
against this batch of data, using entity analytics to determine which users are
the same across the data sources. Again, output of this stage is written as
updates to the global user profiles stored in BigInsights.
Streaming Mode
While the SDA can process social media messages in batch mode using Big
Insights, it can also run in streaming mode using Streams. In this case, you
pass the definition of sentiment targets to BigInsights, which generates the
AQL, and the resulting executable code is passed to Streams. The local and
global analysis happens on Streams in the same manner in which it is
