Database Reference
In-Depth Information
In this example, a user simply dragged a
FileSink
operator from the
Streams standard toolkit onto the palette and connected the
QuoteFilter
stream to the new
FileSink
operator. The
FileSink
operator was renamed
to
QuoteWriter
and the
file
property is set to the
QuoteFile.txt
file.
After running the new application, we see that by default, the tuple attri-
butes are written one row at a time to
QuoteFile.txt
in a comma-separated
value (CSV) form, such as in the following example:
80.43,83.49,10,(1135711808,117000000,0),"IBM"
83.43,83.46,10,(1135711813,283000000,0),"IBM"
83.42,83.46,10,(1135711813,718000000,0),"IBM"
83.42,83.45,1,(1135711814,731000000,0),"IBM"
81.42,83.45,3,(1135711815,751000000,0),"IBM"
Even though applications can be built completely within the Streams
Graphical Editor, this tool set also provides a powerful round-trip linkage
with the Streams Processing Language editor so that applications can be fur-
ther customized by advanced developers. For example, this integration gives
you the ability to implement custom operators, which didn't initially exist in
the Streams toolkits. Developers often find it convenient to start with the
Streams Graphical Editor and switch back and forth between it and the SPL
editor after the initial application has been sketched. Figure 6-9 shows the
SPL editor with the new
QuoteWriter
operator that we added using the
Streams Graphical Editor. The new code that was generated is highlighted at
the bottom of the listing. If we later delete that code, the Streams Graphical
Editor would also show that the
QuoteWriter
operator was removed.
You can also change applications by replacing existing operators. If we
want to send the
QuoteFilter
stream to HDFS in BigInsights rather than
sending it to a normal file, we can simply select the built-in
HDFSFileSink
from the Big Data HDFS toolkit and drag it on top of the
FileSource
operator.
As soon as the new implementation is assigned, the unique properties for the
HDFSFileSink
operator can be configured, and the application would be
ready to go.
Streams Studio can deploy applications and run them on the cluster or in
stand-alone mode on a single server or laptop. Like the Streams Console,
Streams Studio can display the graphs and metrics for running applications.
There are numerous other features for understanding applications, such as
the ability to color or group a graph based on jobs, hosts, and flow rates. It is
