Database Reference
In-Depth Information
2.5.2
Streaming API
Using the Streaming API, we can search for keywords, hashtags, userids, and
geographic bounding boxes simultaneously. The
filter
API facilitates this search and
provides a continuous stream of Tweets matching the search criteria. POST method
is preferred while creating this request because when using the GET method to
retrieve the results, long URLs might be truncated. Listings
2.8
and
2.9
describe
how to connect to the Streaming API with the supplied parameters.
Listing 2.11
Processing the streaming search results
public void ProcessTwitterStream(InputStream is, String
outFilePath) {
BufferedWriter bwrite = null;
try {
/
**
A connection to the streaming API is already
*
created and the response is contained in
*
the InpuStream
*
/
JSONTokener jsonTokener = new JSONTokener(new
InputStreamReader(is, ''UTF-8
''
));
ArrayList<JSONObject> rawtweets = new ArrayList<
JSONObject>();
int nooftweetsuploaded = 0;
//Step 1: Read until the stream is exhausted
while(true) {
try {
JSONObject temp = new JSONObject(jsonTokener);
rawtweets.add(temp);
if (rawtweets.size() >= RECORDS_TO_PROCESS){
Calendar cal = Calendar.getInstance();
String filename = outFilePath + ''tweets_
''
+
cal.getTimeInMillis() + ''.json
''
;
//Step 2: Periodically write the
processed Tweets to a file
bwrite = new BufferedWriter(new
OutputStreamWriter(new
FileOutputStream(filename),
''UTF-8
''
));
nooftweetsuploaded+=RECORDS_TO_PROCESS;
for (JSONObject jobj : rawtweets) {
bwrite.write(jobj.toString());
bwrite.newLine();
}
bwrite.close();
rawtweets.clear();
...
}
Source: Chapter2/streamingapi/StreamingApiExample.java
In method
ProcessTwitterStream
, as in Listing
2.11
, we show how the incoming
stream is processed. The input is read in the form of a continuous stream and
