Database Reference
In-Depth Information
TotalSegments ) will be the DynamoDB table size in GB divided by two. This will
provide the optimal TotalSegments parameter. This is the first formula.
Secondly, we can increase the TotalSegments parameter from 1 through to 4096
gradually; 1 the is minimum (and a better number to start with) and 4096 is the maximum.
But after increasing this number (every increment and every digit) we must make sure that
none of the other applications are stared of their resources. If we feel like other applica-
tions are waiting for this parallel scan to end, then we must decrease this segment count.
Tip
There are a few clients (through which we have written multithreaded program for parallel
scanning) that don't support these many threads. So we need to keep this in our mind too.
The best way of running a parallel scan is letting multiple worker threads run in parallel,
but with low priority, so that other processes and applications will not get affected.
Let's get our hands dirty by visualizing parallel scanning using Java SDK. The first ob-
jective is to create a class to write the code for parallel scanning, implementing the Run-
nable interface. This class has only two methods. The first method is the parameterized
constructor initializing the instance variables. The variable explanation can be referred to
as follows:
client : This initializes the DynamoDB client-related properties, such as AWS
credentials, and so on
itemLimit : This is the maximum number of items to be retrieved by a single
scan request
totalSegments : This is the integer specifying the number of segments to be
created for this table scan
segment : This initializes the segment number running this thread
To understand the variables and their working, go through the following code:
public class ParallelScanner implements Runnable{
private AmazonDynamoDBClient client;
private int itemLimit;
private int totalSegments;
private int segment;
public ParallelScanner(AmazonDynamoDBClient client, int
itemLimit, int totalSegments, int segment) {
Search WWH ::




Custom Search