Information Technology Reference
In-Depth Information
is running. To help you get the best performance out of PLINQ, there are
several methods that control how the parallel task library functions are
accessible using IParallelEnumerable.
Every parallel query begins with a partitioning step. PLINQ needs to par-
tition the input elements and distribute those over the number of tasks
created to perform the query. Partitioning is one of the most important
aspects of PLINQ, so it is important to understand the different
approaches, how PLINQ decides which to use, and how each one works.
First, partitioning can't take much time. That would cause the PLINQ
library to spend too much time partitioning, and too little time actually
processing your data. PLINQ uses four different partitioning algorithms,
based on the input source and the type of query you are creating. The sim-
plest algorithm is range partitioning. Range partitioning divides the input
sequence by the number of tasks and gives each task one set of items. For
example, an input sequence with 1,000 items running on a quad core
machine would create four ranges of 250 items each. Range partitioning
is used only when the query source supports indexing the sequence and
reports how many items are in the sequence. That means range partition-
ing is limited to query sources that are like List<T>, arrays, and other
sequences that support the IList<T> interface. Range partitioning is usu-
ally used when the source of the query supports those operations.
The second choice for partitioning is chunk partitioning. This algorithm
gives each task a “chunk” of input items anytime it requests more work.
The internals of the chunking algorithm will continue to change over time,
so I won't cover the current implementation in depth. You can expect that
the size of chunks will start small, because an input sequence may be small.
That prevents the situation where one task must process an entire small
sequence. You can also expect that as work continues, chunks may grow in
size. That minimizes the threading overhead and helps to maximize
throughput. Chunks may also change in size depending on the time cost
for delegates in the query and the number of elements rejected by
where
clauses. The goal is to have all tasks finish at close to the same time to max-
imize the overall throughput.
The other two partitioning schemes optimize for certain query operations.
First is a striped partition. A striped partition is a special case of range par-
titioning that optimizes processing the beginning elements of a sequence.
Each of the worker threads processes items by skipping N items and then
processing the next M. After processing M items, the worker thread will
