The
work to be done in the data flow task is divided into multiple chunks,
which are called execution units, by the dataflow pipeline engine. Each
represents a group of transformations.
The individual execution unit is
called an execution tree, which can be executed by separate thread along
with other execution trees in a parallel manner.
The memory structure is
also called a data buffer, which gets created by the data flow pipeline
engine and has the scope of each individual execution tree.
An execution
tree normally starts at either the source or an asynchronous
transformation and ends at the first asynchronous transformation or a
destination.
During execution of the execution tree, the source reads the
data, then stores the data to a buffer, executes the transformation in the
buffer and passes the buffer to the next execution tree in the path by
passing the pointers to the buffers
To
see how many execution trees are getting created and how many rows are
getting stored in each buffer for a individual data flow task, you can
enable logging of these events of data flow task:
- PipelineExecutionTrees
- PipelineComponentTime
- PipelineInitialization
- BufferSizeTunning
No comments :
Post a Comment