org.apache.drill.exec.planner.fragment

Interface QueryParallelizer

All Superinterfaces:

ParallelizationParameters

All Known Implementing Classes:

DefaultQueryParallelizer, QueueQueryParallelizer, SimpleParallelizer, SplittingParallelizer

public interface QueryParallelizer
extends ParallelizationParameters

Parallelizes the query plan. Once the optimizer finishes its job by producing a optimized plan, it is the job of this parallelizer to generate a parallel plan out of the optimized physical plan. It does so by using the optimizers estimates for row count etc. There are two kinds of parallelizers as explained below. Currently the difference in both of these parallelizers is only in the memory assignment for the physical operators.

a) Default Parallelizer: It optimistically assumes that the whole cluster is running only the current query and based on heuristics assigns the optimal memory to the buffered operators.

b) Queue Parallelizer: This parallelizer computes the memory that can be allocated at best based on the current cluster state(as to how much memory is available) and also the configuration of the queue that it can run on.

Method Summary

Modifier and Type	Method and Description
QueryWorkUnit	generateWorkUnit(OptionList options, CoordinationProtos.DrillbitEndpoint foremanNode, UserBitShared.QueryId queryId, Collection<CoordinationProtos.DrillbitEndpoint> activeEndpoints, Fragment rootFragment, UserSession session, BitControl.QueryContextInformation queryContextInfo) This is the only function exposed to the consumer of this parallelizer (currently Foreman) to parallelize the plan.

Methods inherited from interface org.apache.drill.exec.planner.fragment.ParallelizationParameters

getAffinityFactor, getMaxGlobalWidth, getMaxWidthPerNode, getSliceTarget

Method Detail

generateWorkUnit

QueryWorkUnit generateWorkUnit(OptionList options,
                               CoordinationProtos.DrillbitEndpoint foremanNode,
                               UserBitShared.QueryId queryId,
                               Collection<CoordinationProtos.DrillbitEndpoint> activeEndpoints,
                               Fragment rootFragment,
                               UserSession session,
                               BitControl.QueryContextInformation queryContextInfo)
                        throws ExecutionSetupException

This is the only function exposed to the consumer of this parallelizer (currently Foreman) to parallelize the plan. The caller transforms the plan to a fragment tree and supplies the required information for the parallelizer to do its job.

Parameters:

options - List of all options that are set for the current session.

foremanNode - Endpoint information of the foreman node.

queryId - Unique ID of the query.

activeEndpoints - Currently active endpoints on which the plan can run.

rootFragment - root of the fragment tree of the transformed physical plan

session - user session object.

queryContextInfo - query context.

Returns:

Executable query plan which contains all the information like minor frags and major frags.

Throws:

ExecutionSetupException