org.apache.drill.exec.work.fragment

Class FragmentExecutor

java.lang.Object

org.apache.drill.exec.work.fragment.FragmentExecutor

All Implemented Interfaces:

Runnable

public class FragmentExecutor
extends Object
implements Runnable

Runs a single fragment on a single Drillbit. Listens/responds to status request and cancellation messages.

Theory of Operation

The FragmentExecutor runs a fragment's RootExec in the run() method in a single thread. While a fragment is running it may be subject to termination requests. The FragmentExecutor is responsible for gracefully handling termination requests for the RootExec. There are two types of termination messages:

1. Cancellation Request: This signals that the fragment and therefore the RootExec need to terminate immediately.
2. Receiver Finished: This signals that a downstream receiver no longer needs anymore data. A fragment may receive multiple receiver finished requests (one for each downstream receiver). The RootExec will only terminate once it has received FragmentExecutor.EventType.RECEIVER_FINISHED messages for all downstream receivers.

The FragmentExecutor processes termination requests appropriately for the RootExec. A Cancellation Request is signaled when cancel() is called. A Receiver Finished event is signaled when FragmentExecutor#receivingFragmentFinished(FragmentHandle) is called. The way in which these signals are handled is the following:

Cancellation Request

There are two ways in which a cancellation request can be handled when cancel() is called.

1. The Cancellation Request is received before the RootExec for the fragment is even started. In this case we can cleanup resources allocated for the fragment and never start a RootExec
2. The Cancellation Request is receive after the RootExec for the fragment is started. In this the cancellation request is sent to the FragmentEventProcessor. If this is not the first cancellation request it is ignored. If this is the first cancellation request the RootExec for this fragment is terminated by interrupting it. Then the run() thread proceeds to cleanup resources normally

Receiver Finished

When FragmentExecutor#receivingFragmentFinished(FragmentHandle) is called, the message is passed to the FragmentEventProcessor if we did not already receive a Cancellation request. Then the finished message is queued in receiverFinishedQueue. The run() polls receiverFinishedQueue and signals the RootExec with RootExec#receivingFragmentFinished(FragmentHandle) appropriately.

Possible Design Flaws / Poorly Defined Behavior

There are still a few aspects of the FragmentExecutor design that are not clear.

1. If we get a Receiver Finished message for one downstream receiver, will we eventually get one from every downstream receiver?
2. What happens when we process a Receiver Finished message for some (but not all) downstream receivers and then we cancel the fragment?
3. What happens when we process a Receiver Finished message for some (but not all) downstream receivers and then we run out of data from the upstream?

Constructor Summary

Constructors

Constructor and Description
FragmentExecutor(ExecutorFragmentContext context, BitControl.PlanFragment fragment, FragmentStatusReporter statusReporter) Create a FragmentExecutor where we need to parse and materialize the root operator.
FragmentExecutor(ExecutorFragmentContext context, BitControl.PlanFragment fragment, FragmentStatusReporter statusReporter, FragmentRoot rootOperator) Create a FragmentExecutor where we already have a root operator in memory.

Method Summary

Modifier and Type	Method and Description
void	cancel() Cancel the execution of this fragment is in an appropriate state.
ExecutorFragmentContext	getContext()
BitControl.FragmentStatus	getStatus() Returns the current fragment status if the fragment is running.
void	receivingFragmentFinished(ExecProtos.FragmentHandle handle) Inform this fragment that one of its downstream partners no longer needs additional records.
void	run()
String	toString()
void	unpause() Resume all the pauses within the current context.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

FragmentExecutor

public FragmentExecutor(ExecutorFragmentContext context,
                        BitControl.PlanFragment fragment,
                        FragmentStatusReporter statusReporter)

Create a FragmentExecutor where we need to parse and materialize the root operator.

Parameters:

context -

fragment -

statusReporter -

FragmentExecutor

public FragmentExecutor(ExecutorFragmentContext context,
                        BitControl.PlanFragment fragment,
                        FragmentStatusReporter statusReporter,
                        FragmentRoot rootOperator)

Create a FragmentExecutor where we already have a root operator in memory.

Parameters:

context -

fragment -

statusReporter -

rootOperator -

Method Detail

toString

public String toString()

Overrides:

toString in class Object

getStatus

public BitControl.FragmentStatus getStatus()

Returns the current fragment status if the fragment is running. Otherwise, returns no status.

Returns:

FragmentStatus or null.

cancel

public void cancel()

Cancel the execution of this fragment is in an appropriate state. Messages come from external.

Note: This will be called from threads Other than the one running this runnable(), so we need to be careful about the state transitions that can result.

unpause

public void unpause()

Resume all the pauses within the current context. Note that this method will be called from threads *other* than the one running this runnable(). Also, this method can be called multiple times.

receivingFragmentFinished

public void receivingFragmentFinished(ExecProtos.FragmentHandle handle)

Inform this fragment that one of its downstream partners no longer needs additional records. This is most commonly called in the case that a limit query is executed.

Parameters:

handle - The downstream FragmentHandle of the Fragment that needs no more records from this Fragment.

run

public void run()

Specified by:

run in interface Runnable

getContext

public ExecutorFragmentContext getContext()