org.apache.drill.yarn.appMaster

Interface ClusterController

All Superinterfaces:

RegistryHandler

All Known Implementing Classes:

ClusterControllerImpl

public interface ClusterController
extends RegistryHandler

Interface which identifies the cluster controller methods that are save to call from the Dispatcher. Methods here are either designed to be called before the event threads start or after they complete. The remainder synchronized to coordinate between event threads.

Method Summary

Modifier and Type	Method and Description
boolean	cancelTask(int id) Cancels the given task, reducing the target task count.
void	containersAllocated(List<org.apache.hadoop.yarn.api.records.Container> containers) The RM has allocated one or more containers in response to container requests submitted to the RM.
void	containersCompleted(List<org.apache.hadoop.yarn.api.records.ContainerStatus> statuses) The Resource Manager reports that containers have completed with the given statuses.
void	containerStarted(org.apache.hadoop.yarn.api.records.ContainerId containerId) The NM reports that a container has successfully started.
void	containerStopped(org.apache.hadoop.yarn.api.records.ContainerId containerId) The Node Manager reports that a container has stopped.
void	enableFailureCheck(boolean flag)
int	getFreeNodeCount()
float	getProgress()
Object	getProperty(String key)
int	getTargetCount() Return the target number of tasks that the controller seeks to maintain.
boolean	isTaskLive(int id)
void	registerLifecycleListener(TaskLifecycleListener listener)
void	registerScheduler(Scheduler resourceGroup)
void	resizeDelta(int delta) Request to resize the Drill cluster by a relative amount.
int	resizeTo(int n) Request to resize the Drill cluster to the given size.
void	setMaxRetries(int value)
void	setProperty(String key, Object value)
void	shutDown() Indicates a request to gracefully shut down the cluster.
void	started() Called after the dispatcher has started YARN and other server components.
void	stopTaskFailed(org.apache.hadoop.yarn.api.records.ContainerId containerId, Throwable t) The Node Manager API reports that a request sent to the NM to stop a task has failed.
boolean	supportsDiskResource() Whether this distribution of YARN supports disk resources.
void	taskStartFailed(org.apache.hadoop.yarn.api.records.ContainerId containerId, Throwable t) The RM API reports that an attempt to start a container has failed locally.
void	tick(long curTime) Called by the timer ("pulse") thread to trigger time-based events.
void	updateRMStatus()
void	visit(ControllerVisitor visitor) Allow an observer to see a consistent view of the controller's state by performing the visit in a synchronized block.
void	visitTasks(TaskVisitor visitor) Allow an observer to see a consistent view of the controller's task state by performing the visit in a synchronized block.
boolean	waitForCompletion() Called by the main thread to wait for the normal shutdown of the controller.

Methods inherited from interface org.apache.drill.yarn.appMaster.RegistryHandler

completionAck, registryDown, releaseHost, reserveHost, startAck

Method Detail

enableFailureCheck

void enableFailureCheck(boolean flag)

registerLifecycleListener

void registerLifecycleListener(TaskLifecycleListener listener)

registerScheduler

void registerScheduler(Scheduler resourceGroup)

setProperty

void setProperty(String key,
                 Object value)

getProperty

Object getProperty(String key)

started

void started()
      throws YarnFacadeException,
             AMException

Called after the dispatcher has started YARN and other server components. The controller can now begin to spin up tasks.

Throws:

YarnFacadeException

AMException

tick

void tick(long curTime)

Called by the timer ("pulse") thread to trigger time-based events.

Parameters:

curTime -

containersAllocated

void containersAllocated(List<org.apache.hadoop.yarn.api.records.Container> containers)

The RM has allocated one or more containers in response to container requests submitted to the RM.

Parameters:

containers - the set of containers provided by YARN

containerStarted

void containerStarted(org.apache.hadoop.yarn.api.records.ContainerId containerId)

The NM reports that a container has successfully started.

Parameters:

containerId - the container which started

taskStartFailed

void taskStartFailed(org.apache.hadoop.yarn.api.records.ContainerId containerId,
                     Throwable t)

The RM API reports that an attempt to start a container has failed locally.

Parameters:

containerId - the container that failed to launch

t - the error that occurred

containerStopped

void containerStopped(org.apache.hadoop.yarn.api.records.ContainerId containerId)

The Node Manager reports that a container has stopped.

Parameters:

containerId -

containersCompleted

void containersCompleted(List<org.apache.hadoop.yarn.api.records.ContainerStatus> statuses)

The Resource Manager reports that containers have completed with the given statuses. Find the task for each container and mark them as completed.

Parameters:

statuses -

getProgress

float getProgress()

stopTaskFailed

void stopTaskFailed(org.apache.hadoop.yarn.api.records.ContainerId containerId,
                    Throwable t)

The Node Manager API reports that a request sent to the NM to stop a task has failed.

Parameters:

containerId - the container that failed to stop

t - the reason that the stop request failed

resizeDelta

void resizeDelta(int delta)

Request to resize the Drill cluster by a relative amount.

Parameters:

delta - the amount of change. Can be positive (to grow) or negative (to shrink the cluster)

resizeTo

int resizeTo(int n)

Request to resize the Drill cluster to the given size.

Parameters:

n - the desired cluster size

shutDown

void shutDown()

Indicates a request to gracefully shut down the cluster.

waitForCompletion

boolean waitForCompletion()

Called by the main thread to wait for the normal shutdown of the controller. Such shutdown occurs when the admin sends a sutdown command from the UI or REST API.

updateRMStatus

void updateRMStatus()

setMaxRetries

void setMaxRetries(int value)

visit

void visit(ControllerVisitor visitor)

Allow an observer to see a consistent view of the controller's state by performing the visit in a synchronized block.

Parameters:

visitor -

visitTasks

void visitTasks(TaskVisitor visitor)

Allow an observer to see a consistent view of the controller's task state by performing the visit in a synchronized block.

Parameters:

visitor -

getTargetCount

int getTargetCount()

Return the target number of tasks that the controller seeks to maintain. This is the sum across all pools.

isTaskLive

boolean isTaskLive(int id)

cancelTask

boolean cancelTask(int id)

Cancels the given task, reducing the target task count. Called from the UI to allow the user to select the specific task to end when reducing cluster size.

Parameters:

id -

supportsDiskResource

boolean supportsDiskResource()

Whether this distribution of YARN supports disk resources.

Returns:

True if this distribution of YARN supports disk resources. False otherwise.

getFreeNodeCount

int getFreeNodeCount()