org.apache.drill.yarn.appMaster

Interface Scheduler

All Known Implementing Classes:

AbstractDrillbitScheduler, AbstractScheduler, BatchScheduler, DrillbitScheduler, PersistentTaskScheduler

public interface Scheduler

The scheduler describes the set of tasks to run. It provides the details required to launch each task and optionally a specification of the containers required to run the task.

Schedulers can manage batch task (which do their job and complete), or persistent tasks (which run until terminated.)

The scheduler tracks task completion (for batch tasks) and task levels (for persistent tasks.)

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	Scheduler.TaskManager

Method Summary

Modifier and Type	Method and Description
void	adjust() Adjust the number of running tasks to better track the desired number.
void	change(int delta) Increase (positive) or decrease (negative) the number of desired tasks by the given amount.
void	completed(Task task)
String	getName()
int[]	getProgress() Return an estimate of progress given as a ratio of (work completed, total work).
int	getRequestTimeoutSec() Maximum amount of time to wait when cancelling a job in the REQUESTING state.
ContainerRequestSpec	getResource() For reporting, get the YARN resources requested by processes in this pool.
int	getTarget() Get the desired number of running tasks.
Scheduler.TaskManager	getTaskManager()
String	getType()
boolean	hasMoreTasks() If this is a batch scheduler, whether all tasks for the batch have completed.
boolean	isTracked() Whether tasks from this scheduler should incorporate app startup/shutdown acknowledgements (acks) into the task lifecycle.
void	limitContainerSize(org.apache.hadoop.yarn.api.records.Resource maxResource)
void	registerState(SchedulerState state) Register the state object that tracks tasks launched by this scheduler.
void	requestTimedOut() Informs the scheduler that a YARN resource request timed out.
int	resize(int level) Set the number of desired tasks to the given level.
void	setPriority(int priority) Controller-assigned priority for this scheduler.

Method Detail

setPriority

void setPriority(int priority)

Controller-assigned priority for this scheduler. Used to differentiate container requests by scheduler.

Parameters:

priority -

registerState

void registerState(SchedulerState state)

Register the state object that tracks tasks launched by this scheduler.

Parameters:

state -

getName

String getName()

getType

String getType()

isTracked

boolean isTracked()

Whether tasks from this scheduler should incorporate app startup/shutdown acknowledgements (acks) into the task lifecycle.

getTaskManager

Scheduler.TaskManager getTaskManager()

getTarget

int getTarget()

Get the desired number of running tasks.

Returns:

The desired number of running tasks

change

void change(int delta)

Increase (positive) or decrease (negative) the number of desired tasks by the given amount.

Parameters:

delta -

resize

int resize(int level)

Set the number of desired tasks to the given level.

Parameters:

level -

Returns:

the actual resize level, which may be lower than the requested level if the system cannot provide the requested level

completed

void completed(Task task)

adjust

void adjust()

Adjust the number of running tasks to better track the desired number. Starts or stops tasks using the SchedulerState registered with registerState(SchedulerState).

getProgress

int[] getProgress()

Return an estimate of progress given as a ratio of (work completed, total work).

Returns:

Estimate of progress.

hasMoreTasks

boolean hasMoreTasks()

If this is a batch scheduler, whether all tasks for the batch have completed. If this is a persistent task scheduler, always returns false.

Returns:

true if the scheduler has more tasks to run, false if the scheduler has no more tasks or manages a set of long-running tasks

getResource

ContainerRequestSpec getResource()

For reporting, get the YARN resources requested by processes in this pool.

Returns:

The request spec.

limitContainerSize

void limitContainerSize(org.apache.hadoop.yarn.api.records.Resource maxResource)
                 throws AMException

Throws:

AMException

getRequestTimeoutSec

int getRequestTimeoutSec()

Maximum amount of time to wait when cancelling a job in the REQUESTING state. YARN will happily wait forever for a resource, this setting forcibly cancels the request at timeout.

Returns:

the number of seconds to wait for timeout. 0 means no timeout

requestTimedOut

void requestTimedOut()

Informs the scheduler that a YARN resource request timed out. The scheduler can either retry or (more productively) assume that the requested node is not available and adjust its target size downward.