org.apache.drill.yarn.appMaster

Class PersistentTaskScheduler

java.lang.Object

org.apache.drill.yarn.appMaster.AbstractScheduler

org.apache.drill.yarn.appMaster.PersistentTaskScheduler

All Implemented Interfaces:

Scheduler

Direct Known Subclasses:

AbstractDrillbitScheduler

public abstract class PersistentTaskScheduler
extends AbstractScheduler

Abstract base class for schedulers that work with persistent (long-running) tasks. Such tasks are intended to run until explicitly shut down (unlike batch tasks that run until some expected completion.)

Provides a target quantity of tasks (see getTarget(), along with operations to increase, decrease or set the target number.

The scheduler acts as a controller: starting new tasks as needed to match the desired target, or stopping tasks as needed when the target level is reduced.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.drill.yarn.appMaster.Scheduler

Scheduler.TaskManager

Field Summary

Fields

Modifier and Type	Field and Description
protected int	quantity

Fields inherited from class org.apache.drill.yarn.appMaster.AbstractScheduler

failCount, isTracked, priority, state, taskManager, taskSpec

Constructor Summary

Constructors

Constructor and Description
PersistentTaskScheduler(String type, String name, int quantity)

Method Summary

Modifier and Type	Method and Description
void	adjust() Adjust the number of running tasks to better match the target by starting or stopping tasks as needed.
void	completed(Task task) Indicate that a task is completed.
int[]	getProgress() Progress for persistent tasks defaults to the ratio of running tasks to target level.
int	getTarget() Get the desired number of running tasks.
boolean	hasMoreTasks() The persistent scheduler has no fixed sequence of tasks to run, it launches a set and is never "done".
void	requestTimedOut() Informs the scheduler that a YARN resource request timed out.
int	resize(int level) Set the number of running tasks to the quantity given.

Methods inherited from class org.apache.drill.yarn.appMaster.AbstractScheduler

addTasks, change, getName, getRequestTimeoutSec, getResource, getTaskManager, getType, isTracked, limitContainerSize, registerState, setPriority, setTaskManager

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

quantity

protected int quantity

Constructor Detail

PersistentTaskScheduler

public PersistentTaskScheduler(String type,
                               String name,
                               int quantity)

Method Detail

resize

public int resize(int level)

Set the number of running tasks to the quantity given.

Parameters:

level - the target number of tasks

Returns:

the actual resize level, which may be lower than the requested level if the system cannot provide the requested level

getTarget

public int getTarget()

Description copied from interface: Scheduler

Get the desired number of running tasks.

Returns:

The desired number of running tasks

completed

public void completed(Task task)

Indicate that a task is completed. Normally occurs only when shutting down excess tasks.

Parameters:

task -

getProgress

public int[] getProgress()

Progress for persistent tasks defaults to the ratio of running tasks to target level. Thus, a persistent cluster will normally report 100% progress.

Returns:

The progress of persistent tasks.

adjust

public void adjust()

Adjust the number of running tasks to better match the target by starting or stopping tasks as needed.

hasMoreTasks

public boolean hasMoreTasks()

The persistent scheduler has no fixed sequence of tasks to run, it launches a set and is never "done". For purposes of completion tracking claim we have no further tasks.

Returns:

false

requestTimedOut

public void requestTimedOut()

Description copied from interface: Scheduler

Informs the scheduler that a YARN resource request timed out. The scheduler can either retry or (more productively) assume that the requested node is not available and adjust its target size downward.