Class PersistentTaskScheduler

All Implemented Interfaces:
Direct Known Subclasses:

public abstract class PersistentTaskScheduler extends AbstractScheduler
Abstract base class for schedulers that work with persistent (long-running) tasks. Such tasks are intended to run until explicitly shut down (unlike batch tasks that run until some expected completion.)

Provides a target quantity of tasks (see getTarget(), along with operations to increase, decrease or set the target number.

The scheduler acts as a controller: starting new tasks as needed to match the desired target, or stopping tasks as needed when the target level is reduced.

  • Field Details

    • quantity

      protected int quantity
  • Constructor Details

    • PersistentTaskScheduler

      public PersistentTaskScheduler(String type, String name, int quantity)
  • Method Details

    • resize

      public int resize(int level)
      Set the number of running tasks to the quantity given.
      level - the target number of tasks
      the actual resize level, which may be lower than the requested level if the system cannot provide the requested level
    • getTarget

      public int getTarget()
      Description copied from interface: Scheduler
      Get the desired number of running tasks.
      The desired number of running tasks
    • completed

      public void completed(Task task)
      Indicate that a task is completed. Normally occurs only when shutting down excess tasks.
      task -
    • getProgress

      public int[] getProgress()
      Progress for persistent tasks defaults to the ratio of running tasks to target level. Thus, a persistent cluster will normally report 100% progress.
      The progress of persistent tasks.
    • adjust

      public void adjust()
      Adjust the number of running tasks to better match the target by starting or stopping tasks as needed.
    • hasMoreTasks

      public boolean hasMoreTasks()
      The persistent scheduler has no fixed sequence of tasks to run, it launches a set and is never "done". For purposes of completion tracking claim we have no further tasks.
    • requestTimedOut

      public void requestTimedOut()
      Description copied from interface: Scheduler
      Informs the scheduler that a YARN resource request timed out. The scheduler can either retry or (more productively) assume that the requested node is not available and adjust its target size downward.