org.apache.drill.yarn.appMaster

Class NodeInventory

java.lang.Object

org.apache.drill.yarn.appMaster.NodeInventory

public class NodeInventory
extends Object

Creates an AM-side inventory of cluster nodes. Used to track node reservations (container allocations) to prevent requesting multiple containers on the same node. Tracks blacklisted nodes that have failed too often. Since YARN will discard our blacklist if we add to many nodes, tracks when a container is allocated on a blacklisted node and signals that the cluster is in a bad state.

Constructor Summary

Constructors

Constructor and Description
NodeInventory(AMYarnFacade yarn)

Method Summary

Modifier and Type	Method and Description
void	blacklist(String hostName)
List<String>	getBlacklist() Return a copy of the blacklist (list of failed nodes) for use in display to the user or similar purpose.
int	getFreeNodeCount() Determine the number of free nodes in the YARN cluster.
boolean	isFailed()
boolean	isInUse(String hostName) Report if the given host name is in use.
void	release(org.apache.hadoop.yarn.api.records.Container container)
void	release(String hostName)
void	reserve(org.apache.hadoop.yarn.api.records.Container container)
void	reserve(String hostName)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

NodeInventory

public NodeInventory(AMYarnFacade yarn)
              throws YarnFacadeException

Throws:

YarnFacadeException

Method Detail

isFailed

public boolean isFailed()

reserve

public void reserve(org.apache.hadoop.yarn.api.records.Container container)

reserve

public void reserve(String hostName)

release

public void release(org.apache.hadoop.yarn.api.records.Container container)

release

public void release(String hostName)

blacklist

public void blacklist(String hostName)

getFreeNodeCount

public int getFreeNodeCount()

Determine the number of free nodes in the YARN cluster. The free set is the set of all YARN nodes minus those that are allocated and those that are blacklisted. Note that a node might be both in use and blacklisted if DoY blacklists a node, but then the user starts a "stray" Drillbit on that same node.

This number is an approximation: the set of nodes managed by YARN can change any time, and in-flight container requests will consume a node, but since the request is not yet completed, we don't know which node will be assigned, so the node does not yet appear in the in-use list.

Returns:

an approximation of the free node count

getBlacklist

public List<String> getBlacklist()

Return a copy of the blacklist (list of failed nodes) for use in display to the user or similar purpose.

Returns:

a copy of the blacklist.

isInUse

public boolean isInUse(String hostName)

Report if the given host name is in use.

Parameters:

hostName -

Returns:

true if the host is reserved (in use by a container) or blacklisted (failed.)