org.apache.drill.yarn.zk

Class ZKRegistry

java.lang.Object

org.apache.drill.yarn.zk.ZKRegistry

All Implemented Interfaces:

DrillbitStatusListener, Pollable, TaskLifecycleListener

public class ZKRegistry
extends Object
implements TaskLifecycleListener, DrillbitStatusListener, Pollable

AM-specific implementation of a Drillbit registry backed by ZooKeeper. Listens to ZK events for registering a Drillbit and deregistering. Alerts the Cluster Controller of these events.

Locking strategy: Receives events from both ZK and the cluster controller, both of which must be synchronized. To prevent deadlocks, this class NEVER calls into the cluster controller while holding a lock. This prevents the following:

ClusterController --> ZKRegistry (OK)
ZK --> ZKRegistry (OK)
ZK --> ZKRegistry --> Cluster Controller (bad)

In the case of registration, ZK calls the registry which must alert the cluster controller. Cluster controller alerting is handled outside the ZK update critical section.

Because ZK events are occur relatively infrequently, any deadlock will occur once in a blue moon, which will make it very hard to reproduce. So, extreme caution is needed at design time to prevent the problem.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ZKRegistry.AckEvent
protected static class	ZKRegistry.DrillbitTracker State of each Drillbit that we've discovered through ZK or launched via the AM.

Nested classes/interfaces inherited from interface org.apache.drill.yarn.appMaster.TaskLifecycleListener

TaskLifecycleListener.Event

Field Summary

Fields

Modifier and Type	Field and Description
static String	CONTROLLER_PROPERTY
static String	ENDPOINT_PROPERTY
static int	UPDATE_PERIOD_MS

Constructor Summary

Constructors

Constructor and Description
ZKRegistry(ZKClusterCoordinatorDriver zkDriver)

Method Summary

Modifier and Type	Method and Description
void	drillbitRegistered(Set<CoordinationProtos.DrillbitEndpoint> registeredDrillbits) Callback from ZK to indicate that one or more drillbits have become registered.
void	drillbitUnregistered(Set<CoordinationProtos.DrillbitEndpoint> unregisteredDrillbits) Callback from ZK to indicate that one or more drillbits have become deregistered from ZK.
void	finish(RegistryHandler handler)
protected Map<String,ZKRegistry.DrillbitTracker>	getRegistryForTesting() Get the current registry for testing.
boolean	isRegistered(Task task) Report whether the given task is still registered in ZK.
List<String>	listUnmanagedDrillits()
void	start(RegistryHandler controller) Called during AM startup to initialize ZK.
void	stateChange(TaskLifecycleListener.Event event, EventContext context) Listen for selected YARN task state changes.
void	tick(long curTime) Periodically check ZK status.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CONTROLLER_PROPERTY

public static final String CONTROLLER_PROPERTY

See Also:

Constant Field Values

UPDATE_PERIOD_MS

public static final int UPDATE_PERIOD_MS

See Also:

Constant Field Values

ENDPOINT_PROPERTY

public static final String ENDPOINT_PROPERTY

See Also:

Constant Field Values

Constructor Detail

ZKRegistry

public ZKRegistry(ZKClusterCoordinatorDriver zkDriver)

Method Detail

start

public void start(RegistryHandler controller)

Called during AM startup to initialize ZK. Checks if any Drillbits are already running. These are "unmanaged" because the AM could not have started them (since they predate the AM.)

drillbitRegistered

public void drillbitRegistered(Set<CoordinationProtos.DrillbitEndpoint> registeredDrillbits)

Callback from ZK to indicate that one or more drillbits have become registered. We handle registrations in a critical section, then alert the cluster controller outside the critical section.

Specified by:

drillbitRegistered in interface DrillbitStatusListener

Parameters:

registeredDrillbits - the set of newly registered drillbits. Note: the complete set of currently registered bits could be different.

drillbitUnregistered

public void drillbitUnregistered(Set<CoordinationProtos.DrillbitEndpoint> unregisteredDrillbits)

Callback from ZK to indicate that one or more drillbits have become deregistered from ZK. We handle the deregistrations in a critical section, but updates to the cluster controller outside of a critical section.

Specified by:

drillbitUnregistered in interface DrillbitStatusListener

Parameters:

unregisteredDrillbits - the set of newly unregistered drillbits.

stateChange

public void stateChange(TaskLifecycleListener.Event event,
                        EventContext context)

Listen for selected YARN task state changes. Called from within the cluster controller's critical section.

Specified by:

stateChange in interface TaskLifecycleListener

isRegistered

public boolean isRegistered(Task task)

Report whether the given task is still registered in ZK. Called while waiting for a deregistration event to catch possible cases where the messages is lost. The message should never be lost, but we've seen cases where tasks hang in this state. This is a potential work-around.

Parameters:

task -

Returns:

True if the given task is regestered. False otherwise.

tick

public void tick(long curTime)

Periodically check ZK status. If the ZK connection has timed out, something is very seriously wrong. Shut the whole Drill cluster down since Drill cannot operate without ZooKeeper.

This method should not be synchronized. It checks only the ZK state, not internal state. Further, if we do reconnect to ZK, then a ZK thread may attempt to update this registry, which will acquire a synchronization lock.

Specified by:

tick in interface Pollable

finish

public void finish(RegistryHandler handler)

listUnmanagedDrillits

public List<String> listUnmanagedDrillits()

getRegistryForTesting

protected Map<String,ZKRegistry.DrillbitTracker> getRegistryForTesting()

Get the current registry for testing. Why for testing? Because this is unsynchronized. In production code, the map may change out from under you.

Returns:

The current registry.