Class DistributedQueryQueue

All Implemented Interfaces:

public class DistributedQueryQueue extends Object implements QueryQueue
Distributed query queue which uses a Zookeeper distributed semaphore to control queuing across the cluster. The distributed queue is actually two queues: one for "small" queries, another for "large" queries. Query size is determined by the Planner's estimate of query cost.

This queue is configured using system options:

Set to true to enable the distributed queue.
The maximum number of large queries to admit. Additional queries wait in the queue.
The maximum number of small queries to admit. Additional queries wait in the queue.
The cost threshold. Queries below this size are small, at or above this size are large..
The maximum time (in milliseconds) a query will wait in the queue before failing.

The above values are refreshed every five seconds. This aids performance a bit in systems with very high query arrival rates.

  • Constructor Details

  • Method Details

    • setMemoryPerNode

      public void setMemoryPerNode(long memoryPerNode)
      Specified by:
      setMemoryPerNode in interface QueryQueue
    • defaultQueryMemoryPerNode

      public long defaultQueryMemoryPerNode(double cost)
      Description copied from interface: QueryQueue
      Return the amount of memory per node when creating a EXPLAIN query plan. Plans to be executed should get the query memory from the lease, as the lease may adjust the default amount on a per-query basis. This means that the memory used to execute the query may differ from the amount shown in an EXPLAIN plan.
      Specified by:
      defaultQueryMemoryPerNode in interface QueryQueue
      assumed memory per node, in bytes, to use when creating an EXPLAIN plan
    • minimumOperatorMemory

      public long minimumOperatorMemory()
      Description copied from interface: QueryQueue
      Optional floor on the amount of memory assigned per operator. This ensures that operators receive a certain amount, separate from any memory slicing. This can oversubscribe node memory if used incorrectly.
      Specified by:
      minimumOperatorMemory in interface QueryQueue
      minimum per-operator memory, in bytes
    • enqueue

      This limits the number of "small" and "large" queries that a Drill cluster will run simultaneously, if queuing is enabled. If the query is unable to run, this will block until it can. Beware that this is called under run(), and so will consume a thread while it waits for the required distributed semaphore.
      Specified by:
      enqueue in interface QueryQueue
      queryId - query identifier
      cost - the query plan
      the query lease which must be passed to #release(QueueLease) upon query completion
      QueryQueue.QueryQueueException - if the underlying ZK queuing mechanism fails
      QueryQueue.QueueTimeoutException - if the query waits too long in the queue
    • enabled

      public boolean enabled()
      Description copied from interface: QueryQueue
      Determine if the queue is enabled.
      Specified by:
      enabled in interface QueryQueue
      true if the query is enabled, false otherwise.
    • getInfo

    • close

      public void close()
      Specified by:
      close in interface QueryQueue