Package org.apache.drill.exec.physical.impl.partitionsender

Class PartitionerDecorator

java.lang.Object
org.apache.drill.exec.physical.impl.partitionsender.PartitionerDecorator
public final class PartitionerDecorator extends Object
Decorator class to hide multiple Partitioner existence from the caller since this class involves multithreaded processing of incoming batches as well as flushing it needs special handling of OperatorStats - stats since stats are not suitable for use in multithreaded environment The algorithm to figure out processing versus wait time is based on following formula: totalWaitTime = totalAllPartitionersProcessingTime - max(sum(processingTime) by partitioner)

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    protected static interface 
    PartitionerDecorator.GeneralExecuteIface
    Helper interface to generalize functionality executed in the thread since it is absolutely the same for partitionBatch and flushOutgoingBatches protected is for testing purposes

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    clear()
    decorator method to call multiple Partitioners clear()
    void
    flushOutgoingBatches(boolean isLastBatch, boolean schemaChanged)
    flushOutgoingBatches - decorator to call real Partitioner(s) flushOutgoingBatches
    PartitionOutgoingBatch
    getOutgoingBatches(int index)
    Helper method to get PartitionOutgoingBatch based on the index since we may have more then one Partitioner As number of Partitioners should be very small AND this method it used very rarely, so it is OK to loop in order to find right partitioner
    void
    initialize()
    decorator method to call multiple Partitioners initialize()
    void
    partitionBatch(RecordBatch incoming)
    partitionBatch - decorator method to call real Partitioner(s) to process incoming batch uses either threading or not threading approach based on number Partitioners

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • partitionBatch

      public void partitionBatch(RecordBatch incoming) throws ExecutionException
      partitionBatch - decorator method to call real Partitioner(s) to process incoming batch uses either threading or not threading approach based on number Partitioners
      Parameters:
      incoming -
      Throws:
      ExecutionException

    • flushOutgoingBatches

      public void flushOutgoingBatches(boolean isLastBatch, boolean schemaChanged) throws ExecutionException
      flushOutgoingBatches - decorator to call real Partitioner(s) flushOutgoingBatches
      Parameters:
      isLastBatch -
      schemaChanged -
      Throws:
      ExecutionException

    • initialize

      public void initialize()
      decorator method to call multiple Partitioners initialize()

    • clear

      public void clear()
      decorator method to call multiple Partitioners clear()

    • getOutgoingBatches

      public PartitionOutgoingBatch getOutgoingBatches(int index)
      Helper method to get PartitionOutgoingBatch based on the index since we may have more then one Partitioner As number of Partitioners should be very small AND this method it used very rarely, so it is OK to loop in order to find right partitioner
      Parameters:
      index - - index of PartitionOutgoingBatch
      Returns:
      PartitionOutgoingBatch