Package org.apache.drill.exec.physical.config

Class AbstractMuxExchange

java.lang.Object
org.apache.drill.exec.physical.base.AbstractBase
org.apache.drill.exec.physical.base.AbstractSingle
org.apache.drill.exec.physical.base.AbstractExchange
org.apache.drill.exec.physical.config.AbstractMuxExchange
All Implemented Interfaces:
Iterable<PhysicalOperator>, GraphValue<PhysicalOperator>, Exchange, PhysicalOperator
Direct Known Subclasses:
OrderedMuxExchange, UnorderedMuxExchange
public abstract class AbstractMuxExchange extends AbstractExchange
Multiplexing Exchange (MuxExchange) is used when results from multiple minor fragments belonging to the same major fragment running on a node need to be collected at one fragment on the same node before distributing the results further. This helps when the sender that is distributing the results has overhead that is proportional to the number of sender instances. An example of such sender is PartitionSender. Each instance of PartitionSender allocates "r" buffers where "r" is the number of receivers. Ex. Drillbit A is assigned 10 minor fragments belonging to the same major fragment. Each of these fragments has a PartitionSender instance which is sending data to 300 receivers. Each PartitionSender needs 300 buffers, so total of 10*300 buffers are needed. With MuxExchange, all 10 fragments send the data directly (without partitioning) to MuxExchange which uses the PartitionSender to partition the incoming data and distribute to receivers. MuxExchange has only one instance per Drillbit per major fragment which means only one instance of PartitionSender per Drillbit per major fragment. With MuxExchange total number of buffers used by PartitionSender for the 10 fragments is 300 instead of earlier number 10*300.

  • Field Details

  • Constructor Details

  • Method Details

    • getReceiverParallelizationInfo

      public ParallelizationInfo getReceiverParallelizationInfo(List<CoordinationProtos.DrillbitEndpoint> senderFragmentEndpoints)
      Description copied from class: AbstractExchange
      Default receiver parallelization width range is [1, Integer.MAX_VALUE] and affinity to nodes where sender fragments are running.
      Specified by:
      getReceiverParallelizationInfo in interface Exchange
      Overrides:
      getReceiverParallelizationInfo in class AbstractExchange
      Parameters:
      senderFragmentEndpoints - Endpoints assigned to receiver fragment if available, otherwise an empty list.
      Returns:
      Receiver ParallelizationInfo.

    • getSender

      public Sender getSender(int minorFragmentId, PhysicalOperator child)
      Description copied from interface: Exchange
      Get the Sender associated with the given minorFragmentId. Cannot be called until after setupSenders() and setupReceivers() have been called.
      Parameters:
      minorFragmentId - The minor fragment id, must be in the range [0, fragment.width).
      child - The feeding node for the requested sender.
      Returns:
      The materialized sender for the given arguments.

    • getSenders

      protected final List<MinorFragmentEndpoint> getSenders(int minorFragmentId)

    • createSenderReceiverMapping

      protected void createSenderReceiverMapping()