Package org.apache.drill.exec.physical.base

Class AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B extends AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>>

java.lang.Object
org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>
Direct Known Subclasses:
AbstractParquetGroupScan.RowGroupScanFilterer
Enclosing class:
AbstractGroupScanWithMetadata<P extends TableMetadataProvider>
protected abstract static class AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B extends AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>> extends Object
This class is responsible for filtering different metadata levels.

  • Field Details

  • Constructor Details

  • Method Details

    • build

      public abstract AbstractGroupScanWithMetadata<?> build()
      Constructs required implementation of AbstractGroupScanWithMetadata with filtered metadata.
      Returns:
      implementation of AbstractGroupScanWithMetadata with filtered metadata

    • table

      public B table(TableMetadata tableMetadata)

    • partitions

      public B partitions(List<PartitionMetadata> partitions)

    • segments

      public B segments(Map<org.apache.hadoop.fs.Path,SegmentMetadata> segments)

    • nonInterestingColumns

      public B nonInterestingColumns(NonInterestingColumnsMetadata nonInterestingColumns)

    • files

      public B files(Map<org.apache.hadoop.fs.Path,FileMetadata> files)

    • limit

      public B limit(int maxRecords)

    • matching

      public B matching(boolean matchAllMetadata)

    • overflow

      public B overflow(MetadataType overflowLevel)

    • filterExpression

      public B filterExpression(LogicalExpression filterExpression)

    • schema

      public B schema(TupleMetadata tableSchema)

    • udfUtilities

      public B udfUtilities(UdfUtilities udfUtilities)

    • context

      public B context(FunctionLookupContext context)

    • isMatchAllMetadata

      public boolean isMatchAllMetadata()

    • getTableMetadata

      public TableMetadata getTableMetadata()

    • getPartitions

      public List<PartitionMetadata> getPartitions()

    • getSegments

      public Map<org.apache.hadoop.fs.Path,SegmentMetadata> getSegments()

    • getFiles

      public Map<org.apache.hadoop.fs.Path,FileMetadata> getFiles()

    • getOverflowLevel

      public MetadataType getOverflowLevel()

    • getFiltered

      protected B getFiltered(OptionManager optionManager, FilterPredicate<?> filterPredicate)
      Produces filtering of metadata and returns AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer to construct resulting group scan.
      Parameters:
      optionManager - option manager
      filterPredicate - filter expression
      Returns:
      this instance with filtered metadata

    • filterTableMetadata

      protected void filterTableMetadata(FilterPredicate<?> filterPredicate, Set<SchemaPath> schemaPathsInExpr)
      Produces filtering of metadata at table level.
      Parameters:
      filterPredicate - filter expression
      schemaPathsInExpr - columns used in filter expression

    • filterSegmentMetadata

      protected void filterSegmentMetadata(OptionManager optionManager, FilterPredicate<?> filterPredicate, Set<SchemaPath> schemaPathsInExpr)
      Produces filtering of metadata at segment level.
      Parameters:
      optionManager - option manager
      filterPredicate - filter expression
      schemaPathsInExpr - columns used in filter expression

    • filterPartitionMetadata

      protected void filterPartitionMetadata(OptionManager optionManager, FilterPredicate<?> filterPredicate, Set<SchemaPath> schemaPathsInExpr)
      Produces filtering of metadata at partition level.
      Parameters:
      optionManager - option manager
      filterPredicate - filter expression
      schemaPathsInExpr - columns used in filter expression

    • filterFileMetadata

      protected void filterFileMetadata(OptionManager optionManager, FilterPredicate<?> filterPredicate, Set<SchemaPath> schemaPathsInExpr)
      Produces filtering of metadata at file level.
      Parameters:
      optionManager - option manager
      filterPredicate - filter expression
      schemaPathsInExpr - columns used in filter expression

    • pruneForSegments

      protected static <T extends BaseMetadata & LocationProvider> Map<org.apache.hadoop.fs.Path,T> pruneForSegments(Map<org.apache.hadoop.fs.Path,T> metadataToPrune, Map<org.apache.hadoop.fs.Path,SegmentMetadata> filteredSegmentMetadata)
      Removes metadata which does not belong to any of partitions in metadata list.
      Type Parameters:
      T - type of metadata to filter
      Parameters:
      metadataToPrune - list of metadata which should be pruned
      filteredSegmentMetadata - list of segment metadata which was pruned
      Returns:
      map with metadata which belongs to pruned partitions

    • pruneForSegments

      protected List<PartitionMetadata> pruneForSegments(List<PartitionMetadata> metadataToPrune, Map<org.apache.hadoop.fs.Path,SegmentMetadata> filteredSegmentMetadata)
      Removes metadata which does not belong to any of partitions in metadata list.
      Parameters:
      metadataToPrune - list of partition metadata which should be pruned
      filteredSegmentMetadata - list of segment metadata which was pruned
      Returns:
      list with metadata which belongs to pruned partitions

    • filterAndGetMetadata

      public <T extends Metadata> List<T> filterAndGetMetadata(Set<SchemaPath> schemaPathsInExpr, Iterable<T> metadataList, FilterPredicate<?> filterPredicate, OptionManager optionManager)
      Produces filtering of specified metadata using specified filter expression and returns filtered metadata.
      Type Parameters:
      T - type of metadata to filter
      Parameters:
      schemaPathsInExpr - columns used in filter expression
      metadataList - metadata to filter
      filterPredicate - filter expression
      optionManager - option manager
      Returns:
      filtered metadata

    • getImplicitColumnStatistics

      protected <T extends Metadata> Map<SchemaPath,ColumnStatistics<?>> getImplicitColumnStatistics(OptionManager optionManager, T metadata, Map<SchemaPath,ColumnStatistics<?>> columnsStatistics)

    • self

      protected abstract B self()