Class AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B extends AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>>

java.lang.Object
org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>
Direct Known Subclasses:
AbstractParquetGroupScan.RowGroupScanFilterer
Enclosing class:
AbstractGroupScanWithMetadata<P extends TableMetadataProvider>

protected abstract static class AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B extends AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>> extends Object
This class is responsible for filtering different metadata levels.
  • Field Details

  • Constructor Details

  • Method Details

    • build

      public abstract AbstractGroupScanWithMetadata<?> build()
      Constructs required implementation of AbstractGroupScanWithMetadata with filtered metadata.
      Returns:
      implementation of AbstractGroupScanWithMetadata with filtered metadata
    • table

      public B table(TableMetadata tableMetadata)
    • partitions

      public B partitions(List<PartitionMetadata> partitions)
    • segments

      public B segments(Map<org.apache.hadoop.fs.Path,SegmentMetadata> segments)
    • nonInterestingColumns

      public B nonInterestingColumns(NonInterestingColumnsMetadata nonInterestingColumns)
    • files

      public B files(Map<org.apache.hadoop.fs.Path,FileMetadata> files)
    • limit

      public B limit(int maxRecords)
    • matching

      public B matching(boolean matchAllMetadata)
    • overflow

      public B overflow(MetadataType overflowLevel)
    • filterExpression

      public B filterExpression(LogicalExpression filterExpression)
    • schema

      public B schema(TupleMetadata tableSchema)
    • udfUtilities

      public B udfUtilities(UdfUtilities udfUtilities)
    • context

      public B context(FunctionLookupContext context)
    • isMatchAllMetadata

      public boolean isMatchAllMetadata()
    • getTableMetadata

      public TableMetadata getTableMetadata()
    • getPartitions

      public List<PartitionMetadata> getPartitions()
    • getSegments

      public Map<org.apache.hadoop.fs.Path,SegmentMetadata> getSegments()
    • getFiles

      public Map<org.apache.hadoop.fs.Path,FileMetadata> getFiles()
    • getOverflowLevel

      public MetadataType getOverflowLevel()
    • getFiltered

      protected B getFiltered(OptionManager optionManager, FilterPredicate<?> filterPredicate)
      Produces filtering of metadata and returns AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer to construct resulting group scan.
      Parameters:
      optionManager - option manager
      filterPredicate - filter expression
      Returns:
      this instance with filtered metadata
    • filterTableMetadata

      protected void filterTableMetadata(FilterPredicate<?> filterPredicate, Set<SchemaPath> schemaPathsInExpr)
      Produces filtering of metadata at table level.
      Parameters:
      filterPredicate - filter expression
      schemaPathsInExpr - columns used in filter expression
    • filterSegmentMetadata

      protected void filterSegmentMetadata(OptionManager optionManager, FilterPredicate<?> filterPredicate, Set<SchemaPath> schemaPathsInExpr)
      Produces filtering of metadata at segment level.
      Parameters:
      optionManager - option manager
      filterPredicate - filter expression
      schemaPathsInExpr - columns used in filter expression
    • filterPartitionMetadata

      protected void filterPartitionMetadata(OptionManager optionManager, FilterPredicate<?> filterPredicate, Set<SchemaPath> schemaPathsInExpr)
      Produces filtering of metadata at partition level.
      Parameters:
      optionManager - option manager
      filterPredicate - filter expression
      schemaPathsInExpr - columns used in filter expression
    • filterFileMetadata

      protected void filterFileMetadata(OptionManager optionManager, FilterPredicate<?> filterPredicate, Set<SchemaPath> schemaPathsInExpr)
      Produces filtering of metadata at file level.
      Parameters:
      optionManager - option manager
      filterPredicate - filter expression
      schemaPathsInExpr - columns used in filter expression
    • pruneForSegments

      protected static <T extends BaseMetadata & LocationProvider> Map<org.apache.hadoop.fs.Path,T> pruneForSegments(Map<org.apache.hadoop.fs.Path,T> metadataToPrune, Map<org.apache.hadoop.fs.Path,SegmentMetadata> filteredSegmentMetadata)
      Removes metadata which does not belong to any of partitions in metadata list.
      Type Parameters:
      T - type of metadata to filter
      Parameters:
      metadataToPrune - list of metadata which should be pruned
      filteredSegmentMetadata - list of segment metadata which was pruned
      Returns:
      map with metadata which belongs to pruned partitions
    • pruneForSegments

      protected List<PartitionMetadata> pruneForSegments(List<PartitionMetadata> metadataToPrune, Map<org.apache.hadoop.fs.Path,SegmentMetadata> filteredSegmentMetadata)
      Removes metadata which does not belong to any of partitions in metadata list.
      Parameters:
      metadataToPrune - list of partition metadata which should be pruned
      filteredSegmentMetadata - list of segment metadata which was pruned
      Returns:
      list with metadata which belongs to pruned partitions
    • filterAndGetMetadata

      public <T extends Metadata> List<T> filterAndGetMetadata(Set<SchemaPath> schemaPathsInExpr, Iterable<T> metadataList, FilterPredicate<?> filterPredicate, OptionManager optionManager)
      Produces filtering of specified metadata using specified filter expression and returns filtered metadata.
      Type Parameters:
      T - type of metadata to filter
      Parameters:
      schemaPathsInExpr - columns used in filter expression
      metadataList - metadata to filter
      filterPredicate - filter expression
      optionManager - option manager
      Returns:
      filtered metadata
    • getImplicitColumnStatistics

      protected <T extends Metadata> Map<SchemaPath,ColumnStatistics<?>> getImplicitColumnStatistics(OptionManager optionManager, T metadata, Map<SchemaPath,ColumnStatistics<?>> columnsStatistics)
    • self

      protected abstract B self()