Class AbstractParquetGroupScan.RowGroupScanFilterer<B extends AbstractParquetGroupScan.RowGroupScanFilterer<B>>
java.lang.Object
org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>
org.apache.drill.exec.store.parquet.AbstractParquetGroupScan.RowGroupScanFilterer<B>
- Enclosing class:
- AbstractParquetGroupScan
protected abstract static class AbstractParquetGroupScan.RowGroupScanFilterer<B extends AbstractParquetGroupScan.RowGroupScanFilterer<B>>
extends AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>
This class is responsible for filtering different metadata levels including row group level.
-
Field Summary
Modifier and TypeFieldDescriptionprotected org.apache.drill.shaded.guava.com.google.common.collect.Multimap<org.apache.hadoop.fs.Path,
RowGroupMetadata> Fields inherited from class org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer
context, files, filterExpression, limit, matchAllMetadata, nonInterestingColumnsMetadata, overflowLevel, partitions, segments, source, tableMetadata, tableSchema, udfUtilities
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionbuild()
Constructs required implementation ofAbstractGroupScanWithMetadata
with filtered metadata.protected void
filterFileMetadata
(OptionManager optionManager, FilterPredicate<?> filterPredicate, Set<SchemaPath> schemaPathsInExpr) Produces filtering of metadata at file level.protected void
filterRowGroupMetadata
(OptionManager optionManager, FilterPredicate<?> filterPredicate) Produces filtering of metadata at row group level.protected B
getFiltered
(OptionManager optionManager, FilterPredicate<?> filterPredicate) Produces filtering of metadata and returnsAbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer
to construct resulting group scan.protected abstract AbstractParquetGroupScan
Returns newAbstractParquetGroupScan
instance to be populated with filtered metadata from thisAbstractParquetGroupScan.RowGroupScanFilterer
instance.org.apache.drill.shaded.guava.com.google.common.collect.Multimap<org.apache.hadoop.fs.Path,
RowGroupMetadata> protected static <T extends BaseMetadata & LocationProvider>
org.apache.drill.shaded.guava.com.google.common.collect.Multimap<org.apache.hadoop.fs.Path,T> pruneForSegments
(org.apache.drill.shaded.guava.com.google.common.collect.Multimap<org.apache.hadoop.fs.Path, T> metadataToPrune, Map<org.apache.hadoop.fs.Path, SegmentMetadata> filteredSegmentMetadata) Removes metadata which does not belong to any of segments in metadata list.rowGroups
(org.apache.drill.shaded.guava.com.google.common.collect.Multimap<org.apache.hadoop.fs.Path, RowGroupMetadata> rowGroups) Methods inherited from class org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer
context, files, filterAndGetMetadata, filterExpression, filterPartitionMetadata, filterSegmentMetadata, filterTableMetadata, getFiles, getImplicitColumnStatistics, getOverflowLevel, getPartitions, getSegments, getTableMetadata, isMatchAllMetadata, limit, matching, nonInterestingColumns, overflow, partitions, pruneForSegments, pruneForSegments, schema, segments, self, table, udfUtilities
-
Field Details
-
rowGroups
protected org.apache.drill.shaded.guava.com.google.common.collect.Multimap<org.apache.hadoop.fs.Path,RowGroupMetadata> rowGroups
-
-
Constructor Details
-
RowGroupScanFilterer
-
-
Method Details
-
rowGroups
public B rowGroups(org.apache.drill.shaded.guava.com.google.common.collect.Multimap<org.apache.hadoop.fs.Path, RowGroupMetadata> rowGroups) -
getNewScan
Returns newAbstractParquetGroupScan
instance to be populated with filtered metadata from thisAbstractParquetGroupScan.RowGroupScanFilterer
instance.- Returns:
- new
AbstractParquetGroupScan
instance
-
getRowGroups
public org.apache.drill.shaded.guava.com.google.common.collect.Multimap<org.apache.hadoop.fs.Path,RowGroupMetadata> getRowGroups() -
build
Description copied from class:AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer
Constructs required implementation ofAbstractGroupScanWithMetadata
with filtered metadata.- Specified by:
build
in classAbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B extends AbstractParquetGroupScan.RowGroupScanFilterer<B>>
- Returns:
- implementation of
AbstractGroupScanWithMetadata
with filtered metadata
-
getFiltered
Description copied from class:AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer
Produces filtering of metadata and returnsAbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer
to construct resulting group scan.- Overrides:
getFiltered
in classAbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B extends AbstractParquetGroupScan.RowGroupScanFilterer<B>>
- Parameters:
optionManager
- option managerfilterPredicate
- filter expression- Returns:
- this instance with filtered metadata
-
filterRowGroupMetadata
protected void filterRowGroupMetadata(OptionManager optionManager, FilterPredicate<?> filterPredicate) Produces filtering of metadata at row group level.- Parameters:
optionManager
- option managerfilterPredicate
- filter expression
-
filterFileMetadata
protected void filterFileMetadata(OptionManager optionManager, FilterPredicate<?> filterPredicate, Set<SchemaPath> schemaPathsInExpr) Produces filtering of metadata at file level.- Overrides:
filterFileMetadata
in classAbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B extends AbstractParquetGroupScan.RowGroupScanFilterer<B>>
- Parameters:
optionManager
- option managerfilterPredicate
- filter expressionschemaPathsInExpr
- columns used in filter expression
-
pruneForSegments
protected static <T extends BaseMetadata & LocationProvider> org.apache.drill.shaded.guava.com.google.common.collect.Multimap<org.apache.hadoop.fs.Path,T> pruneForSegments(org.apache.drill.shaded.guava.com.google.common.collect.Multimap<org.apache.hadoop.fs.Path, T> metadataToPrune, Map<org.apache.hadoop.fs.Path, SegmentMetadata> filteredSegmentMetadata) Removes metadata which does not belong to any of segments in metadata list.- Type Parameters:
T
- type of metadata to filter- Parameters:
metadataToPrune
- list of metadata which should be prunedfilteredSegmentMetadata
- list of segment metadata which was pruned- Returns:
- multimap with metadata which belongs to pruned segments
-