Package org.apache.drill.exec.store.parquet

Class AbstractParquetGroupScan.RowGroupScanFilterer<B extends AbstractParquetGroupScan.RowGroupScanFilterer<B>>

java.lang.Object

org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>

org.apache.drill.exec.store.parquet.AbstractParquetGroupScan.RowGroupScanFilterer<B>

Enclosing class:

AbstractParquetGroupScan

protected abstract static class AbstractParquetGroupScan.RowGroupScanFilterer<B extends AbstractParquetGroupScan.RowGroupScanFilterer<B>>
extends AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>

This class is responsible for filtering different metadata levels including row group level.

Field Summary

Fields

Modifier and Type	Field	Description
protected com.google.common.collect.Multimap<org.apache.hadoop.fs.Path,RowGroupMetadata>	rowGroups

Fields inherited from class org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer

context, files, filterExpression, limit, matchAllMetadata, nonInterestingColumnsMetadata, overflowLevel, partitions, segments, source, tableMetadata, tableSchema, udfUtilities

Constructor Summary

Constructors

Constructor	Description
RowGroupScanFilterer(AbstractGroupScanWithMetadata<?> source)

Method Summary

Modifier and Type	Method	Description
AbstractParquetGroupScan	build()	Constructs required implementation of AbstractGroupScanWithMetadata with filtered metadata.
protected void	filterFileMetadata(OptionManager optionManager, FilterPredicate<?> filterPredicate, Set<SchemaPath> schemaPathsInExpr)	Produces filtering of metadata at file level.
protected void	filterRowGroupMetadata(OptionManager optionManager, FilterPredicate<?> filterPredicate)	Produces filtering of metadata at row group level.
protected B	getFiltered(OptionManager optionManager, FilterPredicate<?> filterPredicate)	Produces filtering of metadata and returns AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer to construct resulting group scan.
protected abstract AbstractParquetGroupScan	getNewScan()	Returns new AbstractParquetGroupScan instance to be populated with filtered metadata from this AbstractParquetGroupScan.RowGroupScanFilterer instance.
com.google.common.collect.Multimap<org.apache.hadoop.fs.Path,RowGroupMetadata>	getRowGroups()
protected static <T extends BaseMetadata & LocationProvider> com.google.common.collect.Multimap<org.apache.hadoop.fs.Path,T>	pruneForSegments(com.google.common.collect.Multimap<org.apache.hadoop.fs.Path,T> metadataToPrune, Map<org.apache.hadoop.fs.Path,SegmentMetadata> filteredSegmentMetadata)	Removes metadata which does not belong to any of segments in metadata list.
B	rowGroups(com.google.common.collect.Multimap<org.apache.hadoop.fs.Path,RowGroupMetadata> rowGroups)

Methods inherited from class org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer

context, files, filterAndGetMetadata, filterExpression, filterPartitionMetadata, filterSegmentMetadata, filterTableMetadata, getFiles, getImplicitColumnStatistics, getOverflowLevel, getPartitions, getSegments, getTableMetadata, isMatchAllMetadata, limit, matching, nonInterestingColumns, overflow, partitions, pruneForSegments, pruneForSegments, schema, segments, self, table, udfUtilities

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

rowGroups

protected com.google.common.collect.Multimap<org.apache.hadoop.fs.Path,RowGroupMetadata> rowGroups

Constructor Details

RowGroupScanFilterer

public RowGroupScanFilterer(AbstractGroupScanWithMetadata<?> source)

Method Details

rowGroups

public B rowGroups(com.google.common.collect.Multimap<org.apache.hadoop.fs.Path,RowGroupMetadata> rowGroups)

getNewScan

protected abstract AbstractParquetGroupScan getNewScan()

Returns new AbstractParquetGroupScan instance to be populated with filtered metadata from this AbstractParquetGroupScan.RowGroupScanFilterer instance.

Returns:

new AbstractParquetGroupScan instance

getRowGroups

public com.google.common.collect.Multimap<org.apache.hadoop.fs.Path,RowGroupMetadata> getRowGroups()

build

public AbstractParquetGroupScan build()

Description copied from class: AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer

Constructs required implementation of AbstractGroupScanWithMetadata with filtered metadata.

Specified by:

build in class AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B extends AbstractParquetGroupScan.RowGroupScanFilterer<B>>

Returns:

implementation of AbstractGroupScanWithMetadata with filtered metadata

getFiltered

protected B getFiltered(OptionManager optionManager,
 FilterPredicate<?> filterPredicate)

Description copied from class: AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer

Produces filtering of metadata and returns AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer to construct resulting group scan.

Overrides:

getFiltered in class AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B extends AbstractParquetGroupScan.RowGroupScanFilterer<B>>

Parameters:

optionManager - option manager

filterPredicate - filter expression

Returns:

this instance with filtered metadata

filterRowGroupMetadata

protected void filterRowGroupMetadata(OptionManager optionManager,
 FilterPredicate<?> filterPredicate)

Produces filtering of metadata at row group level.

Parameters:

optionManager - option manager

filterPredicate - filter expression

filterFileMetadata

protected void filterFileMetadata(OptionManager optionManager,
 FilterPredicate<?> filterPredicate,
 Set<SchemaPath> schemaPathsInExpr)

Produces filtering of metadata at file level.

Overrides:

filterFileMetadata in class AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B extends AbstractParquetGroupScan.RowGroupScanFilterer<B>>

Parameters:

optionManager - option manager

filterPredicate - filter expression

schemaPathsInExpr - columns used in filter expression

pruneForSegments

protected static <T extends BaseMetadata & LocationProvider>
com.google.common.collect.Multimap<org.apache.hadoop.fs.Path,T> pruneForSegments(com.google.common.collect.Multimap<org.apache.hadoop.fs.Path,T> metadataToPrune,
 Map<org.apache.hadoop.fs.Path,SegmentMetadata> filteredSegmentMetadata)

Removes metadata which does not belong to any of segments in metadata list.

Type Parameters:

T - type of metadata to filter

Parameters:

metadataToPrune - list of metadata which should be pruned

filteredSegmentMetadata - list of segment metadata which was pruned

Returns:

multimap with metadata which belongs to pruned segments