org.apache.drill.exec.physical.base

Class AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B extends AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>>

java.lang.Object

org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>

Direct Known Subclasses:

AbstractParquetGroupScan.RowGroupScanFilterer

Enclosing class:

AbstractGroupScanWithMetadata<P extends TableMetadataProvider>

protected abstract static class AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B extends AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>>
extends Object

This class is responsible for filtering different metadata levels.

Field Summary

Fields

Modifier and Type	Field and Description
protected FunctionLookupContext	context
protected Map<org.apache.hadoop.fs.Path,FileMetadata>	files
protected LogicalExpression	filterExpression
protected int	limit
protected boolean	matchAllMetadata
protected NonInterestingColumnsMetadata	nonInterestingColumnsMetadata
protected MetadataType	overflowLevel
protected List<PartitionMetadata>	partitions
protected Map<org.apache.hadoop.fs.Path,SegmentMetadata>	segments
protected AbstractGroupScanWithMetadata<? extends TableMetadataProvider>	source
protected TableMetadata	tableMetadata
protected TupleMetadata	tableSchema
protected UdfUtilities	udfUtilities

Constructor Summary

Constructors

Constructor and Description
GroupScanWithMetadataFilterer(AbstractGroupScanWithMetadata<?> source)

Method Summary

Modifier and Type	Method and Description
abstract AbstractGroupScanWithMetadata<?>	build() Constructs required implementation of AbstractGroupScanWithMetadata with filtered metadata.
B	context(FunctionLookupContext context)
B	files(Map<org.apache.hadoop.fs.Path,FileMetadata> files)
<T extends Metadata> List<T>	filterAndGetMetadata(Set<SchemaPath> schemaPathsInExpr, Iterable<T> metadataList, FilterPredicate<?> filterPredicate, OptionManager optionManager) Produces filtering of specified metadata using specified filter expression and returns filtered metadata.
B	filterExpression(LogicalExpression filterExpression)
protected void	filterFileMetadata(OptionManager optionManager, FilterPredicate<?> filterPredicate, Set<SchemaPath> schemaPathsInExpr) Produces filtering of metadata at file level.
protected void	filterPartitionMetadata(OptionManager optionManager, FilterPredicate<?> filterPredicate, Set<SchemaPath> schemaPathsInExpr) Produces filtering of metadata at partition level.
protected void	filterSegmentMetadata(OptionManager optionManager, FilterPredicate<?> filterPredicate, Set<SchemaPath> schemaPathsInExpr) Produces filtering of metadata at segment level.
protected void	filterTableMetadata(FilterPredicate<?> filterPredicate, Set<SchemaPath> schemaPathsInExpr) Produces filtering of metadata at table level.
Map<org.apache.hadoop.fs.Path,FileMetadata>	getFiles()
protected B	getFiltered(OptionManager optionManager, FilterPredicate<?> filterPredicate) Produces filtering of metadata and returns AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer to construct resulting group scan.
MetadataType	getOverflowLevel()
List<PartitionMetadata>	getPartitions()
Map<org.apache.hadoop.fs.Path,SegmentMetadata>	getSegments()
TableMetadata	getTableMetadata()
boolean	isMatchAllMetadata()
B	limit(int maxRecords)
B	matching(boolean matchAllMetadata)
B	nonInterestingColumns(NonInterestingColumnsMetadata nonInterestingColumns)
B	overflow(MetadataType overflowLevel)
B	partitions(List<PartitionMetadata> partitions)
protected List<PartitionMetadata>	pruneForSegments(List<PartitionMetadata> metadataToPrune, Map<org.apache.hadoop.fs.Path,SegmentMetadata> filteredSegmentMetadata) Removes metadata which does not belong to any of partitions in metadata list.
protected static <T extends BaseMetadata & LocationProvider> Map<org.apache.hadoop.fs.Path,T>	pruneForSegments(Map<org.apache.hadoop.fs.Path,T> metadataToPrune, Map<org.apache.hadoop.fs.Path,SegmentMetadata> filteredSegmentMetadata) Removes metadata which does not belong to any of partitions in metadata list.
B	schema(TupleMetadata tableSchema)
B	segments(Map<org.apache.hadoop.fs.Path,SegmentMetadata> segments)
protected abstract B	self()
B	table(TableMetadata tableMetadata)
B	udfUtilities(UdfUtilities udfUtilities)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

source

protected final AbstractGroupScanWithMetadata<? extends TableMetadataProvider> source

matchAllMetadata

protected boolean matchAllMetadata

tableMetadata

protected TableMetadata tableMetadata

partitions

protected List<PartitionMetadata> partitions

segments

protected Map<org.apache.hadoop.fs.Path,SegmentMetadata> segments

files

protected Map<org.apache.hadoop.fs.Path,FileMetadata> files

nonInterestingColumnsMetadata

protected NonInterestingColumnsMetadata nonInterestingColumnsMetadata

filterExpression

protected LogicalExpression filterExpression

tableSchema

protected TupleMetadata tableSchema

udfUtilities

protected UdfUtilities udfUtilities

context

protected FunctionLookupContext context

limit

protected int limit

overflowLevel

protected MetadataType overflowLevel

Constructor Detail

GroupScanWithMetadataFilterer

public GroupScanWithMetadataFilterer(AbstractGroupScanWithMetadata<?> source)

Method Detail

build

public abstract AbstractGroupScanWithMetadata<?> build()

Constructs required implementation of AbstractGroupScanWithMetadata with filtered metadata.

Returns:

implementation of AbstractGroupScanWithMetadata with filtered metadata

table

public B table(TableMetadata tableMetadata)

partitions

public B partitions(List<PartitionMetadata> partitions)

segments

public B segments(Map<org.apache.hadoop.fs.Path,SegmentMetadata> segments)

nonInterestingColumns

public B nonInterestingColumns(NonInterestingColumnsMetadata nonInterestingColumns)

files

public B files(Map<org.apache.hadoop.fs.Path,FileMetadata> files)

limit

public B limit(int maxRecords)

matching

public B matching(boolean matchAllMetadata)

overflow

public B overflow(MetadataType overflowLevel)

filterExpression

public B filterExpression(LogicalExpression filterExpression)

schema

public B schema(TupleMetadata tableSchema)

udfUtilities

public B udfUtilities(UdfUtilities udfUtilities)

context

public B context(FunctionLookupContext context)

isMatchAllMetadata

public boolean isMatchAllMetadata()

getTableMetadata

public TableMetadata getTableMetadata()

getPartitions

public List<PartitionMetadata> getPartitions()

getSegments

public Map<org.apache.hadoop.fs.Path,SegmentMetadata> getSegments()

getFiles

public Map<org.apache.hadoop.fs.Path,FileMetadata> getFiles()

getOverflowLevel

public MetadataType getOverflowLevel()

getFiltered

protected B getFiltered(OptionManager optionManager,
                        FilterPredicate<?> filterPredicate)

Produces filtering of metadata and returns AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer to construct resulting group scan.

Parameters:

optionManager - option manager

filterPredicate - filter expression

Returns:

this instance with filtered metadata

filterTableMetadata

protected void filterTableMetadata(FilterPredicate<?> filterPredicate,
                                   Set<SchemaPath> schemaPathsInExpr)

Produces filtering of metadata at table level.

Parameters:

filterPredicate - filter expression

schemaPathsInExpr - columns used in filter expression

filterSegmentMetadata

protected void filterSegmentMetadata(OptionManager optionManager,
                                     FilterPredicate<?> filterPredicate,
                                     Set<SchemaPath> schemaPathsInExpr)

Produces filtering of metadata at segment level.

Parameters:

optionManager - option manager

filterPredicate - filter expression

schemaPathsInExpr - columns used in filter expression

filterPartitionMetadata

protected void filterPartitionMetadata(OptionManager optionManager,
                                       FilterPredicate<?> filterPredicate,
                                       Set<SchemaPath> schemaPathsInExpr)

Produces filtering of metadata at partition level.

Parameters:

optionManager - option manager

filterPredicate - filter expression

schemaPathsInExpr - columns used in filter expression

filterFileMetadata

protected void filterFileMetadata(OptionManager optionManager,
                                  FilterPredicate<?> filterPredicate,
                                  Set<SchemaPath> schemaPathsInExpr)

Produces filtering of metadata at file level.

Parameters:

optionManager - option manager

filterPredicate - filter expression

schemaPathsInExpr - columns used in filter expression

pruneForSegments

protected static <T extends BaseMetadata & LocationProvider> Map<org.apache.hadoop.fs.Path,T> pruneForSegments(Map<org.apache.hadoop.fs.Path,T> metadataToPrune,
                                                                                                               Map<org.apache.hadoop.fs.Path,SegmentMetadata> filteredSegmentMetadata)

Removes metadata which does not belong to any of partitions in metadata list.

Type Parameters:

T - type of metadata to filter

Parameters:

metadataToPrune - list of metadata which should be pruned

filteredSegmentMetadata - list of segment metadata which was pruned

Returns:

map with metadata which belongs to pruned partitions

pruneForSegments

protected List<PartitionMetadata> pruneForSegments(List<PartitionMetadata> metadataToPrune,
                                                   Map<org.apache.hadoop.fs.Path,SegmentMetadata> filteredSegmentMetadata)

Removes metadata which does not belong to any of partitions in metadata list.

Parameters:

metadataToPrune - list of partition metadata which should be pruned

filteredSegmentMetadata - list of segment metadata which was pruned

Returns:

list with metadata which belongs to pruned partitions

filterAndGetMetadata

public <T extends Metadata> List<T> filterAndGetMetadata(Set<SchemaPath> schemaPathsInExpr,
                                                         Iterable<T> metadataList,
                                                         FilterPredicate<?> filterPredicate,
                                                         OptionManager optionManager)

Produces filtering of specified metadata using specified filter expression and returns filtered metadata.

Type Parameters:

T - type of metadata to filter

Parameters:

schemaPathsInExpr - columns used in filter expression

metadataList - metadata to filter

filterPredicate - filter expression

optionManager - option manager

Returns:

filtered metadata

self

protected abstract B self()