Package org.apache.drill.exec.physical.base

Class AbstractGroupScan

java.lang.Object

org.apache.drill.exec.physical.base.AbstractBase

org.apache.drill.exec.physical.base.AbstractGroupScan

All Implemented Interfaces:

Iterable<PhysicalOperator>, GraphValue<PhysicalOperator>, FragmentLeaf, GroupScan, HasAffinity, Leaf, PhysicalOperator, Scan

Direct Known Subclasses:

AbstractDbGroupScan, AbstractFileGroupScan, DirectGroupScan, DrillGroupScan, DruidGroupScan, EnumerableGroupScan, GoogleSheetsGroupScan, HBaseGroupScan, HiveScan, HttpGroupScan, IcebergGroupScan, InfoSchemaGroupScan, JdbcGroupScan, KafkaGroupScan, KuduGroupScan, MockGroupScanPOP, MongoGroupScan, OpenTSDBGroupScan, PhoenixGroupScan, SplunkGroupScan, SystemTableScan

public abstract class AbstractGroupScan
extends AbstractBase
implements GroupScan

Field Summary

Fields inherited from class org.apache.drill.exec.physical.base.AbstractBase

INIT_ALLOCATION, initialAllocation, MAX_ALLOCATION, maxAllocation, userName

Fields inherited from interface org.apache.drill.exec.physical.base.GroupScan

ALL_COLUMNS

Constructor Summary

Constructors

Constructor	Description
AbstractGroupScan(String userName)
AbstractGroupScan(AbstractGroupScan that)

Method Summary

Modifier and Type	Method	Description
<T, X, E extends Throwable> T	accept(PhysicalVisitor<T,X,E> physicalVisitor, X value)	Provides capability to build a set of output based on traversing a query graph tree.
GroupScan	applyFilter(LogicalExpression filterExpr, UdfUtilities udfUtilities, FunctionImplementationRegistry functionImplementationRegistry, OptionManager optionManager)
GroupScan	applyLimit(int maxRecords)	By default, return null to indicate row count based prune is not supported.
boolean	canPushdownProjects(List<SchemaPath> columns)	GroupScan should check the list of columns, and see if it could support all the columns in the list.
GroupScan	clone(List<SchemaPath> columns)	Returns a clone of GroupScan instance, except that the new GroupScan will use the provided list of columns .
boolean	enforceWidth()	Deprecated.
AnalyzeInfoProvider	getAnalyzeInfoProvider()	Returns AnalyzeInfoProvider instance which will be used when running ANALYZE statement.
List<SchemaPath>	getColumns()	Returns a list of columns scanned by this group scan
long	getColumnValueCount(SchemaPath column)	By default, throw exception, since group scan does not have exact column value count.
DistributionAffinity	getDistributionAffinity()	Get distribution affinity which describes the parallelization strategy of the operator.
Collection<org.apache.hadoop.fs.Path>	getFiles()	Returns a collection of file names associated with this GroupScan.
LogicalExpression	getFilter()
long	getInitialAllocation()
long	getMaxAllocation()
TableMetadataProvider	getMetadataProvider()	Returns TableMetadataProvider instance which is used for providing metadata for current GroupScan.
int	getMinParallelizationWidth()	At minimum, the GroupScan requires these many fragments to run.
List<EndpointAffinity>	getOperatorAffinity()	Get the list of Endpoints with associated affinities that this operator has preference for.
String	getOperatorType()
List<SchemaPath>	getPartitionColumns()	Returns a list of columns that can be used for partition pruning
ScanStats	getScanStats()
ScanStats	getScanStats(org.apache.calcite.rel.metadata.RelMetadataQuery mq)
ScanStats	getScanStats(PlannerSettings settings)
org.apache.hadoop.fs.Path	getSelectionRoot()	Returns path to the selection root.
TableMetadata	getTableMetadata()
boolean	hasFiles()	Return true if this GroupScan can return its selection as a list of file names (retrieved by getFiles()).
boolean	isDistributed()
boolean	isExecutable()	Describes whether or not a particular physical operator can actually be executed.
Iterator<PhysicalOperator>	iterator()
boolean	supportsFilterPushDown()	Checks whether this group scan supports filter push down.
boolean	supportsLimitPushdown()	Default is not to support limit pushdown.
boolean	supportsPartitionFilterPushdown()	Whether or not this GroupScan supports pushdown of partition filters (directories for filesystems)
boolean	usedMetastore()	Returns true if current group scan uses metadata obtained from the Metastore.

Methods inherited from class org.apache.drill.exec.physical.base.AbstractBase

accept, getCost, getOperatorId, getSVMode, getUserName, isBufferedOperator, setCost, setMaxAllocation, setOperatorId

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.drill.common.graph.GraphValue

accept

Methods inherited from interface org.apache.drill.exec.physical.base.GroupScan

applyAssignments, getDigest, getMaxParallelizationWidth, getSpecificScan

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Methods inherited from interface org.apache.drill.exec.physical.base.PhysicalOperator

getCost, getNewWithChildren, getOperatorId, getSVMode, getUserName, isBufferedOperator, setCost, setMaxAllocation, setOperatorId

Constructor Details

AbstractGroupScan

public AbstractGroupScan(String userName)

AbstractGroupScan

public AbstractGroupScan(AbstractGroupScan that)

Method Details

iterator

public Iterator<PhysicalOperator> iterator()

Specified by:

iterator in interface Iterable<PhysicalOperator>

getOperatorAffinity

public List<EndpointAffinity> getOperatorAffinity()

Description copied from interface: HasAffinity

Get the list of Endpoints with associated affinities that this operator has preference for.

Specified by:

getOperatorAffinity in interface HasAffinity

Returns:

List of EndpointAffinity objects.

isExecutable

public boolean isExecutable()

Description copied from interface: PhysicalOperator

Describes whether or not a particular physical operator can actually be executed. Most physical operators can be executed. However, Exchange nodes cannot be executed. In order to be executed, they must be converted into their Exec sub components.

Specified by:

isExecutable in interface PhysicalOperator

Overrides:

isExecutable in class AbstractBase

accept

public <T,
X,
E extends Throwable> T accept(PhysicalVisitor<T,X,E> physicalVisitor,
 X value)
                                   throws E

Description copied from interface: PhysicalOperator

Provides capability to build a set of output based on traversing a query graph tree.

Specified by:

accept in interface PhysicalOperator

Throws:

E extends Throwable

clone

public GroupScan clone(List<SchemaPath> columns)

Description copied from interface: GroupScan

Returns a clone of GroupScan instance, except that the new GroupScan will use the provided list of columns .

Specified by:

clone in interface GroupScan

isDistributed

public boolean isDistributed()

Specified by:

isDistributed in interface GroupScan

getMinParallelizationWidth

public int getMinParallelizationWidth()

Description copied from interface: GroupScan

At minimum, the GroupScan requires these many fragments to run. Currently, this is used in SimpleParallelizer

Specified by:

getMinParallelizationWidth in interface GroupScan

Returns:

the minimum number of fragments that should run

getScanStats

public ScanStats getScanStats(PlannerSettings settings)

Specified by:

getScanStats in interface GroupScan

getScanStats

public ScanStats getScanStats(org.apache.calcite.rel.metadata.RelMetadataQuery mq)

Specified by:

getScanStats in interface GroupScan

getScanStats

public ScanStats getScanStats()

enforceWidth

@Deprecated
public boolean enforceWidth()

Deprecated.

Description copied from interface: GroupScan

Check if GroupScan enforces width to be maximum parallelization width. Currently, this is used in ExcessiveExchangeIdentifier

Specified by:

enforceWidth in interface GroupScan

Returns:

if maximum width should be enforced

getInitialAllocation

public long getInitialAllocation()

Specified by:

getInitialAllocation in interface PhysicalOperator

Overrides:

getInitialAllocation in class AbstractBase

Returns:

The memory to preallocate for this operator

getMaxAllocation

public long getMaxAllocation()

Specified by:

getMaxAllocation in interface PhysicalOperator

Overrides:

getMaxAllocation in class AbstractBase

Returns:

The maximum memory this operator can allocate

canPushdownProjects

public boolean canPushdownProjects(List<SchemaPath> columns)

Description copied from interface: GroupScan

GroupScan should check the list of columns, and see if it could support all the columns in the list.

Specified by:

canPushdownProjects in interface GroupScan

supportsPartitionFilterPushdown

public boolean supportsPartitionFilterPushdown()

Description copied from interface: GroupScan

Whether or not this GroupScan supports pushdown of partition filters (directories for filesystems)

Specified by:

supportsPartitionFilterPushdown in interface GroupScan

getColumnValueCount

public long getColumnValueCount(SchemaPath column)

By default, throw exception, since group scan does not have exact column value count.

Specified by:

getColumnValueCount in interface GroupScan

getOperatorType

public String getOperatorType()

Specified by:

getOperatorType in interface PhysicalOperator

getColumns

public List<SchemaPath> getColumns()

Description copied from interface: GroupScan

Returns a list of columns scanned by this group scan

Specified by:

getColumns in interface GroupScan

getPartitionColumns

public List<SchemaPath> getPartitionColumns()

Description copied from interface: GroupScan

Returns a list of columns that can be used for partition pruning

Specified by:

getPartitionColumns in interface GroupScan

supportsLimitPushdown

public boolean supportsLimitPushdown()

Default is not to support limit pushdown.

Specified by:

supportsLimitPushdown in interface GroupScan

applyLimit

public GroupScan applyLimit(int maxRecords)

By default, return null to indicate row count based prune is not supported. Each group scan subclass should override, if it supports row count based prune.

Specified by:

applyLimit in interface GroupScan

Parameters:

maxRecords - : the number of rows requested from group scan.

Returns:

a new instance of group scan if the prune is successful. null when either if row-based prune is not supported, or if prune is not successful.

hasFiles

public boolean hasFiles()

Description copied from interface: GroupScan

Return true if this GroupScan can return its selection as a list of file names (retrieved by getFiles()).

Specified by:

hasFiles in interface GroupScan

getSelectionRoot

public org.apache.hadoop.fs.Path getSelectionRoot()

Description copied from interface: GroupScan

Returns path to the selection root. If this GroupScan cannot provide selection root, it returns null.

Specified by:

getSelectionRoot in interface GroupScan

Returns:

path to the selection root

getFiles

public Collection<org.apache.hadoop.fs.Path> getFiles()

Description copied from interface: GroupScan

Returns a collection of file names associated with this GroupScan. This should be called after checking hasFiles(). If this GroupScan cannot provide file names, it returns null.

Specified by:

getFiles in interface GroupScan

Returns:

collection of files paths

getDistributionAffinity

public DistributionAffinity getDistributionAffinity()

Description copied from interface: HasAffinity

Get distribution affinity which describes the parallelization strategy of the operator.

Specified by:

getDistributionAffinity in interface HasAffinity

getFilter

public LogicalExpression getFilter()

Specified by:

getFilter in interface GroupScan

applyFilter

public GroupScan applyFilter(LogicalExpression filterExpr,
 UdfUtilities udfUtilities,
 FunctionImplementationRegistry functionImplementationRegistry,
 OptionManager optionManager)

Specified by:

applyFilter in interface GroupScan

getMetadataProvider

public TableMetadataProvider getMetadataProvider()

Description copied from interface: GroupScan

Returns TableMetadataProvider instance which is used for providing metadata for current GroupScan.

Specified by:

getMetadataProvider in interface GroupScan

Returns:

TableMetadataProvider instance the source of metadata

getTableMetadata

public TableMetadata getTableMetadata()

Specified by:

getTableMetadata in interface GroupScan

usedMetastore

public boolean usedMetastore()

Description copied from interface: GroupScan

Returns true if current group scan uses metadata obtained from the Metastore.

Specified by:

usedMetastore in interface GroupScan

Returns:

true if current group scan uses metadata obtained from the Metastore, false otherwise.

getAnalyzeInfoProvider

public AnalyzeInfoProvider getAnalyzeInfoProvider()

Description copied from interface: GroupScan

Returns AnalyzeInfoProvider instance which will be used when running ANALYZE statement.

Specified by:

getAnalyzeInfoProvider in interface GroupScan

Returns:

AnalyzeInfoProvider instance

supportsFilterPushDown

public boolean supportsFilterPushDown()

Description copied from interface: GroupScan

Checks whether this group scan supports filter push down.

Specified by:

supportsFilterPushDown in interface GroupScan

Returns:

true if this group scan supports filter push down, false otherwise