Package org.apache.drill.exec.store.kafka

Class KafkaGroupScan

java.lang.Object

org.apache.drill.exec.physical.base.AbstractBase

org.apache.drill.exec.physical.base.AbstractGroupScan

org.apache.drill.exec.store.kafka.KafkaGroupScan

All Implemented Interfaces:

Iterable<PhysicalOperator>, GraphValue<PhysicalOperator>, FragmentLeaf, GroupScan, HasAffinity, Leaf, PhysicalOperator, Scan

public class KafkaGroupScan
extends AbstractGroupScan

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	KafkaGroupScan.PartitionScanWork

Field Summary

Fields inherited from class org.apache.drill.exec.physical.base.AbstractBase

INIT_ALLOCATION, initialAllocation, MAX_ALLOCATION, maxAllocation, userName

Fields inherited from interface org.apache.drill.exec.physical.base.GroupScan

ALL_COLUMNS

Constructor Summary

Constructors

Constructor	Description
KafkaGroupScan(String userName, KafkaStoragePluginConfig kafkaStoragePluginConfig, List<SchemaPath> columns, int records, KafkaScanSpec scanSpec, StoragePluginRegistry pluginRegistry)
KafkaGroupScan(String userName, KafkaStoragePlugin kafkaStoragePlugin, List<SchemaPath> columns, int records, KafkaScanSpec kafkaScanSpec)
KafkaGroupScan(KafkaGroupScan that)
KafkaGroupScan(KafkaGroupScan that, int records)
KafkaGroupScan(KafkaGroupScan that, List<SchemaPath> columns)
KafkaGroupScan(KafkaStoragePlugin kafkaStoragePlugin, KafkaScanSpec kafkaScanSpec, List<SchemaPath> columns, int records)

Method Summary

Modifier and Type	Method	Description
void	applyAssignments(List<CoordinationProtos.DrillbitEndpoint> incomingEndpoints)
GroupScan	applyLimit(int maxRecords)	By default, return null to indicate row count based prune is not supported.
boolean	canPushdownProjects(List<SchemaPath> columns)	GroupScan should check the list of columns, and see if it could support all the columns in the list.
GroupScan	clone(List<SchemaPath> columns)	Returns a clone of GroupScan instance, except that the new GroupScan will use the provided list of columns .
GroupScan	cloneWithNewSpec(List<KafkaPartitionScanSpec> partitionScanSpecList)
List<SchemaPath>	getColumns()	Returns a list of columns scanned by this group scan
String	getDigest()	Returns a signature of the GroupScan which should usually be composed of all its attributes which could describe it uniquely.
KafkaScanSpec	getKafkaScanSpec()
KafkaStoragePluginConfig	getKafkaStoragePluginConfig()
int	getMaxParallelizationWidth()
PhysicalOperator	getNewWithChildren(List<PhysicalOperator> children)	Regenerate with this node with a new set of children.
List<EndpointAffinity>	getOperatorAffinity()	Get the list of Endpoints with associated affinities that this operator has preference for.
List<KafkaPartitionScanSpec>	getPartitionScanSpecList()
int	getRecords()
ScanStats	getScanStats()
KafkaSubScan	getSpecificScan(int minorFragmentId)
KafkaStoragePlugin	getStoragePlugin()
boolean	supportsLimitPushdown()	Default is not to support limit pushdown.
String	toString()

Methods inherited from class org.apache.drill.exec.physical.base.AbstractGroupScan

accept, applyFilter, enforceWidth, getAnalyzeInfoProvider, getColumnValueCount, getDistributionAffinity, getFiles, getFilter, getInitialAllocation, getMaxAllocation, getMetadataProvider, getMinParallelizationWidth, getOperatorType, getPartitionColumns, getScanStats, getScanStats, getSelectionRoot, getTableMetadata, hasFiles, isDistributed, isExecutable, iterator, supportsFilterPushDown, supportsPartitionFilterPushdown, usedMetastore

Methods inherited from class org.apache.drill.exec.physical.base.AbstractBase

accept, getCost, getOperatorId, getSVMode, getUserName, isBufferedOperator, setCost, setMaxAllocation, setOperatorId

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.drill.common.graph.GraphValue

accept

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Methods inherited from interface org.apache.drill.exec.physical.base.PhysicalOperator

getCost, getOperatorId, getSVMode, getUserName, isBufferedOperator, setCost, setMaxAllocation, setOperatorId

Constructor Details

KafkaGroupScan

public KafkaGroupScan(String userName,
 KafkaStoragePluginConfig kafkaStoragePluginConfig,
 List<SchemaPath> columns,
 int records,
 KafkaScanSpec scanSpec,
 StoragePluginRegistry pluginRegistry)
               throws ExecutionSetupException

Throws:

ExecutionSetupException

KafkaGroupScan

public KafkaGroupScan(KafkaStoragePlugin kafkaStoragePlugin,
 KafkaScanSpec kafkaScanSpec,
 List<SchemaPath> columns,
 int records)

KafkaGroupScan

public KafkaGroupScan(String userName,
 KafkaStoragePlugin kafkaStoragePlugin,
 List<SchemaPath> columns,
 int records,
 KafkaScanSpec kafkaScanSpec)

KafkaGroupScan

public KafkaGroupScan(KafkaGroupScan that)

KafkaGroupScan

public KafkaGroupScan(KafkaGroupScan that,
 List<SchemaPath> columns)

KafkaGroupScan

public KafkaGroupScan(KafkaGroupScan that,
 int records)

Method Details

applyAssignments

public void applyAssignments(List<CoordinationProtos.DrillbitEndpoint> incomingEndpoints)

applyLimit

public GroupScan applyLimit(int maxRecords)

Description copied from class: AbstractGroupScan

By default, return null to indicate row count based prune is not supported. Each group scan subclass should override, if it supports row count based prune.

Specified by:

applyLimit in interface GroupScan

Overrides:

applyLimit in class AbstractGroupScan

Parameters:

maxRecords - : the number of rows requested from group scan.

Returns:

a new instance of group scan if the prune is successful. null when either if row-based prune is not supported, or if prune is not successful.

supportsLimitPushdown

public boolean supportsLimitPushdown()

Description copied from class: AbstractGroupScan

Default is not to support limit pushdown.

Specified by:

supportsLimitPushdown in interface GroupScan

Overrides:

supportsLimitPushdown in class AbstractGroupScan

getSpecificScan

public KafkaSubScan getSpecificScan(int minorFragmentId)

getMaxParallelizationWidth

public int getMaxParallelizationWidth()

getScanStats

public ScanStats getScanStats()

Overrides:

getScanStats in class AbstractGroupScan

getDigest

public String getDigest()

Description copied from interface: GroupScan

Returns a signature of the GroupScan which should usually be composed of all its attributes which could describe it uniquely.

getNewWithChildren

public PhysicalOperator getNewWithChildren(List<PhysicalOperator> children)

Description copied from interface: PhysicalOperator

Regenerate with this node with a new set of children. This is used in the case of materialization or optimization.

getOperatorAffinity

public List<EndpointAffinity> getOperatorAffinity()

Description copied from interface: HasAffinity

Get the list of Endpoints with associated affinities that this operator has preference for.

Specified by:

getOperatorAffinity in interface HasAffinity

Overrides:

getOperatorAffinity in class AbstractGroupScan

Returns:

List of EndpointAffinity objects.

canPushdownProjects

public boolean canPushdownProjects(List<SchemaPath> columns)

Description copied from interface: GroupScan

GroupScan should check the list of columns, and see if it could support all the columns in the list.

Specified by:

canPushdownProjects in interface GroupScan

Overrides:

canPushdownProjects in class AbstractGroupScan

clone

public GroupScan clone(List<SchemaPath> columns)

Description copied from interface: GroupScan

Returns a clone of GroupScan instance, except that the new GroupScan will use the provided list of columns .

Specified by:

clone in interface GroupScan

Overrides:

clone in class AbstractGroupScan

cloneWithNewSpec

public GroupScan cloneWithNewSpec(List<KafkaPartitionScanSpec> partitionScanSpecList)

getKafkaStoragePluginConfig

public KafkaStoragePluginConfig getKafkaStoragePluginConfig()

getColumns

public List<SchemaPath> getColumns()

Description copied from interface: GroupScan

Returns a list of columns scanned by this group scan

Specified by:

getColumns in interface GroupScan

Overrides:

getColumns in class AbstractGroupScan

getRecords

public int getRecords()

getKafkaScanSpec

public KafkaScanSpec getKafkaScanSpec()

getStoragePlugin

public KafkaStoragePlugin getStoragePlugin()

toString

public String toString()

Overrides:

toString in class Object

getPartitionScanSpecList

public List<KafkaPartitionScanSpec> getPartitionScanSpecList()