Class DeltaGroupScan
java.lang.Object
org.apache.drill.exec.physical.base.AbstractBase
org.apache.drill.exec.physical.base.AbstractGroupScan
org.apache.drill.exec.physical.base.AbstractFileGroupScan
org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata<ParquetMetadataProvider>
org.apache.drill.exec.store.parquet.AbstractParquetGroupScan
org.apache.drill.exec.store.delta.DeltaGroupScan
- All Implemented Interfaces:
Iterable<PhysicalOperator>
,GraphValue<PhysicalOperator>
,FileGroupScan
,FragmentLeaf
,GroupScan
,HasAffinity
,Leaf
,PhysicalOperator
,Scan
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.drill.exec.store.parquet.AbstractParquetGroupScan
AbstractParquetGroupScan.RowGroupScanFilterer<B extends AbstractParquetGroupScan.RowGroupScanFilterer<B>>
Nested classes/interfaces inherited from class org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata
AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B extends AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>>
-
Field Summary
Fields inherited from class org.apache.drill.exec.store.parquet.AbstractParquetGroupScan
entries, mappings, readerConfig, rowGroups
Fields inherited from class org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata
columns, files, fileSet, filter, limit, matchAllMetadata, metadataProvider, nonInterestingColumnsMetadata, partitionColumns, partitions, segments, tableMetadata, usedMetastore
Fields inherited from class org.apache.drill.exec.physical.base.AbstractBase
INIT_ALLOCATION, initialAllocation, MAX_ALLOCATION, maxAllocation, userName
Fields inherited from interface org.apache.drill.exec.physical.base.GroupScan
ALL_COLUMNS
-
Constructor Summary
ConstructorDescriptionDeltaGroupScan
(String userName, List<ReadEntryWithPath> entries, StoragePluginConfig storageConfig, FormatPluginConfig formatConfig, List<SchemaPath> columns, TupleMetadata schema, String path, ParquetReaderConfig readerConfig, LogicalExpression condition, Integer limit, Map<org.apache.hadoop.fs.Path, Map<String, String>> partitionHolder, StoragePluginRegistry pluginRegistry) -
Method Summary
Modifier and TypeMethodDescriptionapplyLimit
(int maxRecords) By default, return null to indicate row count based prune is not supported.builder()
clone
(List<SchemaPath> columns) Returns a clone of GroupScan instance, except that the new GroupScan will use the provided list of columns .clone
(FileSelection selection) protected AbstractParquetGroupScan
cloneWithFileSelection
(Collection<org.apache.hadoop.fs.Path> filePaths) ReturnsTableMetadataProviderBuilder
instance which may provide metadata without using Drill Metastore.protected Collection<CoordinationProtos.DrillbitEndpoint>
protected AbstractParquetGroupScan.RowGroupScanFilterer<?>
Returns holder for metadata values which provides API to filter metadata and build new group scan instance using filtered metadata.getNewWithChildren
(List<PhysicalOperator> children) Regenerate with this node with a new set of children.getPartitionValues
(LocationProvider locationProvider) getPath()
getSpecificScan
(int minorFragmentId) protected boolean
boolean
Default is not to support limit pushdown.ReturnsTableMetadataProviderBuilder
instance based on specifiedMetadataProviderManager
source.toString()
Methods inherited from class org.apache.drill.exec.store.parquet.AbstractParquetGroupScan
applyAssignments, applyFilter, canPushdownProjects, getEntries, getFiles, getMaxParallelizationWidth, getOperatorAffinity, getReadEntries, getReaderConfig, getReaderConfigForSerialization, getRowGroupsMetadata, modifyFileSelection, pruneForPartitions, pruneRowGroupsForFiles, supportsFilterPushDown
Methods inherited from class org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata
checkMetadataConsistency, getColumns, getColumnValueCount, getDigest, getFileSet, getFilesMetadata, getFilter, getFilterPredicate, getFilterPredicate, getFilterString, getLimit, getMetadataProvider, getNextOrEmpty, getNonInterestingColumnsMetadata, getPartitionColumns, getPartitionsMetadata, getPartitionValue, getScanStats, getSegmentsMetadata, getTableMetadata, getTypeForColumn, hasFiles, init, isAllDataPruned, isGroupScanFullyMatchesFilter, isImplicitOrPartCol, isMatchAllMetadata, limitMetadata, pruneForPartitions, setFilter, setFilterForRuntime, usedMetastore
Methods inherited from class org.apache.drill.exec.physical.base.AbstractFileGroupScan
supportsPartitionFilterPushdown
Methods inherited from class org.apache.drill.exec.physical.base.AbstractGroupScan
accept, enforceWidth, getAnalyzeInfoProvider, getDistributionAffinity, getInitialAllocation, getMaxAllocation, getMinParallelizationWidth, getOperatorType, getScanStats, getScanStats, getSelectionRoot, isDistributed, isExecutable, iterator
Methods inherited from class org.apache.drill.exec.physical.base.AbstractBase
accept, getCost, getOperatorId, getSVMode, getUserName, isBufferedOperator, setCost, setMaxAllocation, setOperatorId
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
Methods inherited from interface org.apache.drill.common.graph.GraphValue
accept
Methods inherited from interface org.apache.drill.exec.physical.base.GroupScan
enforceWidth, getAnalyzeInfoProvider, getMinParallelizationWidth, getScanStats, getScanStats, getSelectionRoot, isDistributed
Methods inherited from interface org.apache.drill.exec.physical.base.HasAffinity
getDistributionAffinity
Methods inherited from interface java.lang.Iterable
forEach, iterator, spliterator
Methods inherited from interface org.apache.drill.exec.physical.base.PhysicalOperator
accept, getCost, getInitialAllocation, getMaxAllocation, getOperatorId, getOperatorType, getSVMode, getUserName, isBufferedOperator, isExecutable, setCost, setMaxAllocation, setOperatorId
-
Constructor Details
-
DeltaGroupScan
public DeltaGroupScan(String userName, List<ReadEntryWithPath> entries, StoragePluginConfig storageConfig, FormatPluginConfig formatConfig, List<SchemaPath> columns, TupleMetadata schema, String path, ParquetReaderConfig readerConfig, LogicalExpression condition, Integer limit, Map<org.apache.hadoop.fs.Path, Map<String, throws IOExceptionString>> partitionHolder, StoragePluginRegistry pluginRegistry) - Throws:
IOException
-
-
Method Details
-
tableMetadataProviderBuilder
protected DeltaParquetTableMetadataProvider.Builder tableMetadataProviderBuilder(MetadataProviderManager source) Description copied from class:AbstractGroupScanWithMetadata
ReturnsTableMetadataProviderBuilder
instance based on specifiedMetadataProviderManager
source.- Specified by:
tableMetadataProviderBuilder
in classAbstractGroupScanWithMetadata<ParquetMetadataProvider>
- Parameters:
source
- metadata provider manager- Returns:
TableMetadataProviderBuilder
instance
-
defaultTableMetadataProviderBuilder
protected DeltaParquetTableMetadataProvider.Builder defaultTableMetadataProviderBuilder(MetadataProviderManager source) Description copied from class:AbstractGroupScanWithMetadata
ReturnsTableMetadataProviderBuilder
instance which may provide metadata without using Drill Metastore.- Specified by:
defaultTableMetadataProviderBuilder
in classAbstractParquetGroupScan
- Parameters:
source
- metadata provider manager- Returns:
TableMetadataProviderBuilder
instance
-
builder
-
clone
Description copied from interface:GroupScan
Returns a clone of GroupScan instance, except that the new GroupScan will use the provided list of columns .- Specified by:
clone
in interfaceGroupScan
- Overrides:
clone
in classAbstractGroupScan
-
applyLimit
Description copied from class:AbstractGroupScan
By default, return null to indicate row count based prune is not supported. Each group scan subclass should override, if it supports row count based prune.- Specified by:
applyLimit
in interfaceGroupScan
- Overrides:
applyLimit
in classAbstractParquetGroupScan
- Parameters:
maxRecords
- : the number of rows requested from group scan.- Returns:
- a new instance of group scan if the prune is successful. null when either if row-based prune is not supported, or if prune is not successful.
-
getSpecificScan
-
clone
- Specified by:
clone
in interfaceFileGroupScan
- Overrides:
clone
in classAbstractFileGroupScan
- Throws:
IOException
-
getFilterer
Description copied from class:AbstractGroupScanWithMetadata
Returns holder for metadata values which provides API to filter metadata and build new group scan instance using filtered metadata.- Specified by:
getFilterer
in classAbstractParquetGroupScan
-
getDrillbits
- Specified by:
getDrillbits
in classAbstractParquetGroupScan
-
cloneWithFileSelection
protected AbstractParquetGroupScan cloneWithFileSelection(Collection<org.apache.hadoop.fs.Path> filePaths) throws IOException - Specified by:
cloneWithFileSelection
in classAbstractParquetGroupScan
- Throws:
IOException
-
supportsFileImplicitColumns
protected boolean supportsFileImplicitColumns()- Specified by:
supportsFileImplicitColumns
in classAbstractGroupScanWithMetadata<ParquetMetadataProvider>
-
getPartitionValues
- Specified by:
getPartitionValues
in classAbstractGroupScanWithMetadata<ParquetMetadataProvider>
-
getNewWithChildren
Description copied from interface:PhysicalOperator
Regenerate with this node with a new set of children. This is used in the case of materialization or optimization. -
supportsLimitPushdown
public boolean supportsLimitPushdown()Description copied from class:AbstractGroupScan
Default is not to support limit pushdown.- Specified by:
supportsLimitPushdown
in interfaceGroupScan
- Overrides:
supportsLimitPushdown
in classAbstractGroupScanWithMetadata<ParquetMetadataProvider>
-
getSchema
- Overrides:
getSchema
in classAbstractGroupScanWithMetadata<ParquetMetadataProvider>
-
getStorageConfig
-
getFormatConfig
-
getPath
-
getCondition
-
getPartitionHolder
-
toString
-