org.apache.drill.exec.physical.impl.scan.v3.schema

Class ProjectionSchemaTracker

java.lang.Object

org.apache.drill.exec.physical.impl.scan.v3.schema.AbstractSchemaTracker

org.apache.drill.exec.physical.impl.scan.v3.schema.ProjectionSchemaTracker

All Implemented Interfaces:

ScanSchemaTracker

public class ProjectionSchemaTracker
extends AbstractSchemaTracker

Schema tracker for the "normal" case in which schema starts from a simple projection list of column names, optionally with a provided schema. The schema evolves by locating implicit columns, then having he reader define column types, and so on.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.drill.exec.physical.impl.scan.v3.schema.ScanSchemaTracker

ScanSchemaTracker.ProjectionType

Field Summary

Fields inherited from class org.apache.drill.exec.physical.impl.scan.v3.schema.AbstractSchemaTracker

errorContext, isResolved, schema

Constructor Summary

Constructors

Constructor and Description
ProjectionSchemaTracker(ScanProjectionParser.ProjectionParseResult parseResult, boolean allowSchemaChange, CustomErrorContext errorContext)
ProjectionSchemaTracker(TupleMetadata definedSchema, ScanProjectionParser.ProjectionParseResult parseResult, CustomErrorContext errorContext)

Method Summary

Modifier and Type	Method and Description
void	applyEarlyReaderSchema(TupleMetadata readerSchema) If a reader can define a schema before reading data, apply that schema to the scan schema.
void	applyProvidedSchema(TupleMetadata providedSchema)
void	applyReaderSchema(TupleMetadata readerOutputSchema, CustomErrorContext errorContext) Once a reader has read a batch, the reader will have provided a type for each projected column which the reader knows about.
ProjectedColumn	columnProjection(String colName) Return the projection for a column, if any.
void	expandImplicitCol(ColumnMetadata resolved, ImplicitColumnMarker marker) Drill defines a wildcard to expand not just reader columns, but also partition columns.
ProjectionFilter	projectionFilter(CustomErrorContext errorContext) Set up a projection filter using the reader input schema returned from AbstractSchemaTracker.readerInputSchema().

Methods inherited from class org.apache.drill.exec.physical.impl.scan.v3.schema.AbstractSchemaTracker

applyImplicitCols, checkResolved, errorContext, internalSchema, isResolved, missingColumns, outputSchema, projectionType, readerInputSchema, resolveMissingCols, schemaVersion, validateProjection

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ProjectionSchemaTracker

public ProjectionSchemaTracker(TupleMetadata definedSchema,
                               ScanProjectionParser.ProjectionParseResult parseResult,
                               CustomErrorContext errorContext)

ProjectionSchemaTracker

public ProjectionSchemaTracker(ScanProjectionParser.ProjectionParseResult parseResult,
                               boolean allowSchemaChange,
                               CustomErrorContext errorContext)

Method Detail

columnProjection

public ProjectedColumn columnProjection(String colName)

Description copied from interface: ScanSchemaTracker

Return the projection for a column, if any.

applyProvidedSchema

public void applyProvidedSchema(TupleMetadata providedSchema)

applyEarlyReaderSchema

public void applyEarlyReaderSchema(TupleMetadata readerSchema)

Description copied from interface: ScanSchemaTracker

If a reader can define a schema before reading data, apply that schema to the scan schema. Allows the scan to report its output schema before the first batch of data if the scan schema becomes resolved after the early reader schema.

projectionFilter

public ProjectionFilter projectionFilter(CustomErrorContext errorContext)

Set up a projection filter using the reader input schema returned from AbstractSchemaTracker.readerInputSchema().

• If this is an empty projection (@{code SELECT COUNT(*)}), then noting can be projected at all.
• If this is an explicit projection (@code SELECT a, b)}, then the set of top-level columns is fixed, though the types are unknown. Maps allow new members depending on the map projection: a generic projection (m) allows new members, a specific projection (m.a, m.b) does not allow new members.
• If the schema has been resolved and is now fixed (closed), then no new columns are allowed either at the top level or in maps.
• If the schema is open (SELECT * for the first reader, or schema change is allowed in the second reader), and we have no columns, then just project everything.
• If the schema is open, but we have seen some columns, then columns can still be added, but existing columns must match the existing schema.

Static filters handle the simple "none" and "starting from nothing all" cases. The dynamic schema filter handles the case of existing columns whether dynamic or static.

Parameters:

errorContext - the reader-specific error context to use if errors are found

Returns:

a filter used to decide which reader columns to project during reading

applyReaderSchema

public void applyReaderSchema(TupleMetadata readerOutputSchema,
                              CustomErrorContext errorContext)

Description copied from interface: ScanSchemaTracker

Once a reader has read a batch, the reader will have provided a type for each projected column which the reader knows about. For a wildcard projection, the reader will have added all the columns that it found. This call takes the reader output schema and merges it with the current scan schema to resolve dynamic types to concrete types and to add newly discovered columns.

The process can raise an exception if the reader projects a column that it shouldn't (which is not actually possible because of the way the ResultSetLoader works.) An error can also occur if the reader provides a type different than that already defined in the scan schema by a defined schema, a provided schema, or a previous reader in the same scan. In such cases, the reader is expected to have converted its input type to the specified type, which was presumably selected because the reader is capable of the required conversion.

Parameters:

readerOutputSchema - the actual schema produced by a reader when reading a record batch

errorContext - the reader-specific error context to use if errors are found

expandImplicitCol

public void expandImplicitCol(ColumnMetadata resolved,
                              ImplicitColumnMarker marker)

Description copied from interface: ScanSchemaTracker

Drill defines a wildcard to expand not just reader columns, but also partition columns. When the implicit column handlers sees that the query has a wildcard (by calling #isProjectAll()), the handler then determines which partition columns are needed and calls this method to add each one.