org.apache.drill.exec.physical.impl.scan.v3.lifecycle

Class MissingColumnHandlerBuilder

java.lang.Object

org.apache.drill.exec.physical.impl.scan.v3.lifecycle.MissingColumnHandlerBuilder

public class MissingColumnHandlerBuilder
extends Object

Builds the handler which provides values for columns in an explicit project list but for which the reader provides no values. Obtains types from a defined or provided schema, or using a configurable default type. Fills in null values, or a default value configured in a provided schema.

The set of missing columns may differ per reader or even per batch within a reader. If reader 1 reads all columns, but reader 2 reads a subset, then this class will use the column types from reader 1 when creating the columns missing from reader 2.

Unfortunately, Drill cannot predict the future, so the opposite scenario will end badly: Reader 2 comes first, omits column "c", this class chooses a default value, then Reader 1 wants the column to be some other type. The query will fail with a type mismatch error.

Specifically, the mechanism uses the following rules to infer column type:

• For resolved columns (those with a type), use that type. If the type is non-nullable, fill in a default value (generally 0 or blank.) A column is resolved if given by a defined schema, a provided schema or a prior reader.
• For unresolved columns (those without a type), use the default type configured in this builder. If no type is provied, use a "default default" of Nullable INT, Drill's classic choice.

Note that Drill is not magic: relying on the default type is likely to cause a type conflict across readers or across scans. A default has no way of knowing if it matches the same column read in some other fragment on some other node.

Work is separated in a schema-time part (to resolve column types) and a read-time part (to create and fill the needed vectors.)

Caveats

The project mechanism handles nested "missing" columns as mentioned above. This works to create null columns within maps that are defined by the data source. However, the mechanism does not currently handle creating null columns within repeated maps or lists. Doing so is possible, but requires adding a level of cardinality computation to create the proper number of "inner" values.

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	allowRequiredNullColumns
static TypeProtos.MajorType	DEFAULT_NULL_TYPE
protected TupleMetadata	inputSchema
protected TypeProtos.MajorType	nullType
protected TupleMetadata	outputSchema
protected ResultVectorCache	vectorCache

Constructor Summary

Constructors

Constructor and Description
MissingColumnHandlerBuilder()

Method Summary

Modifier and Type	Method and Description
MissingColumnHandlerBuilder	allowRequiredNullColumns(boolean flag)
StaticBatchBuilder	build()
TupleMetadata	buildSchema()
MissingColumnHandlerBuilder	inputSchema(TupleMetadata inputSchema)
MissingColumnHandlerBuilder	nullType(TypeProtos.MajorType nullType)
MissingColumnHandlerBuilder	vectorCache(ResultVectorCache vectorCache)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_NULL_TYPE

public static final TypeProtos.MajorType DEFAULT_NULL_TYPE

inputSchema

protected TupleMetadata inputSchema

nullType

protected TypeProtos.MajorType nullType

allowRequiredNullColumns

protected boolean allowRequiredNullColumns

vectorCache

protected ResultVectorCache vectorCache

outputSchema

protected TupleMetadata outputSchema

Constructor Detail

MissingColumnHandlerBuilder

public MissingColumnHandlerBuilder()

Method Detail

inputSchema

public MissingColumnHandlerBuilder inputSchema(TupleMetadata inputSchema)

nullType

public MissingColumnHandlerBuilder nullType(TypeProtos.MajorType nullType)

allowRequiredNullColumns

public MissingColumnHandlerBuilder allowRequiredNullColumns(boolean flag)

vectorCache

public MissingColumnHandlerBuilder vectorCache(ResultVectorCache vectorCache)

buildSchema

public TupleMetadata buildSchema()

build

public StaticBatchBuilder build()