org.apache.drill.exec.physical.impl.scan.v3.schema

Class SchemaUtils

java.lang.Object

org.apache.drill.exec.physical.impl.scan.v3.schema.SchemaUtils

public class SchemaUtils
extends Object

Set of schema utilities that don't fit well as methods on the column or tuple classes.

Provides methods to check if a column is consistent with the projection requested for a query. Used for scans: the reader offers certain columns and the scan operator must decide whether to accept them, and if so, if the column that has actually appeared is consistent with the projection schema path provided by the planner. An obvious example is if projection asks for a[0] (and array), but the reader offer up a as a non-array column.

Checks are reasonable, but not complete. Particularly in the DICT case, projection depends on multiple factors, such as the type of the key and values. This class does not (yet) handle that complexity. Instead, the goal is no false negatives for the complex cases, while catching the simple cases.

The Project operator or other consuming operator is the final arbitrator of whether a particular column satisfies a particular projection. This class tries to catch those errors early to provide better error messages.

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	logger

Constructor Summary

Constructors

Constructor and Description
SchemaUtils()

Method Summary

Modifier and Type	Method and Description
static void	copyMapProperties(ProjectedColumn source, ColumnMetadata dest)
static String	implicitColType(ColumnMetadata col)
static boolean	isConsistent(ProjectedColumn colReq, ColumnMetadata readCol) Check if the given read column is consistent with the projection requested for that column.
static boolean	isExcludedFromWildcard(ColumnMetadata col)
static boolean	isImplicit(ColumnMetadata col)
static boolean	isProjectAll(TupleMetadata tuple)
static boolean	isProjectNone(TupleMetadata tuple)
static boolean	isStrict(TupleMetadata schema)
static void	markAsPartition(ColumnMetadata col, int level)
static void	markExcludeFromWildcard(ColumnMetadata col)
static void	markImplicit(ColumnMetadata col, String value)
static void	markProjectAll(ColumnMetadata col)
static void	markStrict(TupleMetadata schema)
static void	mergeColProperties(ColumnMetadata existing, ColumnMetadata revised)
static void	mergeProperties(Propertied existing, Propertied revised)
static ScanProjectionParser.ProjectionParseResult	projectAll()
static ScanProjectionParser.ProjectionParseResult	projectNone()
static void	verifyCompatibility(ProjectedColumn colReq, ColumnMetadata actual, String source, CustomErrorContext errorContext) Perform the column-level projection as described in #isConsistent(RequestedColumn, ColumnMetadata), and raise a UserException if the column is not consistent with projection.
static void	verifyConsistency(ColumnMetadata existing, ColumnMetadata revised, String source, CustomErrorContext errorContext)
static void	verifyProjection(ColumnMetadata existing, ColumnMetadata revised, String source, CustomErrorContext errorContext)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

protected static final org.slf4j.Logger logger

Constructor Detail

SchemaUtils

public SchemaUtils()

Method Detail

isConsistent

public static boolean isConsistent(ProjectedColumn colReq,
                                   ColumnMetadata readCol)

Check if the given read column is consistent with the projection requested for that column. Does not handle subtleties such as DICT key types, actual types in a UNION, etc.

Parameters:

colReq - the column-level projection description

readCol - metadata for the column which the reader has actually produced

Returns:

true if the column is consistent with projection (or if the column is too complex to check), false if the column is not consistent and represents an error case. Also returns true if the column is not projected, as any type of column can be ignored

verifyCompatibility

public static void verifyCompatibility(ProjectedColumn colReq,
                                       ColumnMetadata actual,
                                       String source,
                                       CustomErrorContext errorContext)

Perform the column-level projection as described in #isConsistent(RequestedColumn, ColumnMetadata), and raise a UserException if the column is not consistent with projection.

Parameters:

colReq - the column-level projection description

actual - metadata for the column which the reader has actually produced

errorContext - additional error context to pass along in the exception

Throws:

UserException - if the read column is not consistent with the projection description for the column

verifyConsistency

public static void verifyConsistency(ColumnMetadata existing,
                                     ColumnMetadata revised,
                                     String source,
                                     CustomErrorContext errorContext)

verifyProjection

public static void verifyProjection(ColumnMetadata existing,
                                    ColumnMetadata revised,
                                    String source,
                                    CustomErrorContext errorContext)

mergeColProperties

public static void mergeColProperties(ColumnMetadata existing,
                                      ColumnMetadata revised)

mergeProperties

public static void mergeProperties(Propertied existing,
                                   Propertied revised)

isStrict

public static boolean isStrict(TupleMetadata schema)

markStrict

public static void markStrict(TupleMetadata schema)

implicitColType

public static String implicitColType(ColumnMetadata col)

isImplicit

public static boolean isImplicit(ColumnMetadata col)

markImplicit

public static void markImplicit(ColumnMetadata col,
                                String value)

markAsPartition

public static void markAsPartition(ColumnMetadata col,
                                   int level)

markExcludeFromWildcard

public static void markExcludeFromWildcard(ColumnMetadata col)

isExcludedFromWildcard

public static boolean isExcludedFromWildcard(ColumnMetadata col)

projectAll

public static ScanProjectionParser.ProjectionParseResult projectAll()

markProjectAll

public static void markProjectAll(ColumnMetadata col)

projectNone

public static ScanProjectionParser.ProjectionParseResult projectNone()

isProjectAll

public static boolean isProjectAll(TupleMetadata tuple)

isProjectNone

public static boolean isProjectNone(TupleMetadata tuple)

copyMapProperties

public static void copyMapProperties(ProjectedColumn source,
                                     ColumnMetadata dest)