Package org.apache.drill.exec.physical.impl.scan.v3.schema

Class SchemaUtils

java.lang.Object
org.apache.drill.exec.physical.impl.scan.v3.schema.SchemaUtils
public class SchemaUtils extends Object
Set of schema utilities that don't fit well as methods on the column or tuple classes.

Provides methods to check if a column is consistent with the projection requested for a query. Used for scans: the reader offers certain columns and the scan operator must decide whether to accept them, and if so, if the column that has actually appeared is consistent with the projection schema path provided by the planner. An obvious example is if projection asks for a[0] (and array), but the reader offer up a as a non-array column.

Checks are reasonable, but not complete. Particularly in the DICT case, projection depends on multiple factors, such as the type of the key and values. This class does not (yet) handle that complexity. Instead, the goal is no false negatives for the complex cases, while catching the simple cases.

The Project operator or other consuming operator is the final arbitrator of whether a particular column satisfies a particular projection. This class tries to catch those errors early to provide better error messages.

  • Field Details

    • logger

      protected static final org.slf4j.Logger logger

  • Constructor Details

    • SchemaUtils

      public SchemaUtils()

  • Method Details

    • isConsistent

      public static boolean isConsistent(ProjectedColumn colReq, ColumnMetadata readCol)
      Check if the given read column is consistent with the projection requested for that column. Does not handle subtleties such as DICT key types, actual types in a UNION, etc.
      Parameters:
      colReq - the column-level projection description
      readCol - metadata for the column which the reader has actually produced
      Returns:
      true if the column is consistent with projection (or if the column is too complex to check), false if the column is not consistent and represents an error case. Also returns true if the column is not projected, as any type of column can be ignored

    • verifyCompatibility

      public static void verifyCompatibility(ProjectedColumn colReq, ColumnMetadata actual, String source, CustomErrorContext errorContext)
      Perform the column-level projection as described in isConsistent(ProjectedColumn, ColumnMetadata), and raise a UserException if the column is not consistent with projection.
      Parameters:
      colReq - the column-level projection description
      actual - metadata for the column which the reader has actually produced
      errorContext - additional error context to pass along in the exception
      Throws:
      UserException - if the read column is not consistent with the projection description for the column

    • verifyConsistency

      public static void verifyConsistency(ColumnMetadata existing, ColumnMetadata revised, String source, CustomErrorContext errorContext)

    • verifyProjection

      public static void verifyProjection(ColumnMetadata existing, ColumnMetadata revised, String source, CustomErrorContext errorContext)

    • mergeColProperties

      public static void mergeColProperties(ColumnMetadata existing, ColumnMetadata revised)

    • mergeProperties

      public static void mergeProperties(Propertied existing, Propertied revised)

    • isStrict

      public static boolean isStrict(TupleMetadata schema)

    • markStrict

      public static void markStrict(TupleMetadata schema)

    • implicitColType

      public static String implicitColType(ColumnMetadata col)

    • isImplicit

      public static boolean isImplicit(ColumnMetadata col)

    • markImplicit

      public static void markImplicit(ColumnMetadata col, String value)

    • markAsPartition

      public static void markAsPartition(ColumnMetadata col, int level)

    • markExcludeFromWildcard

      public static void markExcludeFromWildcard(ColumnMetadata col)

    • isExcludedFromWildcard

      public static boolean isExcludedFromWildcard(ColumnMetadata col)

    • projectAll

      public static ScanProjectionParser.ProjectionParseResult projectAll()

    • markProjectAll

      public static void markProjectAll(ColumnMetadata col)

    • projectNone

      public static ScanProjectionParser.ProjectionParseResult projectNone()

    • isProjectAll

      public static boolean isProjectAll(TupleMetadata tuple)

    • isProjectNone

      public static boolean isProjectNone(TupleMetadata tuple)

    • copyMapProperties

      public static void copyMapProperties(ProjectedColumn source, ColumnMetadata dest)

    • getColumnMetadata

      public static ColumnMetadata getColumnMetadata(String name, org.apache.calcite.rel.type.RelDataType relDataType)
      Converts specified RelDataType relDataType into ColumnMetadata. For the case when specified relDataType is struct, map with recursively converted children will be created.
      Parameters:
      name - filed name
      relDataType - filed type
      Returns:
      ColumnMetadata which corresponds to specified RelDataType relDataType