Apache Drill 1.21.0 Release Notes

Release date: Feb 21, 2023

Today, we’re happy to announce the availability of Drill 1.21.0. You can download it here.

Sub-task

  • [DRILL-8198] - XML EVF2 reader provideSchema usage
  • [DRILL-8199] - Convert Excel EVF1 to EVF2
  • [DRILL-8208] - Create builder for SqlSelect
  • [DRILL-8209] - Introduce rule for converting join with distinct input to semi-join
  • [DRILL-8210] - Add substring convertlet
  • [DRILL-8211] - Replace deprecated RelNode.getChildExps with Project.getProjects
  • [DRILL-8212] - Join queries fail with StackOverflowError
  • [DRILL-8213] - Replace deprecated RelNode.getRows with RelNode.estimateRowCount
  • [DRILL-8214] - Replace EnumerableTableScan usage with LogicalTableScan
  • [DRILL-8216] - Use EVF-based JSON reader for Values operator
  • [DRILL-8242] - Fix output for HttpHelperFunctions
  • [DRILL-8303] - Add support for inserts into JDBC storage

Bug

  • [DRILL-8086] - Convert the CSV (AKA “compliant text”) reader to EVF V2
  • [DRILL-8181] - Accept nullable args in HTTP plugin UDFs, fix HikariCP default parm names
  • [DRILL-8183] - Respect the ExecConstant.SSL_HANDSHAKE_TIMEOUT option
  • [DRILL-8244] - HTTP_Request Not Passing Down Config Variable
  • [DRILL-8299] - HashMap key type mismatch in MetadataContext
  • [DRILL-8320] - Prevent Infinite Pagination for Index Paginator
  • [DRILL-8327] - GoogleSheets not Reporting Schemata to Info_Schema
  • [DRILL-8349] - GoogleSheets Not Registering Schemas with Non Default Name
  • [DRILL-8351] - UserRemoteException : SYSTEM ERROR: NullPointerException: Schema is currently null. You must call buildSchema(SelectionVectorMode) before this container can return a schema.
  • [DRILL-8372] - Unfreed buffers when running a LIMIT 0 query over delimited text
  • [DRILL-8392] - Empty Tables Causes Index Out of Bounds Exception on PDF Reader
  • [DRILL-8394] - ANALYZE TABLE … COMPUTE STATISTICS fails with a trailing slash
  • [DRILL-8398] - Fix GitHub Actions to use proper JDK version
  • [DRILL-8399] - MS Access Reader Misinterprets Data Types
  • [DRILL-8400] - Fix pruning partitions with pushed transitive predicates
  • [DRILL-8401] - Skip nested MAP column without children when creating parquet tables

New Feature

  • [DRILL-8287] - Add Support for Keyset Based Pagination
  • [DRILL-8289] - Add Threat Hunting Functions

Improvement

  • [DRILL-8117] - Upgrade unit tests to the cluster fixture framework
  • [DRILL-8132] - Improvement of RPC
  • [DRILL-8136] - Overhaul implict type casting logic
  • [DRILL-8140] - Add JSON Post Body to HTTP Rest Storage Plugin
  • [DRILL-8149] - format-excel plugin needs to support POI IOUtils byte array overrides to support big files
  • [DRILL-8153] - Convert OAuth REST APIs to JSON
  • [DRILL-8155] - Introduce new plugin authentication modes
  • [DRILL-8161] - Add Global Credentials to HTTP Storage Plugin
  • [DRILL-8162] - Add OpenAPI Specification documentation for Drill’s REST API
  • [DRILL-8167] - Add JSON Config Options to Format Config
  • [DRILL-8169] - Add UDFs to HTTP Plugin to Facilitate Joins
  • [DRILL-8171] - Convert SequenceFile to EVF2
  • [DRILL-8174] - Convert Avro format to EVF2
  • [DRILL-8177] - Bump default TLS version to 1.3
  • [DRILL-8179] - Convert LTSV Format Plugin to EVF2
  • [DRILL-8186] - Convert XML format to EVF2
  • [DRILL-8202] - Add Options to Skip Malformed JSON Records to HTTP Plugin
  • [DRILL-8204] - Allow Provided Schema for HTTP Plugin in JSON Mode
  • [DRILL-8220] - Add User Translation Support for OAuth Enabled Plugins
  • [DRILL-8223] - Refactor auth modes dropping DRILL_PROCESS and allowing credential providers everywhere
  • [DRILL-8225] - Update LogParser and Yauaa dependencies
  • [DRILL-8229] - Add Parameter to Skip Malformed Records to HTTP UDF
  • [DRILL-8232] - Add support for user credentials to VaultCredentialsProvider
  • [DRILL-8235] - Add Storage Plugin for Google Sheets
  • [DRILL-8236] - Move HttpHelperFunctions to use JSON2 reader
  • [DRILL-8243] - Move JSON Config Options Out of HTTP Plugin
  • [DRILL-8261] - Make the XML format plugin’s attributes MAP an implicit column
  • [DRILL-8269] - Classified metrics for easier maintenance
  • [DRILL-8277] - Publish SNAPSHOT artifacts to the Apache snapshots repo from CI
  • [DRILL-8290] - Early exit from recursive file listing for LIMIT 0 queries
  • [DRILL-8301] - Standardise on UTF-8 encoding for char to byte (and vice versa) conversions
  • [DRILL-8302] - tidy up some char conversions
  • [DRILL-8305] - Add Implicit Fields to Google Sheets Reader
  • [DRILL-8309] - Upgrade slf4j to 2.0.x
  • [DRILL-8310] - Convert Syslog Format to EVF V2
  • [DRILL-8311] - Convert SPSS Format Plugin to EVF V2
  • [DRILL-8314] - Add support for automatically retrying and disabling broken storage plugins
  • [DRILL-8315] - Convert SAS Format Plugin to EVF V2
  • [DRILL-8322] - Add a list of scanned plugin names to the query profile
  • [DRILL-8335] - Add Ability to Query GoogleSheets Tabs by Index
  • [DRILL-8340] - Add Additional Date Manipulation Functions (Part 1)
  • [DRILL-8341] - Add Scanned Plugin List to Sys Profiles Table
  • [DRILL-8342] - Add Automatic Retry for Rate Limited APIs
  • [DRILL-8354] - Add IS_EMPTY Function.
  • [DRILL-8356] - Add File Name to GoogleSheets Plugin
  • [DRILL-8357] - Add new config options to the Splunk storage plugin
  • [DRILL-8358] - Storage plugin for querying other Apache Drill clusters
  • [DRILL-8359] - Add mount and unmount command support to the filesystem plugin
  • [DRILL-8360] - Add Provided Schema for XML Reader
  • [DRILL-8364] - Add Support for OAuth Enabled File Systems
  • [DRILL-8368] - Update Yauaa to 7.9.0
  • [DRILL-8370] - Upgrade splunk-sdk-java to 1.9.3
  • [DRILL-8371] - Add Write/Append Capability to Splunk Plugin
  • [DRILL-8376] - Add Distribution UDFs
  • [DRILL-8378] - Support doing Maven releases using modern JDKs
  • [DRILL-8381] - Add support for filtered aggregate calls
  • [DRILL-8384] - Add Format Plugin for Microsoft Access
  • [DRILL-8386] - Add Support for User Translation for Cassandra
  • [DRILL-8387] - Add Support for User Translation to ElasticSearch Plugin
  • [DRILL-8390] - Minor Improvements to PDF Reader
  • [DRILL-8391] - Set autocomplete=”off” on the password field of web UI login forms
  • [DRILL-8395] - Add Support for INSERT and Drop Table to GoogleSheets Plugin

Wish

  • [DRILL-8035] - Update Janino to 3.1.7 version
  • [DRILL-8163] - Increment the Drill version number to 2.0.0-SNAPSHOT

Task

  • [DRILL-6076] - Reduce the default memory from a total of 13GB to 5GB
  • [DRILL-8270] - Delete obsolete zookeeper patch (tech debt)
  • [DRILL-8271] - Make Storage and Format Config Case Insensitive
  • [DRILL-8276] - Add Support for User Translation for Splunk
  • [DRILL-8304] - Update Calcite to 1.32
  • [DRILL-8317] - Convert LogRegex Format Plugin to EVF V2
  • [DRILL-8325] - Convert PDF Format Plugin to EVF V2
  • [DRILL-8330] - Convert ESRI Shape File Reader to EVF2
  • [DRILL-8350] - Convert PCAP Format Plugin to EVF2
  • [DRILL-8362] - upgrade excel-streaming-reader v4.0.5
  • [DRILL-8363] - upgrade postgresql to 42.4.3 due to security issue
  • [DRILL-8379] - Update Calcite to 1.33.0
  • [DRILL-8396] - Update checkstyle version
← Apache Drill 1.21.1 Release NotesApache Drill 1.20.3 Release Notes →