Apache Drill 1.20.0 Release Notes

Release date: February 25, 2022

Today, we’re happy to announce the availability of Drill 1.20.0. You can download it here.

New Feature

  • [DRILL-7983] - Add a REST API to support the get running or completed profiles
  • [DRILL-7985] - Support Mongo aggregate, union, project, limit, sort pushdowns
  • [DRILL-8027] - Format plugin for Apache Iceberg
  • [DRILL-8073] - Add support for persistent table and storage aliases
  • [DRILL-8107] - Hadoop2 backport Maven profile

Improvement

  • [DRILL-1282] - Add read and write support for Parquet v2
  • [DRILL-7951] - Add Response Metadata Fields to HTTP Storage Plugin
  • [DRILL-7955] - Extend XOR bitwise functions to match AND and OR
  • [DRILL-7969] - Add support for reading and writing Parquet files using Brotli, LZ4 and Zstandard codecs
  • [DRILL-7970] - Add URL Parameters to HTTP Plugin
  • [DRILL-7971] - Support Elasticsearch authentication
  • [DRILL-7981] - Bump commons-compress from 1.20 to 1.21 for CVE-2021-36090
  • [DRILL-7988] - Add credentials provider support for API connections in HTTP plugin
  • [DRILL-7989] - Use the UTC formatter in the JSON reader
  • [DRILL-7991] - Add Configuration Option to HDF5 Reader to Skip Preview
  • [DRILL-7995] - Add ability to query OCI OS
  • [DRILL-7999] - Base Docker image on maven:3.8.2-jdk-11 and openjdk-11
  • [DRILL-8005] - Add Writer to JDBC Storage Plugin
  • [DRILL-8006] - Leading and Trailing Whitespace Causes Query Failures in Excel Files
  • [DRILL-8008] - Add Config Option to HTTP Plugin to Skip SSL Validation
  • [DRILL-8011] - Add Dropbox File System to Drill
  • [DRILL-8014] - Bump Calcite to 1.21.0r5
  • [DRILL-8016] - Default to lazy outbound connections for storage-jdbc and storage-splunk
  • [DRILL-8017] - Fix LGTM Issues Relating to Equals and HashCode Functions
  • [DRILL-8018] - Bump Splunk SDK to version 1.7.1
  • [DRILL-8019] - Use HTTP error codes to signal failures in the REST API
  • [DRILL-8020] - Add JSON Configuration Options to HTTP Rest Plugin
  • [DRILL-8021] - Add the srv protocol support for the mongo storage
  • [DRILL-8022] - Add Provided Schema Support for Excel Reader
  • [DRILL-8028] - Add PDF Format Plugin
  • [DRILL-8029] - Upgrade project parent POM to 24 version
  • [DRILL-8032] - Remove obsolete Python scripts from the project
  • [DRILL-8039] - More LGTM Code Cleanup
  • [DRILL-8040] - Return an HTTP error code from failed REST API operations
  • [DRILL-8047] - Add a custom authn provider for HashiCorp Vault
  • [DRILL-8054] - Add SAS Format Plugin
  • [DRILL-8056] - Add OAuth2 Support for HTTP Rest Plugin
  • [DRILL-8057] - INFORMATION_SCHEMA filter push down is inefficient
  • [DRILL-8060] - Avoid the NPE in the Filter of Iceberg Implementor
  • [DRILL-8072] - Fix NPE in HTTP Post Requests
  • [DRILL-8076] - Remove unused Vault token BOOT opt
  • [DRILL-8078] - Add WEEK to Date Extract
  • [DRILL-8080] - Enable the describe Case-Insensitive for Phoenix
  • [DRILL-8081] - Maven Connection timed out
  • [DRILL-8092] - Add Auto Pagination to HTTP Storage Plugin
  • [DRILL-8095] - format-excel reader - upgrade to POI 5.2.0

Bug

  • [DRILL-3193] - TestDrillbitResilience#interruptingWhileFragmentIsBlockedInAcquiringSendingTicket hangs and fails
  • [DRILL-3194] - TestDrillbitResilience#memoryLeaksWhenFailed hangs
  • [DRILL-5612] - Random failure in TestMergeJoinWithSchemaChanges
  • [DRILL-6105] - SYSTEM ERROR: NullPointerException
  • [DRILL-7926] - The “ age “ function is not working properly.
  • [DRILL-7947] - Unable to aggregate or filter nullable decimals in large record sets
  • [DRILL-7948] - Unable to query file with required fixed_len_byte_array decimal columns
  • [DRILL-7950] - Drill classpath includes both avatica-core:jar:1.15.0 and avatica:jar:1.17.0
  • [DRILL-7952] - Fix Scan Stats for Mongo Plugin
  • [DRILL-7968] - ANALYZE TABLE … REFRESH METADATA fails with FLOAT4 column
  • [DRILL-7972] - ClassCastException when joining RDBMS and Elasticsearch tables
  • [DRILL-7975] - Connection to Splunk Drill Storage Plugin fails intermittently
  • [DRILL-7976] - Fix XSS Vulnerability
  • [DRILL-7979] - Self-Closing XML Tags Cause Schema Change Exceptions
  • [DRILL-7986] - Drill crashes when using the streaming connection with arrays
  • [DRILL-7990] - Unable to disable requireTail in HTTP storage plugin API
  • [DRILL-7997] - Fix the type mismatch of mongo module in eclipse
  • [DRILL-8001] - Some new Docker images are based on OpenJDK JRE images but full JDK is required
  • [DRILL-8002] - Expose clear error message for the case of errors from ElasticSearch
  • [DRILL-8004] - Splunk Sends Newlines in INT Fields, Causing Exceptions
  • [DRILL-8009] - DrillConnectionImpl#isValid() doesn’t correspond JDBC API
  • [DRILL-8010] - Build fails with unresolved incubator-iceberg dependency
  • [DRILL-8026] - Sqlline 1.12 upgrade
  • [DRILL-8041] - Fix mongo scan spec deserialization
  • [DRILL-8042] - Select star from MongoDB with aggregated pipeline fails with empty $project error
  • [DRILL-8043] - Excel Reader Regression Fix
  • [DRILL-8045] - Fix storage export regression introduced by DRILL-8040
  • [DRILL-8046] - Enabling ignored test cases for SchemaChange
  • [DRILL-8049] - Make rule names for Iceberg plugin instances unique
  • [DRILL-8050] - Fix license-maven-plugin unknown file extension warnings
  • [DRILL-8053] - Reduce Docker image size
  • [DRILL-8058] - NPE: Cannot invoke “org.apache.calcite.rel.core.TableScan.getTable()” because “scan” is null
  • [DRILL-8064] - Cannot query text files with names that contain a backslash
  • [DRILL-8066] - Cannot convert non-finite floating point literals
  • [DRILL-8074] - Upgrade log4j because of CVE-2021-44228
  • [DRILL-8090] - LIMIT clause is pushed down to an invalid OFFSET-FETCH clause for MS SQL Server
  • [DRILL-8104] - Upgrade protobuf-java because of CVE-2021-22569
  • [DRILL-8108] - Excel Reader Fails with Duplicate Columns
  • [DRILL-8112] - Excel Reader Ignores HeaderRow Config Param
  • [DRILL-8114] - Prevent applying Iceberg project on non-Iceberg tables
  • [DRILL-8118] - Add Option to Allow Disk Use on Mongo Queries
  • [DRILL-8119] - MongoDB ObjectID becomes MAP under store.mongo.bson.record.reader = false
  • [DRILL-8122] - Change kafka metadata obtaining due to KAFKA-5697
  • [DRILL-8126] - Ignore OAuth Parameter in Storage Plugin
  • [DRILL-8129] - Storage-phoenix cannot resolve OSGi bundle apache-ds.jdbm1
  • [DRILL-8130] - Upgrade Hadoop 2 to 2.10.1 because of CVE-2020-9492
  • [DRILL-8131] - Infinite planning when storage-jdbc or storage-phoenix is enabled
  • [DRILL-8134] - Regression: cannot query Parquet INT96 columns as timestamps
  • [DRILL-8144] - Cannot launch Drill 1.20 RC 4 on Windows

Test

  • [DRILL-3967] - Broken Test: TestDrillbitResilience.cancelAfterEverythingIsCompleted()

Wish

Task

  • [DRILL-7942] - Update Mockito to latest 3.x
  • [DRILL-7943] - Update Hamcrest to 2.2
  • [DRILL-7949] - documentation error - missing link
  • [DRILL-7982] - Timed out waiting for container port to open
  • [DRILL-7994] - Dependency version updates for severe vulnerabilities
  • [DRILL-8024] - Bump Mongo client and Testcontainers version to latest
  • [DRILL-8025] - Bump HBase client version to 2.4.8
  • [DRILL-8111] - Remove lombok usage

Sub-task

  • [DRILL-7973] - Fix GitHub CI intermittent failures
  • [DRILL-7996] - Documentation for OCI OS Storage Plugin
  • [DRILL-8030] - Intermittent TestDrillbitResilience cancelInMiddleOfFetchingResults and foreman_runTryEnd failures
  • [DRILL-8061] - Add Impersonation Support for Phoenix
← Apache Drill 1.20.1 Release NotesApache Drill 1.19.0 Release Notes →