Blog

Drill 1.21.1 Released

Apache Drill 1.21.1 has been released.

Announcing Drill 1.21!

Announcing Drill 1.21: New Connectors, Functions and Much Better Stability.

Drill 1.20.3 Released

Apache Drill 1.20.3 has been released.

Drill 1.20.2 Released

Apache Drill 1.20.2 has been released.

Drill 1.20.1 Released

Apache Drill 1.20.1 has been released.

Drill 1.20 Released

Apache Drill 1.20 has been released providing the Apache Phoenix storage plugin with impersonation support, the Apache Iceberg format plugin, expanded push down support for MongoDB, persistent table and storage aliases, a release for Hadoop 2 environments, write support in the JDBC storage plugin, SAS and PDF format plugins, pagination and OAuth support in the HTTP storage plugin, a HashiCorp Vault authentication and credential storage providers and read/write support for all compression codecs and both format versions of Parquet.

2021 in Review

A short post to share Cong Luo's slide deck reviewing the Drill project in 2021.

The reports of my death have been greatly exaggerated

There's a somewhat breathless post entitled "The Death of Apache Drill" in a blog that has as a theme the imminent demise of technologies previously or currently associated with Hadoop, with the exception Trino (formerly known as PrestoSQL).

Drill provider for Airflow

In its provider package release this month, the Apache Airflow project added a provider for interacting with Apache Drill. This allows data engineers and data scientists to incorporate Drill queries in their Airflow DAGs, enabling the automation of big data and data science workflows.

Streaming data from the Drill REST API

The release of Apache Drill 1.19 saw a major change under the hood of Drill's REST API with the introduction of a streaming data path for query results moving from Drill and over the network to the initiating client. The result is better memory utilisation, less blocking and a more reliable API.

Drill 1.19 Released

Apache Drill 1.19 has been released providing new storage plugins for Cassandra, Elasticsearch and Splunk, updating several storage plugins to EVF, adding Linux ARM64 support and JSON reader improvements, and more.

Drill 1.18 Released

Apache Drill 1.18 has been released providing Drill Metadata management "Drill Metastore", Format Plugins for HDF5 and SPSS, Storage Plugins for Generic HTTP REST APIs and Apache Druid, Support for DICT type in RowSet Framework, Dynamic credit based flow control, Support for injecting BufferManager into UDF, Drill RDBMS Metastore"

Drill 1.17 Released

Apache Drill 1.17's highlights are: Hadoop, Kafka, Sqlline, and Calcite upgrades, .zip compression, file/table function for schema provisioning, new UDFs, and more.

Drill User Meetup 2019

Drill User Meetup coming soon.

Drill 1.16 Released

Apache Drill 1.16's highlights are: table statistics, schema provisioning for text files, parquet metadata caching improvements, several Web UI improvements, and more.

Drill 1.15 Released

Apache Drill 1.15's highlights are: SQLLine upgrade, index support, ability to secure znodes with custom ACLs, INFORMATION_SCHEMA files table, systemfunctions table, and more.

Learning Apache Drill book from O'Reilly Media

A new book for learning to work with Apache Drill from two Drill comitters has been published through O'Reilly Media. The book contains helpful explanations and recipes for different types of user from big data admins to data scientists. It also presents information relevant to developers interested in customising Drill, contributing to the open source project or just taking a peek under the hood.

Drill User Meetup 2018

Drill User Meetup coming soon.

Drill Developer Day 2018

Drill Developer Day coming soon.

Drill 1.14 Released

Apache Drill 1.14's highlights are: Ability to run Drill in a Docker container, ability to export and save storage plugin configurations, a storage plugin configuration file to manage storage plugin configurations, an image metadata format plugin, option to set Hive properties at the session level, and more.

Drill 1.13 Released

Apache Drill 1.13's highlights are: YARN support, support for HTTP Kerberos authentication using SPNEGO, SQL syntax highlighting of queries, and user and distribution specific configuration checks during startup.

Running SQL Queries on Amazon S3

Drill enables you to run SQL queries directly on data in S3. There's no need to ingest the data into a managed cluster or transform the data. This is a step-by-step tutorial on how to use Drill with S3.

Drill 1.12 Released

Apache Drill 1.12's highlights are: Kafka and OpenTSDB storage plugins, SSL and network encryption support, queue-based memory assignment for buffering operators, networking functions, and the ability to prevent users from accessing paths outside the root of a workspace.

Drill 1.11 Released

Apache Drill 1.11's highlights are: Cryptography-related functions, spill to disk for the hash aggregate operator, Format plugin support for PCAP files, ability to change the HDFS block Size for Parquet files, ability to store query profiles in memory, configurable CTAS directory and file permissions option, support for network encryption, relative paths stored in the metadata file, and support for ANSI_QUOTES.

Drill 1.10 Released

Apache Drill 1.10's highlights are: CTTAS, improved fault tolerance, Drill version and statistics in Web Console, implicit interpretation of INT96, and Kerberos authentication.

Drill 1.9 Released

Apache Drill 1.9's highlights are: asynchronous Parquet reader, Parquet filter pushdown, dynamic UDF support, and HTTPD format plugin.

Drill 1.8 Released

Apache Drill 1.8's highlights are: metadata cache pruning, IF EXISTS support, DESCRIBE SCHEMA command, multi-byte delimiter support, and new parameters for filter selectivity estimates.

Drill 1.7 Released

Apache Drill 1.7's highlights are: Monitoring via JMX, Hive CHAR data type support, and HBase 1.x support.

Drill 1.6 Released

Apache Drill 1.6's highlights are: Inbound impersonation and additional custom window frames.

Drill 1.5 Released

Apache Drill 1.5's highlights are: Authentication and security for the Web interface and REST API, experimental query support for Apache Kudu (incubating), an improved memory allocator, and configurable caching for Hive metadata.

Drill 1.4 Released

Apache Drill 1.4's highlights are: "select with options" queries that can change storage plugin settings, improved behavior when parsing CSV file header names, a variable to set non-pretty (e.g. compact) printing of JSON, and better drillbit.log files that include query text.

Drill 1.3 Released

Drill 1.3 has been released. Users can now query Hadoop sequence files and text delimited files with headers. In addition, this release provides significant performance and usability improvements for working with Amazon S3. Drill 1.3 also adds support for heterogeneous types, enabling queries on datasets with columns that have more than one data type (commonly seen in JSON files, MongoDB collections, etc.).

Drill 1.2 Released

Drill 1.2 has been released. This release includes a new JDBC storage plugin for querying relational databases, as well as new window functions such as `NTILE`, `FIRST_VALUE`, `LAST_VALUE`, `LEAD` and `LAG`. This release addresses 210 JIRAs, including performance, stability and documentation enhancements.

Drill Tutorial at NoSQL Now! 2015

Attend the 3-hour Drill tutorial at the NoSQL Now! 2015 conference to learn more about Apache Drill.

Drill 1.1 Released

Drill 1.1 has been released, providing window functions, automatic partitioning, improved MongoDB support and more. This release addresses 162 JIRAs.

The Apache Software Foundation Announces Apache Drill 1.0

The Apache Software Foundation announced today the availability of Apache Drill 1.0, the schema-free SQL query engine for Apache Hadoop, NoSQL and cloud storage.

Drill 1.0 Released

Drill 1.0 has been released, representing a major milestone for the Drill community. Drill in now production-ready, making it easier than ever to explore and analyze data in non-relational datastores.

Drill 0.9 Released

The community has just released Drill 0.9, which includes 199 resolved JIRAs and numerous enhancements.

Apache Parquet Graduates to a Top-Level Project

The Apache Parquet project has graduated to a top-level project (TLP) at the Apache Software Foundation.

Drill 0.8 Released

The community has just released Drill 0.8, which includes 243 resolved JIRAs and numerous enhancements.

MicroStrategy Announces Drill Support

MicroStrategy announced that it is now certified with Apache Drill, providing a first-class BI experience for Hadoop and NoSQL databases.

Schema-free JSON Data Infrastructure

JSON has emerged as the de-facto standard data exchange format. Data infrastructure technologies such as Apache Drill, MongoDB and Elasticsearch are embracing JSON as their native data models, bringing game-changing ease-of-use and agility to developers and analysts.

Drill 0.7 Released

The community has just released Drill 0.7, which includes 228 resolved JIRAs and numerous enhancements.

What's Coming in 2015?

Drill is now a top-level project, and the community is expanding rapidly. Find out more about some of the new features planned for 2015.

Apache Drill Q&A Panelist Spotlight

Join us on Twitter for a live Q&A on Wednesday, December 17.

Apache Drill Graduates to a Top-Level Project

Drill has graduated to a Top-Level Project at Apache. This marks a significant accomplishment for the Drill community, which now includes dozens of developers working at a variety of companies.

SQL on MongoDB

The MongoDB storage plugin for Drill enables analytical queries on MongoDB databases. Drill's schema-free JSON data model is a natural fit for MongoDB's data model.

Want to contribute a blog post? Check out the source for some of the existing posts to see how it's done. When you're ready, email your Markdown file to dev@drill.apache.org.

Third-Party Articles

How to Turn Raw Data from Yelp into Insights in Minutes with Apache Drill (Nov 13, 2014)

How to Use SQL, Hadoop, Drill, REST, JSON, NoSQL, and HBase in a Simple REST Client (Jan 6, 2015)