Package org.apache.drill.exec.store.parquet.compression

Class DrillCompressionCodecFactory

java.lang.Object
org.apache.drill.exec.store.parquet.compression.DrillCompressionCodecFactory
All Implemented Interfaces:
org.apache.parquet.compression.CompressionCodecFactory
public class DrillCompressionCodecFactory extends Object implements org.apache.parquet.compression.CompressionCodecFactory
A delegating compression codec factory that returns (de)compressors based on https://github.com/airlift/aircompressor when possible and falls back to parquet-mr otherwise. The aircompressor lib was introduced into Drill because of difficulties encountered with the JNI-based implementations of lzo, lz4 and zstd in parquet-mr. By modifying the constant AIRCOMPRESSOR_CODECS it is possible to choose which codecs should be routed to which lib. In addition, this class implements parquet-mr's CompressionCodecFactory interface meaning that swapping this factory for e.g. one in parquet-mr will have minimal impact on code in Drill relying on a CompressCodecFactory.

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.parquet.compression.CompressionCodecFactory

    org.apache.parquet.compression.CompressionCodecFactory.BytesInputCompressor, org.apache.parquet.compression.CompressionCodecFactory.BytesInputDecompressor

  • Constructor Summary

    Constructors
    Constructor
    Description
    DrillCompressionCodecFactory(org.apache.hadoop.conf.Configuration config, org.apache.parquet.bytes.ByteBufferAllocator allocator, int pageSize)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static org.apache.parquet.compression.CompressionCodecFactory
    createDirectCodecFactory(org.apache.hadoop.conf.Configuration config, org.apache.parquet.bytes.ByteBufferAllocator allocator, int pageSize)
     
    org.apache.parquet.compression.CompressionCodecFactory.BytesInputCompressor
    getCompressor(org.apache.parquet.hadoop.metadata.CompressionCodecName codecName)
     
    org.apache.parquet.compression.CompressionCodecFactory.BytesInputDecompressor
    getDecompressor(org.apache.parquet.hadoop.metadata.CompressionCodecName codecName)
     
    void
    release()
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • DrillCompressionCodecFactory

      public DrillCompressionCodecFactory(org.apache.hadoop.conf.Configuration config, org.apache.parquet.bytes.ByteBufferAllocator allocator, int pageSize)

  • Method Details

    • createDirectCodecFactory

      public static org.apache.parquet.compression.CompressionCodecFactory createDirectCodecFactory(org.apache.hadoop.conf.Configuration config, org.apache.parquet.bytes.ByteBufferAllocator allocator, int pageSize)

    • getCompressor

      public org.apache.parquet.compression.CompressionCodecFactory.BytesInputCompressor getCompressor(org.apache.parquet.hadoop.metadata.CompressionCodecName codecName)
      Specified by:
      getCompressor in interface org.apache.parquet.compression.CompressionCodecFactory

    • getDecompressor

      public org.apache.parquet.compression.CompressionCodecFactory.BytesInputDecompressor getDecompressor(org.apache.parquet.hadoop.metadata.CompressionCodecName codecName)
      Specified by:
      getDecompressor in interface org.apache.parquet.compression.CompressionCodecFactory

    • release

      public void release()
      Specified by:
      release in interface org.apache.parquet.compression.CompressionCodecFactory