org.apache.drill.exec.store.parquet

Class FooterGatherer

java.lang.Object

org.apache.drill.exec.store.parquet.FooterGatherer

public class FooterGatherer
extends Object

Constructor Summary

Constructors

Constructor and Description
FooterGatherer()

Method Summary

Modifier and Type	Method and Description
static List<org.apache.parquet.hadoop.Footer>	getFooters(org.apache.hadoop.conf.Configuration conf, List<org.apache.hadoop.fs.FileStatus> statuses, int parallelism) A function to get a list of footers.
static org.apache.parquet.hadoop.Footer	readFooter(org.apache.hadoop.conf.Configuration config, org.apache.hadoop.fs.FileStatus status) An updated footer reader that tries to read the entire footer without knowing the length.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FooterGatherer

public FooterGatherer()

Method Detail

getFooters

public static List<org.apache.parquet.hadoop.Footer> getFooters(org.apache.hadoop.conf.Configuration conf,
                                                                List<org.apache.hadoop.fs.FileStatus> statuses,
                                                                int parallelism)
                                                         throws IOException

A function to get a list of footers.

Parameters:

conf - configuration for file system

statuses - list of file statuses

parallelism - parallelism

Returns:

a list of footers

Throws:

IOException

readFooter

public static org.apache.parquet.hadoop.Footer readFooter(org.apache.hadoop.conf.Configuration config,
                                                          org.apache.hadoop.fs.FileStatus status)
                                                   throws IOException

An updated footer reader that tries to read the entire footer without knowing the length. This should reduce the amount of seek/read roundtrips in most workloads.

Parameters:

config - configuration for file system

status - file status

Returns:

Footer

Throws:

IOException