org.apache.drill.exec.store.parquet.columnreaders

Class ColumnReader<V extends ValueVector>

java.lang.Object

org.apache.drill.exec.store.parquet.columnreaders.ColumnReader<V>

Direct Known Subclasses:

NullableFixedByteAlignedReaders.CorruptionDetectingNullableDateReader, NullableFixedByteAlignedReaders.NullableCorruptDateReader, NullableFixedByteAlignedReaders.NullableDateReader, NullableFixedByteAlignedReaders.NullableIntervalReader, VarLengthColumn

public abstract class ColumnReader<V extends ValueVector>
extends Object

Field Summary

Fields

Modifier and Type	Field and Description
static Set<org.apache.parquet.column.Encoding>	DICTIONARY_ENCODINGS
static Set<org.apache.parquet.column.Encoding>	VALUE_ENCODINGS
protected DrillBuf	vectorData

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	ColumnReader(ParquetRecordReader parentReader, org.apache.parquet.column.ColumnDescriptor descriptor, org.apache.parquet.hadoop.metadata.ColumnChunkMetaData columnChunkMetaData, boolean fixedLength, V v, org.apache.parquet.format.SchemaElement schemaElement)

Method Summary

Modifier and Type	Method and Description
int	capacity()
protected boolean	checkVectorCapacityReached()
void	clear()
boolean	determineSize(long recordsReadInCurrentPass) Determines the size of a single value in a variable column.
int	getRecordsReadInCurrentPass()
protected void	hitRowGroupEnd()
protected void	postPageRead()
protected boolean	processPageData(int recordsToReadInThisPass)
void	processPages(long recordsToReadInThisPass)
Future<Long>	processPagesAsync(long recordsToReadInThisPass)
protected abstract void	readField(long recordsToRead)
static int	readIntLittleEndian(DrillBuf in, int offset) This is copied out of Parquet library, didn't want to deal with the unnecessary throws statement they had declared
boolean	readPage() Read a page.
Future<Boolean>	readPageAsync()
protected void	readRecords(int recordsToRead)
protected Future<Integer>	readRecordsAsync(int recordsToRead)
protected int	readRecordsInBulk(int recordsToReadInThisPass)
void	readValues(long recordsToRead)
protected boolean	recordsRequireDecoding()
void	reset()
protected int	totalValuesReadAndReadyToReadInPage()
void	updatePosition()
void	updateReadyToReadPosition()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DICTIONARY_ENCODINGS

public static final Set<org.apache.parquet.column.Encoding> DICTIONARY_ENCODINGS

VALUE_ENCODINGS

public static final Set<org.apache.parquet.column.Encoding> VALUE_ENCODINGS

vectorData

protected DrillBuf vectorData

Constructor Detail

ColumnReader

protected ColumnReader(ParquetRecordReader parentReader,
                       org.apache.parquet.column.ColumnDescriptor descriptor,
                       org.apache.parquet.hadoop.metadata.ColumnChunkMetaData columnChunkMetaData,
                       boolean fixedLength,
                       V v,
                       org.apache.parquet.format.SchemaElement schemaElement)
                throws ExecutionSetupException

Throws:

ExecutionSetupException

Method Detail

getRecordsReadInCurrentPass

public int getRecordsReadInCurrentPass()

processPagesAsync

public Future<Long> processPagesAsync(long recordsToReadInThisPass)

processPages

public void processPages(long recordsToReadInThisPass)
                  throws IOException

Throws:

IOException

clear

public void clear()

readValues

public void readValues(long recordsToRead)

readField

protected abstract void readField(long recordsToRead)

determineSize

public boolean determineSize(long recordsReadInCurrentPass)
                      throws IOException

Determines the size of a single value in a variable column. Return value indicates if we have finished a row group and should stop reading

Parameters:

recordsReadInCurrentPass - records read in current pass

Returns:

true if we should stop reading

Throws:

IOException

readRecordsAsync

protected Future<Integer> readRecordsAsync(int recordsToRead)

readRecords

protected void readRecords(int recordsToRead)

readRecordsInBulk

protected int readRecordsInBulk(int recordsToReadInThisPass)
                         throws IOException

Throws:

IOException

recordsRequireDecoding

protected boolean recordsRequireDecoding()

processPageData

protected boolean processPageData(int recordsToReadInThisPass)
                           throws IOException

Throws:

IOException

updatePosition

public void updatePosition()

updateReadyToReadPosition

public void updateReadyToReadPosition()

reset

public void reset()

capacity

public int capacity()

readPageAsync

public Future<Boolean> readPageAsync()

readPage

public boolean readPage()
                 throws IOException

Read a page. If we need more data, exit the read loop and return true.

Returns:

true if we need more data and page is not read successfully

Throws:

IOException

totalValuesReadAndReadyToReadInPage

protected int totalValuesReadAndReadyToReadInPage()

postPageRead

protected void postPageRead()

hitRowGroupEnd

protected void hitRowGroupEnd()

checkVectorCapacityReached

protected boolean checkVectorCapacityReached()

readIntLittleEndian

public static int readIntLittleEndian(DrillBuf in,
                                      int offset)

This is copied out of Parquet library, didn't want to deal with the unnecessary throws statement they had declared

Parameters:

in - incoming data

offset - offset

Returns:

little endian integer