org.apache.drill.exec.store.parquet

Class ParquetRecordWriter

java.lang.Object

org.apache.drill.exec.store.AbstractRecordWriter

org.apache.drill.exec.store.ParquetOutputRecordWriter

org.apache.drill.exec.store.parquet.ParquetRecordWriter

All Implemented Interfaces:

RecordWriter

public class ParquetRecordWriter
extends ParquetOutputRecordWriter

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	ParquetRecordWriter.DictParquetConverter
class	ParquetRecordWriter.MapParquetConverter
class	ParquetRecordWriter.RepeatedDictParquetConverter
class	ParquetRecordWriter.RepeatedListParquetConverter
class	ParquetRecordWriter.RepeatedMapParquetConverter

Nested classes/interfaces inherited from class org.apache.drill.exec.store.ParquetOutputRecordWriter

ParquetOutputRecordWriter.BaseFieldConverter, ParquetOutputRecordWriter.BigIntParquetConverter, ParquetOutputRecordWriter.BitParquetConverter, ParquetOutputRecordWriter.DateParquetConverter, ParquetOutputRecordWriter.Decimal18ParquetConverter, ParquetOutputRecordWriter.Decimal28DenseParquetConverter, ParquetOutputRecordWriter.Decimal28SparseParquetConverter, ParquetOutputRecordWriter.Decimal38DenseParquetConverter, ParquetOutputRecordWriter.Decimal38SparseParquetConverter, ParquetOutputRecordWriter.Decimal9ParquetConverter, ParquetOutputRecordWriter.Float4ParquetConverter, ParquetOutputRecordWriter.Float8ParquetConverter, ParquetOutputRecordWriter.IntervalDayParquetConverter, ParquetOutputRecordWriter.IntervalParquetConverter, ParquetOutputRecordWriter.IntervalYearParquetConverter, ParquetOutputRecordWriter.IntParquetConverter, ParquetOutputRecordWriter.NullableFieldConverter, ParquetOutputRecordWriter.RepeatedFieldConverter, ParquetOutputRecordWriter.RequiredFieldConverter, ParquetOutputRecordWriter.SmallIntParquetConverter, ParquetOutputRecordWriter.TimeParquetConverter, ParquetOutputRecordWriter.TimeStampParquetConverter, ParquetOutputRecordWriter.TinyIntParquetConverter, ParquetOutputRecordWriter.UInt1ParquetConverter, ParquetOutputRecordWriter.UInt2ParquetConverter, ParquetOutputRecordWriter.UInt4ParquetConverter, ParquetOutputRecordWriter.UInt8ParquetConverter, ParquetOutputRecordWriter.Var16CharParquetConverter, ParquetOutputRecordWriter.VarBinaryParquetConverter, ParquetOutputRecordWriter.VarCharParquetConverter, ParquetOutputRecordWriter.VarDecimalParquetConverter

Field Summary

Fields

Modifier and Type	Field and Description
static String	DRILL_VERSION_PROPERTY
static String	WRITER_VERSION_PROPERTY

Fields inherited from class org.apache.drill.exec.store.ParquetOutputRecordWriter

ELEMENT, LIST, ZERO_IDX

Constructor Summary

Constructors

Constructor and Description
ParquetRecordWriter(FragmentContext context, ParquetWriter writer)

Method Summary

Modifier and Type	Method and Description
void	abort()
void	checkForNewPartition(int index) Check if the writer should start a new partition, and if so, start a new partition
void	cleanup()
void	endRecord() Called after adding all fields in a particular record are added using add{TypeHolder}(fieldId, TypeHolder) methods.
EventBasedRecordWriter.FieldConverter	getNewDictConverter(int fieldId, String fieldName, FieldReader reader)
EventBasedRecordWriter.FieldConverter	getNewMapConverter(int fieldId, String fieldName, FieldReader reader) Add the field value given in valueHolder at the given column number fieldId.
EventBasedRecordWriter.FieldConverter	getNewRepeatedDictConverter(int fieldId, String fieldName, FieldReader reader)
EventBasedRecordWriter.FieldConverter	getNewRepeatedListConverter(int fieldId, String fieldName, FieldReader reader)
EventBasedRecordWriter.FieldConverter	getNewRepeatedMapConverter(int fieldId, String fieldName, FieldReader reader)
protected org.apache.parquet.schema.PrimitiveType	getPrimitiveType(MaterializedField field)
void	init(Map<String,String> writerOptions) Initialize the writer.
void	startRecord() Called before starting writing fields in a record.
boolean	supportsField(MaterializedField field) Checks whether this writer supports writing of the given field.
void	updateSchema(VectorAccessible batch) Update the schema in RecordWriter.

Methods inherited from class org.apache.drill.exec.store.ParquetOutputRecordWriter

getNewBigIntConverter, getNewBitConverter, getNewDateConverter, getNewDecimal18Converter, getNewDecimal28DenseConverter, getNewDecimal28SparseConverter, getNewDecimal38DenseConverter, getNewDecimal38SparseConverter, getNewDecimal9Converter, getNewFloat4Converter, getNewFloat8Converter, getNewIntConverter, getNewIntervalConverter, getNewIntervalDayConverter, getNewIntervalYearConverter, getNewNullableBigIntConverter, getNewNullableBitConverter, getNewNullableDateConverter, getNewNullableDecimal18Converter, getNewNullableDecimal28DenseConverter, getNewNullableDecimal28SparseConverter, getNewNullableDecimal38DenseConverter, getNewNullableDecimal38SparseConverter, getNewNullableDecimal9Converter, getNewNullableFloat4Converter, getNewNullableFloat8Converter, getNewNullableIntConverter, getNewNullableIntervalConverter, getNewNullableIntervalDayConverter, getNewNullableIntervalYearConverter, getNewNullableSmallIntConverter, getNewNullableTimeConverter, getNewNullableTimeStampConverter, getNewNullableTinyIntConverter, getNewNullableUInt1Converter, getNewNullableUInt2Converter, getNewNullableUInt4Converter, getNewNullableUInt8Converter, getNewNullableVar16CharConverter, getNewNullableVarBinaryConverter, getNewNullableVarCharConverter, getNewNullableVarDecimalConverter, getNewRepeatedBigIntConverter, getNewRepeatedBitConverter, getNewRepeatedDateConverter, getNewRepeatedDecimal18Converter, getNewRepeatedDecimal28DenseConverter, getNewRepeatedDecimal28SparseConverter, getNewRepeatedDecimal38DenseConverter, getNewRepeatedDecimal38SparseConverter, getNewRepeatedDecimal9Converter, getNewRepeatedFloat4Converter, getNewRepeatedFloat8Converter, getNewRepeatedIntConverter, getNewRepeatedIntervalConverter, getNewRepeatedIntervalDayConverter, getNewRepeatedIntervalYearConverter, getNewRepeatedSmallIntConverter, getNewRepeatedTimeConverter, getNewRepeatedTimeStampConverter, getNewRepeatedTinyIntConverter, getNewRepeatedUInt1Converter, getNewRepeatedUInt2Converter, getNewRepeatedUInt4Converter, getNewRepeatedUInt8Converter, getNewRepeatedVar16CharConverter, getNewRepeatedVarBinaryConverter, getNewRepeatedVarCharConverter, getNewRepeatedVarDecimalConverter, getNewSmallIntConverter, getNewTimeConverter, getNewTimeStampConverter, getNewTinyIntConverter, getNewUInt1Converter, getNewUInt2Converter, getNewUInt4Converter, getNewUInt8Converter, getNewVar16CharConverter, getNewVarBinaryConverter, getNewVarCharConverter, getNewVarDecimalConverter, setUp

Methods inherited from class org.apache.drill.exec.store.AbstractRecordWriter

getNewUnionConverter, newPartition, postProcessing, setPartitionVector

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.drill.exec.store.RecordWriter

getNewUnionConverter, postProcessing

Field Detail

DRILL_VERSION_PROPERTY

public static final String DRILL_VERSION_PROPERTY

See Also:

Constant Field Values

WRITER_VERSION_PROPERTY

public static final String WRITER_VERSION_PROPERTY

See Also:

Constant Field Values

Constructor Detail

ParquetRecordWriter

public ParquetRecordWriter(FragmentContext context,
                           ParquetWriter writer)
                    throws OutOfMemoryException

Throws:

OutOfMemoryException

Method Detail

init

public void init(Map<String,String> writerOptions)
          throws IOException

Description copied from interface: RecordWriter

Initialize the writer.

Parameters:

writerOptions - Contains key, value pair of settings.

Throws:

IOException

updateSchema

public void updateSchema(VectorAccessible batch)
                  throws IOException

Description copied from interface: RecordWriter

Update the schema in RecordWriter. Called at least once before starting writing the records.

Throws:

IOException

supportsField

public boolean supportsField(MaterializedField field)

Description copied from interface: RecordWriter

Checks whether this writer supports writing of the given field.

Specified by:

supportsField in interface RecordWriter

Overrides:

supportsField in class AbstractRecordWriter

getPrimitiveType

protected org.apache.parquet.schema.PrimitiveType getPrimitiveType(MaterializedField field)

Specified by:

getPrimitiveType in class ParquetOutputRecordWriter

checkForNewPartition

public void checkForNewPartition(int index)

Description copied from interface: RecordWriter

Check if the writer should start a new partition, and if so, start a new partition

Specified by:

checkForNewPartition in interface RecordWriter

Overrides:

checkForNewPartition in class AbstractRecordWriter

getNewMapConverter

public EventBasedRecordWriter.FieldConverter getNewMapConverter(int fieldId,
                                                                String fieldName,
                                                                FieldReader reader)

Description copied from interface: RecordWriter

Add the field value given in valueHolder at the given column number fieldId.

Specified by:

getNewMapConverter in interface RecordWriter

Overrides:

getNewMapConverter in class AbstractRecordWriter

getNewRepeatedMapConverter

public EventBasedRecordWriter.FieldConverter getNewRepeatedMapConverter(int fieldId,
                                                                        String fieldName,
                                                                        FieldReader reader)

Specified by:

getNewRepeatedMapConverter in interface RecordWriter

Overrides:

getNewRepeatedMapConverter in class AbstractRecordWriter

getNewRepeatedListConverter

public EventBasedRecordWriter.FieldConverter getNewRepeatedListConverter(int fieldId,
                                                                         String fieldName,
                                                                         FieldReader reader)

Specified by:

getNewRepeatedListConverter in interface RecordWriter

Overrides:

getNewRepeatedListConverter in class AbstractRecordWriter

getNewDictConverter

public EventBasedRecordWriter.FieldConverter getNewDictConverter(int fieldId,
                                                                 String fieldName,
                                                                 FieldReader reader)

Specified by:

getNewDictConverter in interface RecordWriter

Overrides:

getNewDictConverter in class AbstractRecordWriter

getNewRepeatedDictConverter

public EventBasedRecordWriter.FieldConverter getNewRepeatedDictConverter(int fieldId,
                                                                         String fieldName,
                                                                         FieldReader reader)

Specified by:

getNewRepeatedDictConverter in interface RecordWriter

Overrides:

getNewRepeatedDictConverter in class AbstractRecordWriter

startRecord

public void startRecord()
                 throws IOException

Description copied from interface: RecordWriter

Called before starting writing fields in a record.

Throws:

IOException

endRecord

public void endRecord()
               throws IOException

Description copied from interface: RecordWriter

Called after adding all fields in a particular record are added using add{TypeHolder}(fieldId, TypeHolder) methods.

Throws:

IOException

abort

public void abort()
           throws IOException

Throws:

IOException

cleanup

public void cleanup()
             throws IOException

Throws:

IOException