org.apache.drill.exec.store.easy.json

Class JsonStatisticsRecordWriter

java.lang.Object

org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter

org.apache.drill.exec.store.easy.json.JsonStatisticsRecordWriter

All Implemented Interfaces:

StatisticsRecordCollector, StatisticsRecordWriter

public class JsonStatisticsRecordWriter
extends JSONBaseStatisticsRecordWriter
implements StatisticsRecordWriter

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter

JSONBaseStatisticsRecordWriter.BigIntJsonConverter, JSONBaseStatisticsRecordWriter.BitJsonConverter, JSONBaseStatisticsRecordWriter.DateJsonConverter, JSONBaseStatisticsRecordWriter.Decimal18JsonConverter, JSONBaseStatisticsRecordWriter.Decimal28DenseJsonConverter, JSONBaseStatisticsRecordWriter.Decimal28SparseJsonConverter, JSONBaseStatisticsRecordWriter.Decimal38DenseJsonConverter, JSONBaseStatisticsRecordWriter.Decimal38SparseJsonConverter, JSONBaseStatisticsRecordWriter.Decimal9JsonConverter, JSONBaseStatisticsRecordWriter.Float4JsonConverter, JSONBaseStatisticsRecordWriter.Float8JsonConverter, JSONBaseStatisticsRecordWriter.IntervalDayJsonConverter, JSONBaseStatisticsRecordWriter.IntervalJsonConverter, JSONBaseStatisticsRecordWriter.IntervalYearJsonConverter, JSONBaseStatisticsRecordWriter.IntJsonConverter, JSONBaseStatisticsRecordWriter.NullableBigIntJsonConverter, JSONBaseStatisticsRecordWriter.NullableBitJsonConverter, JSONBaseStatisticsRecordWriter.NullableDateJsonConverter, JSONBaseStatisticsRecordWriter.NullableDecimal18JsonConverter, JSONBaseStatisticsRecordWriter.NullableDecimal28DenseJsonConverter, JSONBaseStatisticsRecordWriter.NullableDecimal28SparseJsonConverter, JSONBaseStatisticsRecordWriter.NullableDecimal38DenseJsonConverter, JSONBaseStatisticsRecordWriter.NullableDecimal38SparseJsonConverter, JSONBaseStatisticsRecordWriter.NullableDecimal9JsonConverter, JSONBaseStatisticsRecordWriter.NullableFloat4JsonConverter, JSONBaseStatisticsRecordWriter.NullableFloat8JsonConverter, JSONBaseStatisticsRecordWriter.NullableIntervalDayJsonConverter, JSONBaseStatisticsRecordWriter.NullableIntervalJsonConverter, JSONBaseStatisticsRecordWriter.NullableIntervalYearJsonConverter, JSONBaseStatisticsRecordWriter.NullableIntJsonConverter, JSONBaseStatisticsRecordWriter.NullableSmallIntJsonConverter, JSONBaseStatisticsRecordWriter.NullableTimeJsonConverter, JSONBaseStatisticsRecordWriter.NullableTimeStampJsonConverter, JSONBaseStatisticsRecordWriter.NullableTinyIntJsonConverter, JSONBaseStatisticsRecordWriter.NullableUInt1JsonConverter, JSONBaseStatisticsRecordWriter.NullableUInt2JsonConverter, JSONBaseStatisticsRecordWriter.NullableUInt4JsonConverter, JSONBaseStatisticsRecordWriter.NullableUInt8JsonConverter, JSONBaseStatisticsRecordWriter.NullableVar16CharJsonConverter, JSONBaseStatisticsRecordWriter.NullableVarBinaryJsonConverter, JSONBaseStatisticsRecordWriter.NullableVarCharJsonConverter, JSONBaseStatisticsRecordWriter.NullableVarDecimalJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedBigIntJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedBitJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedDateJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedDecimal18JsonConverter, JSONBaseStatisticsRecordWriter.RepeatedDecimal28DenseJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedDecimal28SparseJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedDecimal38DenseJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedDecimal38SparseJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedDecimal9JsonConverter, JSONBaseStatisticsRecordWriter.RepeatedFloat4JsonConverter, JSONBaseStatisticsRecordWriter.RepeatedFloat8JsonConverter, JSONBaseStatisticsRecordWriter.RepeatedIntervalDayJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedIntervalJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedIntervalYearJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedIntJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedSmallIntJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedTimeJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedTimeStampJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedTinyIntJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedUInt1JsonConverter, JSONBaseStatisticsRecordWriter.RepeatedUInt2JsonConverter, JSONBaseStatisticsRecordWriter.RepeatedUInt4JsonConverter, JSONBaseStatisticsRecordWriter.RepeatedUInt8JsonConverter, JSONBaseStatisticsRecordWriter.RepeatedVar16CharJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedVarBinaryJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedVarCharJsonConverter, JSONBaseStatisticsRecordWriter.RepeatedVarDecimalJsonConverter, JSONBaseStatisticsRecordWriter.SmallIntJsonConverter, JSONBaseStatisticsRecordWriter.TimeJsonConverter, JSONBaseStatisticsRecordWriter.TimeStampJsonConverter, JSONBaseStatisticsRecordWriter.TinyIntJsonConverter, JSONBaseStatisticsRecordWriter.UInt1JsonConverter, JSONBaseStatisticsRecordWriter.UInt2JsonConverter, JSONBaseStatisticsRecordWriter.UInt4JsonConverter, JSONBaseStatisticsRecordWriter.UInt8JsonConverter, JSONBaseStatisticsRecordWriter.Var16CharJsonConverter, JSONBaseStatisticsRecordWriter.VarBinaryJsonConverter, JSONBaseStatisticsRecordWriter.VarCharJsonConverter, JSONBaseStatisticsRecordWriter.VarDecimalJsonConverter

Field Summary

Fields inherited from class org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter

gen, skipNullFields

Constructor Summary

Constructors

Constructor and Description
JsonStatisticsRecordWriter(org.apache.hadoop.conf.Configuration fsConf, FormatPlugin formatPlugin)

Method Summary

Modifier and Type	Method and Description
void	abort()
void	checkForNewPartition(int index) Check if the writer should start a new partition, and if so, start a new partition
void	cleanup()
void	endStatisticsRecord() Called after adding all fields in a particular statistics record are added using add{TypeHolder}(fieldId, TypeHolder) methods.
void	flushBlockingWriter() For a blocking writer, called after processing all the records to flush out the writes
EventBasedRecordWriter.FieldConverter	getNewBigIntConverter(int fieldId, String fieldName, FieldReader reader) Add the field value given in valueHolder at the given column number fieldId.
EventBasedRecordWriter.FieldConverter	getNewDateConverter(int fieldId, String fieldName, FieldReader reader) Add the field value given in valueHolder at the given column number fieldId.
EventBasedRecordWriter.FieldConverter	getNewIntConverter(int fieldId, String fieldName, FieldReader reader) Add the field value given in valueHolder at the given column number fieldId.
EventBasedRecordWriter.FieldConverter	getNewNullableBigIntConverter(int fieldId, String fieldName, FieldReader reader) Add the field value given in valueHolder at the given column number fieldId.
EventBasedRecordWriter.FieldConverter	getNewNullableFloat8Converter(int fieldId, String fieldName, FieldReader reader) Add the field value given in valueHolder at the given column number fieldId.
EventBasedRecordWriter.FieldConverter	getNewNullableVarBinaryConverter(int fieldId, String fieldName, FieldReader reader) Add the field value given in valueHolder at the given column number fieldId.
EventBasedRecordWriter.FieldConverter	getNewVarCharConverter(int fieldId, String fieldName, FieldReader reader) Add the field value given in valueHolder at the given column number fieldId.
DrillStatsTable.TableStatistics	getStatistics() Returns DrillStatsTable.TableStatistics instance with statistics collected using this StatisticsRecordCollector.
boolean	hasStatistics() Returns true if this StatisticsRecordCollector has non-empty statistics.
void	init(Map<String,String> writerOptions) Initialize the writer.
boolean	isBlockingWriter() Returns if the writer is a blocking writer i.e.
void	startStatisticsRecord() Called before starting writing fields in a record.
void	updateSchema(VectorAccessible batch) Update the schema in RecordWriter.

Methods inherited from class org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter

getNewBitConverter, getNewDecimal18Converter, getNewDecimal28DenseConverter, getNewDecimal28SparseConverter, getNewDecimal38DenseConverter, getNewDecimal38SparseConverter, getNewDecimal9Converter, getNewFloat4Converter, getNewFloat8Converter, getNewIntervalConverter, getNewIntervalDayConverter, getNewIntervalYearConverter, getNewNullableBitConverter, getNewNullableDateConverter, getNewNullableDecimal18Converter, getNewNullableDecimal28DenseConverter, getNewNullableDecimal28SparseConverter, getNewNullableDecimal38DenseConverter, getNewNullableDecimal38SparseConverter, getNewNullableDecimal9Converter, getNewNullableFloat4Converter, getNewNullableIntConverter, getNewNullableIntervalConverter, getNewNullableIntervalDayConverter, getNewNullableIntervalYearConverter, getNewNullableSmallIntConverter, getNewNullableTimeConverter, getNewNullableTimeStampConverter, getNewNullableTinyIntConverter, getNewNullableUInt1Converter, getNewNullableUInt2Converter, getNewNullableUInt4Converter, getNewNullableUInt8Converter, getNewNullableVar16CharConverter, getNewNullableVarCharConverter, getNewNullableVarDecimalConverter, getNewRepeatedBigIntConverter, getNewRepeatedBitConverter, getNewRepeatedDateConverter, getNewRepeatedDecimal18Converter, getNewRepeatedDecimal28DenseConverter, getNewRepeatedDecimal28SparseConverter, getNewRepeatedDecimal38DenseConverter, getNewRepeatedDecimal38SparseConverter, getNewRepeatedDecimal9Converter, getNewRepeatedFloat4Converter, getNewRepeatedFloat8Converter, getNewRepeatedIntConverter, getNewRepeatedIntervalConverter, getNewRepeatedIntervalDayConverter, getNewRepeatedIntervalYearConverter, getNewRepeatedSmallIntConverter, getNewRepeatedTimeConverter, getNewRepeatedTimeStampConverter, getNewRepeatedTinyIntConverter, getNewRepeatedUInt1Converter, getNewRepeatedUInt2Converter, getNewRepeatedUInt4Converter, getNewRepeatedUInt8Converter, getNewRepeatedVar16CharConverter, getNewRepeatedVarBinaryConverter, getNewRepeatedVarCharConverter, getNewRepeatedVarDecimalConverter, getNewSmallIntConverter, getNewTimeConverter, getNewTimeStampConverter, getNewTinyIntConverter, getNewUInt1Converter, getNewUInt2Converter, getNewUInt4Converter, getNewUInt8Converter, getNewVar16CharConverter, getNewVarBinaryConverter, getNewVarDecimalConverter

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.drill.exec.store.StatisticsRecordCollector

getNewBitConverter, getNewDecimal18Converter, getNewDecimal28DenseConverter, getNewDecimal28SparseConverter, getNewDecimal38DenseConverter, getNewDecimal38SparseConverter, getNewDecimal9Converter, getNewFloat4Converter, getNewFloat8Converter, getNewIntervalConverter, getNewIntervalDayConverter, getNewIntervalYearConverter, getNewNullableBitConverter, getNewNullableDateConverter, getNewNullableDecimal18Converter, getNewNullableDecimal28DenseConverter, getNewNullableDecimal28SparseConverter, getNewNullableDecimal38DenseConverter, getNewNullableDecimal38SparseConverter, getNewNullableDecimal9Converter, getNewNullableFloat4Converter, getNewNullableIntConverter, getNewNullableIntervalConverter, getNewNullableIntervalDayConverter, getNewNullableIntervalYearConverter, getNewNullableSmallIntConverter, getNewNullableTimeConverter, getNewNullableTimeStampConverter, getNewNullableTinyIntConverter, getNewNullableUInt1Converter, getNewNullableUInt2Converter, getNewNullableUInt4Converter, getNewNullableUInt8Converter, getNewNullableVar16CharConverter, getNewNullableVarCharConverter, getNewNullableVarDecimalConverter, getNewRepeatedBigIntConverter, getNewRepeatedBitConverter, getNewRepeatedDateConverter, getNewRepeatedDecimal18Converter, getNewRepeatedDecimal28DenseConverter, getNewRepeatedDecimal28SparseConverter, getNewRepeatedDecimal38DenseConverter, getNewRepeatedDecimal38SparseConverter, getNewRepeatedDecimal9Converter, getNewRepeatedFloat4Converter, getNewRepeatedFloat8Converter, getNewRepeatedIntConverter, getNewRepeatedIntervalConverter, getNewRepeatedIntervalDayConverter, getNewRepeatedIntervalYearConverter, getNewRepeatedSmallIntConverter, getNewRepeatedTimeConverter, getNewRepeatedTimeStampConverter, getNewRepeatedTinyIntConverter, getNewRepeatedUInt1Converter, getNewRepeatedUInt2Converter, getNewRepeatedUInt4Converter, getNewRepeatedUInt8Converter, getNewRepeatedVar16CharConverter, getNewRepeatedVarBinaryConverter, getNewRepeatedVarCharConverter, getNewRepeatedVarDecimalConverter, getNewSmallIntConverter, getNewTimeConverter, getNewTimeStampConverter, getNewTinyIntConverter, getNewUInt1Converter, getNewUInt2Converter, getNewUInt4Converter, getNewUInt8Converter, getNewVar16CharConverter, getNewVarBinaryConverter, getNewVarDecimalConverter

Constructor Detail

JsonStatisticsRecordWriter

public JsonStatisticsRecordWriter(org.apache.hadoop.conf.Configuration fsConf,
                                  FormatPlugin formatPlugin)

Method Detail

init

public void init(Map<String,String> writerOptions)

Description copied from interface: StatisticsRecordWriter

Initialize the writer.

Specified by:

init in interface StatisticsRecordWriter

Parameters:

writerOptions - Contains key, value pair of settings.

updateSchema

public void updateSchema(VectorAccessible batch)

Description copied from interface: StatisticsRecordWriter

Update the schema in RecordWriter. Called at least once before starting writing the records.

Specified by:

updateSchema in interface StatisticsRecordWriter

isBlockingWriter

public boolean isBlockingWriter()

Description copied from interface: StatisticsRecordWriter

Returns if the writer is a blocking writer i.e. consumes all input before writing it out

Specified by:

isBlockingWriter in interface StatisticsRecordWriter

Returns:

TRUE, if writer is blocking. FALSE, otherwise

checkForNewPartition

public void checkForNewPartition(int index)

Description copied from interface: StatisticsRecordWriter

Check if the writer should start a new partition, and if so, start a new partition

Specified by:

checkForNewPartition in interface StatisticsRecordWriter

getNewBigIntConverter

public EventBasedRecordWriter.FieldConverter getNewBigIntConverter(int fieldId,
                                                                   String fieldName,
                                                                   FieldReader reader)

Description copied from interface: StatisticsRecordCollector

Add the field value given in valueHolder at the given column number fieldId.

Specified by:

getNewBigIntConverter in interface StatisticsRecordCollector

Overrides:

getNewBigIntConverter in class JSONBaseStatisticsRecordWriter

getNewIntConverter

public EventBasedRecordWriter.FieldConverter getNewIntConverter(int fieldId,
                                                                String fieldName,
                                                                FieldReader reader)

Description copied from interface: StatisticsRecordCollector

Add the field value given in valueHolder at the given column number fieldId.

Specified by:

getNewIntConverter in interface StatisticsRecordCollector

Overrides:

getNewIntConverter in class JSONBaseStatisticsRecordWriter

getNewDateConverter

public EventBasedRecordWriter.FieldConverter getNewDateConverter(int fieldId,
                                                                 String fieldName,
                                                                 FieldReader reader)

Description copied from interface: StatisticsRecordCollector

Add the field value given in valueHolder at the given column number fieldId.

Specified by:

getNewDateConverter in interface StatisticsRecordCollector

Overrides:

getNewDateConverter in class JSONBaseStatisticsRecordWriter

getNewVarCharConverter

public EventBasedRecordWriter.FieldConverter getNewVarCharConverter(int fieldId,
                                                                    String fieldName,
                                                                    FieldReader reader)

Description copied from interface: StatisticsRecordCollector

Add the field value given in valueHolder at the given column number fieldId.

Specified by:

getNewVarCharConverter in interface StatisticsRecordCollector

Overrides:

getNewVarCharConverter in class JSONBaseStatisticsRecordWriter

getNewNullableBigIntConverter

public EventBasedRecordWriter.FieldConverter getNewNullableBigIntConverter(int fieldId,
                                                                           String fieldName,
                                                                           FieldReader reader)

Description copied from interface: StatisticsRecordCollector

Add the field value given in valueHolder at the given column number fieldId.

Specified by:

getNewNullableBigIntConverter in interface StatisticsRecordCollector

Overrides:

getNewNullableBigIntConverter in class JSONBaseStatisticsRecordWriter

getNewNullableVarBinaryConverter

public EventBasedRecordWriter.FieldConverter getNewNullableVarBinaryConverter(int fieldId,
                                                                              String fieldName,
                                                                              FieldReader reader)

Description copied from interface: StatisticsRecordCollector

Add the field value given in valueHolder at the given column number fieldId.

Specified by:

getNewNullableVarBinaryConverter in interface StatisticsRecordCollector

Overrides:

getNewNullableVarBinaryConverter in class JSONBaseStatisticsRecordWriter

getNewNullableFloat8Converter

public EventBasedRecordWriter.FieldConverter getNewNullableFloat8Converter(int fieldId,
                                                                           String fieldName,
                                                                           FieldReader reader)

Description copied from interface: StatisticsRecordCollector

Add the field value given in valueHolder at the given column number fieldId.

Specified by:

getNewNullableFloat8Converter in interface StatisticsRecordCollector

Overrides:

getNewNullableFloat8Converter in class JSONBaseStatisticsRecordWriter

startStatisticsRecord

public void startStatisticsRecord()

Description copied from interface: StatisticsRecordCollector

Called before starting writing fields in a record.

Specified by:

startStatisticsRecord in interface StatisticsRecordCollector

endStatisticsRecord

public void endStatisticsRecord()

Description copied from interface: StatisticsRecordCollector

Called after adding all fields in a particular statistics record are added using add{TypeHolder}(fieldId, TypeHolder) methods.

Specified by:

endStatisticsRecord in interface StatisticsRecordCollector

hasStatistics

public boolean hasStatistics()

Description copied from interface: StatisticsRecordCollector

Returns true if this StatisticsRecordCollector has non-empty statistics.

Specified by:

hasStatistics in interface StatisticsRecordCollector

Returns:

if this StatisticsRecordCollector has non-empty statistics

getStatistics

public DrillStatsTable.TableStatistics getStatistics()

Description copied from interface: StatisticsRecordCollector

Returns DrillStatsTable.TableStatistics instance with statistics collected using this StatisticsRecordCollector.

Specified by:

getStatistics in interface StatisticsRecordCollector

Returns:

DrillStatsTable.TableStatistics instance

flushBlockingWriter

public void flushBlockingWriter()
                         throws IOException

Description copied from interface: StatisticsRecordWriter

For a blocking writer, called after processing all the records to flush out the writes

Specified by:

flushBlockingWriter in interface StatisticsRecordWriter

Throws:

IOException

abort

public void abort()

Specified by:

abort in interface StatisticsRecordWriter

cleanup

public void cleanup()

Specified by:

cleanup in interface StatisticsRecordWriter