org.apache.drill.exec.store.hive.readers

Class HiveTextRecordReader

java.lang.Object

org.apache.drill.exec.store.AbstractRecordReader

org.apache.drill.exec.store.hive.readers.HiveDefaultRecordReader

org.apache.drill.exec.store.hive.readers.HiveTextRecordReader

All Implemented Interfaces:

AutoCloseable, RecordReader

public class HiveTextRecordReader
extends HiveDefaultRecordReader

Reading of Hive table stored in text format may require to skip few header/footer records. This class extends default reader to add the functionality.

Field Summary

Fields inherited from class org.apache.drill.exec.store.hive.readers.HiveDefaultRecordReader

empty, logger, mapredReader, outputWriter, partitionDeserializer, partitionToTableSchemaConverter, TARGET_RECORD_COUNT, valueHolder

Fields inherited from class org.apache.drill.exec.store.AbstractRecordReader

DEFAULT_TEXT_COLS_TO_READ

Fields inherited from interface org.apache.drill.exec.store.RecordReader

ALLOCATOR_INITIAL_RESERVATION, ALLOCATOR_MAX_RESERVATION

Constructor Summary

Constructors

Constructor and Description
HiveTextRecordReader(HiveTableWithColumnCache table, HivePartition partition, Collection<org.apache.hadoop.mapred.InputSplit> inputSplits, List<SchemaPath> projectedColumns, FragmentContext context, org.apache.hadoop.hive.conf.HiveConf hiveConf, org.apache.hadoop.security.UserGroupInformation proxyUgi) Constructor matching super.

Method Summary

Modifier and Type	Method and Description
protected void	internalInit(Properties hiveTableProperties) Default implementation does nothing, used to apply skip header/footer functionality
int	next() Reads batch of records skipping footer rows when necessary.

Methods inherited from class org.apache.drill.exec.store.hive.readers.HiveDefaultRecordReader

close, hasNextValue, readHiveRecordAndInsertIntoRecordBatch, setup

Methods inherited from class org.apache.drill.exec.store.AbstractRecordReader

allocate, getColumns, getDefaultColumnsToRead, hasNext, isSkipQuery, isStarQuery, setColumns, toString, transformColumns

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

HiveTextRecordReader

public HiveTextRecordReader(HiveTableWithColumnCache table,
                            HivePartition partition,
                            Collection<org.apache.hadoop.mapred.InputSplit> inputSplits,
                            List<SchemaPath> projectedColumns,
                            FragmentContext context,
                            org.apache.hadoop.hive.conf.HiveConf hiveConf,
                            org.apache.hadoop.security.UserGroupInformation proxyUgi)

Constructor matching super.

Parameters:

table - metadata about Hive table being read

partition - holder of metadata about table partitioning

inputSplits - input splits for reading data from distributed storage

projectedColumns - target columns for scan

context - fragmentContext of fragment

hiveConf - Hive configuration

proxyUgi - user/group info to be used for initialization

Method Detail

internalInit

protected void internalInit(Properties hiveTableProperties)

Description copied from class: HiveDefaultRecordReader

Default implementation does nothing, used to apply skip header/footer functionality

Overrides:

internalInit in class HiveDefaultRecordReader

Parameters:

hiveTableProperties - hive table properties

next

public int next()

Reads batch of records skipping footer rows when necessary.

Specified by:

next in interface RecordReader

Overrides:

next in class HiveDefaultRecordReader

Returns:

count of read records