Package org.apache.drill.exec.store.kafka

Class KafkaRecordReader

java.lang.Object

org.apache.drill.exec.store.kafka.KafkaRecordReader

All Implemented Interfaces:

ManagedReader<SchemaNegotiator>

public class KafkaRecordReader
extends Object
implements ManagedReader<SchemaNegotiator>

Constructor Summary

Constructors

Constructor	Description
KafkaRecordReader(KafkaPartitionScanSpec subScanSpec, OptionManager options, KafkaStoragePlugin plugin, int maxRecords)

Method Summary

Modifier and Type	Method	Description
void	close()	Release resources.
boolean	next()	KafkaConsumer.poll will fetch 500 messages per poll call.
boolean	open(SchemaNegotiator negotiator)	Setup the record reader.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

KafkaRecordReader

public KafkaRecordReader(KafkaPartitionScanSpec subScanSpec,
 OptionManager options,
 KafkaStoragePlugin plugin,
 int maxRecords)

Method Details

open

public boolean open(SchemaNegotiator negotiator)

Description copied from interface: ManagedReader

Setup the record reader. Called just before the first call to next(). Allocate resources here, not in the constructor. Example: open files, allocate buffers, etc.

Specified by:

open in interface ManagedReader<SchemaNegotiator>

Parameters:

negotiator - mechanism to negotiate select and table schemas, then create the row set reader used to load data into value vectors

Returns:

true if the reader is open and ready to read (possibly no) rows. false for a "soft" failure in which no schema or data is available, but the scanner should not fail, it should move onto another reader

next

public boolean next()

KafkaConsumer.poll will fetch 500 messages per poll call. So hasNext will take care of polling multiple times for this given batch next invocation

Specified by:

next in interface ManagedReader<SchemaNegotiator>

Returns:

true if more data may be available (and so next() should be called again, false to indicate that EOF was reached

close

public void close()

Description copied from interface: ManagedReader

Release resources. Called just after a failure, when the scanner is cancelled, or after next() returns EOF. Release all resources and close files. Guaranteed to be called if open() returns normally; will not be called if open() throws an exception.

Specified by:

close in interface ManagedReader<SchemaNegotiator>

toString

public String toString()

Overrides:

toString in class Object