org.apache.drill.exec.store.http

Class HttpBatchReader

java.lang.Object

org.apache.drill.exec.store.http.HttpBatchReader

All Implemented Interfaces:

ManagedReader<SchemaNegotiator>

Direct Known Subclasses:

HttpCSVBatchReader, HttpXMLBatchReader

public class HttpBatchReader
extends Object
implements ManagedReader<SchemaNegotiator>

Field Summary

Fields

Modifier and Type	Field and Description
protected String	baseUrl
protected ImplicitColumnUtils.ImplicitColumns	implicitColumns
protected Paginator	paginator

Constructor Summary

Constructors

Constructor and Description
HttpBatchReader(HttpSubScan subScan)
HttpBatchReader(HttpSubScan subScan, Paginator paginator)

Method Summary

Modifier and Type	Method and Description
protected void	addFilters(okhttp3.HttpUrl.Builder urlBuilder, List<String> params, Map<String,String> filters) Convert equality filter conditions into HTTP query parameters Parameters must appear in the order defined in the config.
protected void	buildImplicitColumns()
protected okhttp3.HttpUrl	buildUrl()
void	close() Release resources.
protected boolean	implicitColumnsAreProjected()
boolean	next() Read the next batch.
boolean	open(SchemaNegotiator negotiator) Setup the record reader.
protected void	populateImplicitFieldMap(SimpleHttp http)
protected HttpProxyConfig	proxySettings(com.typesafe.config.Config drillConfig, okhttp3.HttpUrl url)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

paginator

protected final Paginator paginator

baseUrl

protected String baseUrl

implicitColumns

protected ImplicitColumnUtils.ImplicitColumns implicitColumns

Constructor Detail

HttpBatchReader

public HttpBatchReader(HttpSubScan subScan)

HttpBatchReader

public HttpBatchReader(HttpSubScan subScan,
                       Paginator paginator)

Method Detail

open

public boolean open(SchemaNegotiator negotiator)

Description copied from interface: ManagedReader

Setup the record reader. Called just before the first call to next(). Allocate resources here, not in the constructor. Example: open files, allocate buffers, etc.

Specified by:

open in interface ManagedReader<SchemaNegotiator>

Parameters:

negotiator - mechanism to negotiate select and table schemas, then create the row set reader used to load data into value vectors

Returns:

true if the reader is open and ready to read (possibly no) rows. false for a "soft" failure in which no schema or data is available, but the scanner should not fail, it should move onto another reader

buildImplicitColumns

protected void buildImplicitColumns()

populateImplicitFieldMap

protected void populateImplicitFieldMap(SimpleHttp http)

implicitColumnsAreProjected

protected boolean implicitColumnsAreProjected()

buildUrl

protected okhttp3.HttpUrl buildUrl()

addFilters

protected void addFilters(okhttp3.HttpUrl.Builder urlBuilder,
                          List<String> params,
                          Map<String,String> filters)

Convert equality filter conditions into HTTP query parameters Parameters must appear in the order defined in the config.

proxySettings

protected HttpProxyConfig proxySettings(com.typesafe.config.Config drillConfig,
                                        okhttp3.HttpUrl url)

next

public boolean next()

Description copied from interface: ManagedReader

Read the next batch. Reading continues until either EOF, or until the mutator indicates that the batch is full. The batch is considered valid if it is non-empty. Returning true with an empty batch is valid, and is helpful on the very first batch (returning schema only.) An empty batch with a false return code indicates EOF and the batch will be discarded. A non-empty batch along with a false return result indicates a final, valid batch, but that EOF was reached and no more data is available.

This somewhat complex protocol avoids the need to allocate a final batch just to find out that no more data is available; it allows EOF to be returned along with the final batch.

Specified by:

next in interface ManagedReader<SchemaNegotiator>

Returns:

true if more data may be available (and so next() should be called again, false to indicate that EOF was reached

close

public void close()

Description copied from interface: ManagedReader

Release resources. Called just after a failure, when the scanner is cancelled, or after next() returns EOF. Release all resources and close files. Guaranteed to be called if open() returns normally; will not be called if open() throws an exception.

Specified by:

close in interface ManagedReader<SchemaNegotiator>