Package org.apache.drill.exec.store.http.paginator

Class Paginator

java.lang.Object

org.apache.drill.exec.store.http.paginator.Paginator

All Implemented Interfaces:

Iterator<String>

Direct Known Subclasses:

IndexPaginator, OffsetPaginator, PagePaginator

public abstract class Paginator
extends Object
implements Iterator<String>

This class is the abstraction for the Paginator class. There are different pagination methods, however, all essentially require the query engine to generate URLs to retrieve the next batch of data and also to determine whether the URL has more data. The Offset and Page paginators work either with a limit or without, but function slightly differently. If a limit is specified and pushed down, the paginator will generate a list of URLs with the appropriate pagination parameters. In the future this could be parallelized, however in the V1 all these requests are run in series.

Field Summary

Fields

Modifier and Type	Field	Description
protected okhttp3.HttpUrl.Builder	builder
protected final int	limit
protected final HttpPaginatorConfig.PaginatorMethod	MODE
protected final int	pageSize
protected boolean	partialPageReceived

Constructor Summary

Constructors

Constructor	Description
Paginator(okhttp3.HttpUrl.Builder builder, HttpPaginatorConfig.PaginatorMethod mode, int pageSize, int limit)

Method Summary

Modifier and Type	Method	Description
HttpPaginatorConfig.PaginatorMethod	getMode()
int	getPageSize()
void	notifyPartialPage()	This method is used in pagination queries when no limit is present.
void	setBuilder(okhttp3.HttpUrl.Builder builder)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining, hasNext, next, remove

Field Details

pageSize

protected final int pageSize

MODE

protected final HttpPaginatorConfig.PaginatorMethod MODE

limit

protected final int limit

partialPageReceived

protected boolean partialPageReceived

builder

protected okhttp3.HttpUrl.Builder builder

Constructor Details

Paginator

public Paginator(okhttp3.HttpUrl.Builder builder,
 HttpPaginatorConfig.PaginatorMethod mode,
 int pageSize,
 int limit)

Method Details

setBuilder

public void setBuilder(okhttp3.HttpUrl.Builder builder)

notifyPartialPage

public void notifyPartialPage()

This method is used in pagination queries when no limit is present. The intended flow is that if no limit is present, if the batch reader encounters a page which has less data than the page size, the batch reader should call this method to stop the Paginator from generating additional URLs to call. In the event that the API simply runs out of data, the reader will return false anyway and the pagination will stop.

getPageSize

public int getPageSize()

getMode

public HttpPaginatorConfig.PaginatorMethod getMode()