org.apache.drill.exec.physical.impl.scan.v3.file

Class FileScanLifecycle

java.lang.Object

org.apache.drill.exec.physical.impl.scan.v3.lifecycle.ScanLifecycle

org.apache.drill.exec.physical.impl.scan.v3.file.FileScanLifecycle

public class FileScanLifecycle
extends ScanLifecycle

The file scan framework adds into the scan framework support for reading from DFS splits (a file and a block) and for the file-related implicit and partition columns. The file scan builder gathers file-related options for the scan as a whole, including the list of splits. The associated FileSchemaNegotiator passes file information to each reader.

Only a single storage plugin uses the file scan framework: the FileSystemPlugin via the EasyFormatPlugin. To make client code as simple as possible, the Drill file system and list of files is passed though this framework to the FileReaderFactory, then to the FileSchemaNegotiator which presents them to the reader. This approach avoids the need for each format handle this common boilerplate code.

The FileScanOptions holds the list of splits to scan. The FileReaderFactory iterates over those splits, and creates each reader just-in-time to process that split.

Implicit columns are defined here at the beginning of the scan as part of the scan schema mechanism. Each consists of a column "marker" that identifies the column purposes. Then, on each file, the implicit column is resolved to a value specific to that file. A StaticBatchBuilder then fills in the needed column values for each batch which the reader produces.

Constructor Summary

Constructors

Constructor and Description
FileScanLifecycle(OperatorContext context, FileScanLifecycleBuilder options)

Method Summary

Modifier and Type	Method and Description
void	close()
FileScanLifecycleBuilder	fileScanOptions()
DrillFileSystem	fileSystem()
ImplicitFileColumnsHandler	implicitColumnsHandler()
protected SchemaNegotiatorImpl	newNegotiator(ReaderLifecycle readerLifecycle)

Methods inherited from class org.apache.drill.exec.physical.impl.scan.v3.lifecycle.ScanLifecycle

allocator, batchCount, context, errorContext, hasOutputSchema, nextReader, options, outputSchema, readerFactory, rowCount, schemaTracker, tallyBatch, vectorCache

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FileScanLifecycle

public FileScanLifecycle(OperatorContext context,
                         FileScanLifecycleBuilder options)

Method Detail

fileScanOptions

public FileScanLifecycleBuilder fileScanOptions()

fileSystem

public DrillFileSystem fileSystem()

implicitColumnsHandler

public ImplicitFileColumnsHandler implicitColumnsHandler()

newNegotiator

protected SchemaNegotiatorImpl newNegotiator(ReaderLifecycle readerLifecycle)

Overrides:

newNegotiator in class ScanLifecycle

close

public void close()

Overrides:

close in class ScanLifecycle