Class FileScanFramework

All Implemented Interfaces:
Direct Known Subclasses:

public class FileScanFramework extends ManagedScanFramework
The file scan framework adds into the scan framework support for implicit reading from DFS splits (a file and a block). Since this framework is file-based, it also adds support for file metadata (AKA implicit columns. The file scan framework brings together a number of components:
  • The set of options defined by the base framework.
  • The set of files and/or blocks to read.
  • The file system configuration to use for working with the files or blocks.
  • The factory class to create a reader for each of the files or blocks defined above. (Readers are created one-by-one as files are read.)
  • Options as defined by the base class.

The framework iterates over file descriptions, creating readers at the moment they are needed. This allows simpler logic because, at the point of reader creation, we have a file system, context and so on.