Package org.apache.drill.exec.store.dfs
Class DrillFileSystem
java.lang.Object
org.apache.hadoop.conf.Configured
org.apache.hadoop.fs.FileSystem
org.apache.drill.exec.store.dfs.DrillFileSystem
- All Implemented Interfaces:
Closeable,AutoCloseable,OpenFileTracker,org.apache.hadoop.conf.Configurable,org.apache.hadoop.fs.PathCapabilities,org.apache.hadoop.security.token.DelegationTokenIssuer
DrillFileSystem is the wrapper around the actual FileSystem implementation. The
DrillFileSystem is
immutable.
If OperatorStats are provided it returns an instrumented FSDataInputStream to
measure IO wait time and tracking file open/close operations.-
Nested Class Summary
Nested ClassesNested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem
org.apache.hadoop.fs.FileSystem.DirectoryEntries, org.apache.hadoop.fs.FileSystem.DirListingIterator<T extends org.apache.hadoop.fs.FileStatus>, org.apache.hadoop.fs.FileSystem.Statistics -
Field Summary
FieldsFields inherited from class org.apache.hadoop.fs.FileSystem
DEFAULT_FS, FS_DEFAULT_NAME_KEY, LOG, SHUTDOWN_HOOK_PRIORITY, statistics, TRASH_PREFIX, USER_HOME_PREFIX -
Constructor Summary
ConstructorsConstructorDescriptionDrillFileSystem(org.apache.hadoop.conf.Configuration fsConf) DrillFileSystem(org.apache.hadoop.conf.Configuration fsConf, OperatorStats operatorStats) -
Method Summary
Modifier and TypeMethodDescriptionvoidaccess(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.permission.FsAction mode) org.apache.hadoop.security.token.Token<?>[]addDelegationTokens(String renewer, org.apache.hadoop.security.Credentials credentials) org.apache.hadoop.fs.FSDataOutputStreamappend(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.FSDataOutputStreamappend(org.apache.hadoop.fs.Path f, int bufferSize) org.apache.hadoop.fs.FSDataOutputStreamappend(org.apache.hadoop.fs.Path f, int bufferSize, org.apache.hadoop.util.Progressable progress) booleancancelDeleteOnExit(org.apache.hadoop.fs.Path f) voidclose()voidcompleteLocalOutput(org.apache.hadoop.fs.Path fsOutputFile, org.apache.hadoop.fs.Path tmpLocalFile) voidconcat(org.apache.hadoop.fs.Path trg, org.apache.hadoop.fs.Path[] psrcs) voidcopyFromLocalFile(boolean delSrc, boolean overwrite, org.apache.hadoop.fs.Path[] srcs, org.apache.hadoop.fs.Path dst) voidcopyFromLocalFile(boolean delSrc, boolean overwrite, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) voidcopyFromLocalFile(boolean delSrc, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) voidcopyFromLocalFile(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) voidcopyToLocalFile(boolean delSrc, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) voidcopyToLocalFile(boolean delSrc, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst, boolean useRawLocalFileSystem) voidcopyToLocalFile(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) org.apache.hadoop.fs.FSDataOutputStreamcreate(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.FSDataOutputStreamcreate(org.apache.hadoop.fs.Path f, boolean overwrite) org.apache.hadoop.fs.FSDataOutputStreamcreate(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize) org.apache.hadoop.fs.FSDataOutputStreamcreate(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, short replication, long blockSize) org.apache.hadoop.fs.FSDataOutputStreamcreate(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) org.apache.hadoop.fs.FSDataOutputStreamcreate(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, org.apache.hadoop.util.Progressable progress) org.apache.hadoop.fs.FSDataOutputStreamcreate(org.apache.hadoop.fs.Path f, short replication) org.apache.hadoop.fs.FSDataOutputStreamcreate(org.apache.hadoop.fs.Path f, short replication, org.apache.hadoop.util.Progressable progress) org.apache.hadoop.fs.FSDataOutputStreamcreate(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) org.apache.hadoop.fs.FSDataOutputStreamcreate(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, EnumSet<org.apache.hadoop.fs.CreateFlag> flags, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) org.apache.hadoop.fs.FSDataOutputStreamcreate(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, EnumSet<org.apache.hadoop.fs.CreateFlag> flags, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress, org.apache.hadoop.fs.Options.ChecksumOpt checksumOpt) org.apache.hadoop.fs.FSDataOutputStreamcreate(org.apache.hadoop.fs.Path f, org.apache.hadoop.util.Progressable progress) booleancreateNewFile(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.FSDataOutputStreamcreateNonRecursive(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) Deprecated.org.apache.hadoop.fs.FSDataOutputStreamcreateNonRecursive(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) Deprecated.org.apache.hadoop.fs.FSDataOutputStreamcreateNonRecursive(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, EnumSet<org.apache.hadoop.fs.CreateFlag> flags, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) Deprecated.org.apache.hadoop.fs.PathcreateSnapshot(org.apache.hadoop.fs.Path path, String snapshotName) voidcreateSymlink(org.apache.hadoop.fs.Path target, org.apache.hadoop.fs.Path link, boolean createParent) booleandelete(org.apache.hadoop.fs.Path f) Deprecated.booleandelete(org.apache.hadoop.fs.Path f, boolean recursive) booleandeleteOnExit(org.apache.hadoop.fs.Path f) voiddeleteSnapshot(org.apache.hadoop.fs.Path path, String snapshotName) booleanexists(org.apache.hadoop.fs.Path f) voidfileClosed(DrillFSDataInputStream fsDataInputStream) Remove the givenDrillFSDataInputStreamfrom opened file list.voidfileOpened(org.apache.hadoop.fs.Path path, DrillFSDataInputStream fsDataInputStream) Add new file location andDrillFSDataInputStreamto list.org.apache.hadoop.fs.permission.AclStatusgetAclStatus(org.apache.hadoop.fs.Path path) longgetBlockSize(org.apache.hadoop.fs.Path f) Deprecated.org.apache.hadoop.fs.FileSystem[]org.apache.hadoop.io.compress.CompressionCodecgetCodec(org.apache.hadoop.fs.Path path) Returns theCompressionCodecfor a given file.org.apache.hadoop.conf.ConfigurationgetConf()Returns a copy ofConfigurationfor thisDrillFileSystem.org.apache.hadoop.fs.ContentSummarygetContentSummary(org.apache.hadoop.fs.Path f) longDeprecated.longgetDefaultBlockSize(org.apache.hadoop.fs.Path f) shortDeprecated.shortgetDefaultReplication(org.apache.hadoop.fs.Path path) org.apache.hadoop.security.token.Token<?> getDelegationToken(String renewer) org.apache.hadoop.fs.BlockLocation[]getFileBlockLocations(org.apache.hadoop.fs.FileStatus file, long start, long len) org.apache.hadoop.fs.BlockLocation[]getFileBlockLocations(org.apache.hadoop.fs.Path p, long start, long len) org.apache.hadoop.fs.FileChecksumgetFileChecksum(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.FileChecksumgetFileChecksum(org.apache.hadoop.fs.Path f, long length) org.apache.hadoop.fs.FileStatusgetFileLinkStatus(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.FileStatusgetFileStatus(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.PathlonggetLength(org.apache.hadoop.fs.Path f) Deprecated.org.apache.hadoop.fs.PathgetLinkTarget(org.apache.hadoop.fs.Path f) getName()Deprecated.shortgetReplication(org.apache.hadoop.fs.Path src) Deprecated.org.apache.hadoop.fs.FsServerDefaultsDeprecated.org.apache.hadoop.fs.FsServerDefaultsgetServerDefaults(org.apache.hadoop.fs.Path p) org.apache.hadoop.fs.FsStatusorg.apache.hadoop.fs.FsStatusgetStatus(org.apache.hadoop.fs.Path p) org.apache.hadoop.fs.FileSystemReturns a copy of the underlying file system.getUri()longgetUsed()org.apache.hadoop.fs.Pathbyte[]getXAttrs(org.apache.hadoop.fs.Path path) org.apache.hadoop.fs.FileStatus[]globStatus(org.apache.hadoop.fs.Path pathPattern) org.apache.hadoop.fs.FileStatus[]globStatus(org.apache.hadoop.fs.Path pathPattern, org.apache.hadoop.fs.PathFilter filter) voidinitialize(URI name, org.apache.hadoop.conf.Configuration conf) This method should never be used onDrillFileSystemsinceDrillFileSystemis immutable.booleanisDirectory(org.apache.hadoop.fs.Path f) booleanisFile(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.Path> listCorruptFileBlocks(org.apache.hadoop.fs.Path path) org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.LocatedFileStatus> listFiles(org.apache.hadoop.fs.Path f, boolean recursive) org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.LocatedFileStatus> listLocatedStatus(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.FileStatus[]listStatus(org.apache.hadoop.fs.Path f) org.apache.hadoop.fs.FileStatus[]listStatus(org.apache.hadoop.fs.Path[] files) org.apache.hadoop.fs.FileStatus[]listStatus(org.apache.hadoop.fs.Path[] files, org.apache.hadoop.fs.PathFilter filter) org.apache.hadoop.fs.FileStatus[]listStatus(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.PathFilter filter) org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.FileStatus> listStatusIterator(org.apache.hadoop.fs.Path p) listXAttrs(org.apache.hadoop.fs.Path path) org.apache.hadoop.fs.PathmakeQualified(org.apache.hadoop.fs.Path path) booleanmkdirs(org.apache.hadoop.fs.Path folderPath) booleanmkdirs(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission) voidmodifyAclEntries(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) voidmoveFromLocalFile(org.apache.hadoop.fs.Path[] srcs, org.apache.hadoop.fs.Path dst) voidmoveFromLocalFile(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) voidmoveToLocalFile(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) org.apache.hadoop.fs.FSDataInputStreamopen(org.apache.hadoop.fs.Path f) If OperatorStats are provided return a instrumentedFSDataInputStream.org.apache.hadoop.fs.FSDataInputStreamopen(org.apache.hadoop.fs.Path f, int bufferSize) If OperatorStats are provided return a instrumentedFSDataInputStream.openPossiblyCompressedStream(org.apache.hadoop.fs.Path path) Returns an InputStream from a Hadoop path.voidremoveAcl(org.apache.hadoop.fs.Path path) voidremoveAclEntries(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) voidremoveDefaultAcl(org.apache.hadoop.fs.Path path) voidremoveXAttr(org.apache.hadoop.fs.Path path, String name) booleanrename(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) voidrenameSnapshot(org.apache.hadoop.fs.Path path, String snapshotOldName, String snapshotNewName) org.apache.hadoop.fs.PathresolvePath(org.apache.hadoop.fs.Path p) voidvoidsetConf(org.apache.hadoop.conf.Configuration conf) This method should never be used onDrillFileSystemsinceDrillFileSystemis immutable.voidvoidsetPermission(org.apache.hadoop.fs.Path p, org.apache.hadoop.fs.permission.FsPermission permission) booleansetReplication(org.apache.hadoop.fs.Path src, short replication) voidsetTimes(org.apache.hadoop.fs.Path p, long mtime, long atime) voidsetVerifyChecksum(boolean verifyChecksum) This method should never be used onDrillFileSystemsinceDrillFileSystemis immutable.voidsetWorkingDirectory(org.apache.hadoop.fs.Path new_dir) This method should never be used onDrillFileSystemsinceDrillFileSystemis immutable.voidsetWriteChecksum(boolean writeChecksum) This method should never be used onDrillFileSystemsinceDrillFileSystemis immutable.voidvoidsetXAttr(org.apache.hadoop.fs.Path path, String name, byte[] value, EnumSet<org.apache.hadoop.fs.XAttrSetFlag> flag) org.apache.hadoop.fs.PathstartLocalOutput(org.apache.hadoop.fs.Path fsOutputFile, org.apache.hadoop.fs.Path tmpLocalFile) booleanbooleantruncate(org.apache.hadoop.fs.Path f, long newLength) Methods inherited from class org.apache.hadoop.fs.FileSystem
append, append, appendFile, areSymlinksEnabled, canonicalizeUri, checkPath, clearStatistics, closeAll, closeAllForUGI, create, createDataInputStreamBuilder, createDataInputStreamBuilder, createDataOutputStreamBuilder, createFile, createMultipartUploader, createPathHandle, createSnapshot, enableSymlinks, fixRelativePart, get, get, get, getAdditionalTokenIssuers, getAllStatistics, getAllStoragePolicies, getCanonicalUri, getDefaultPort, getDefaultUri, getFileSystemClass, getFSofPath, getGlobalStorageStatistics, getInitialWorkingDirectory, getLocal, getNamed, getPathHandle, getQuotaUsage, getStatistics, getStatistics, getStoragePolicy, getStorageStatistics, getTrashRoot, getTrashRoots, getUsed, hasPathCapability, listLocatedStatus, listStatusBatch, mkdirs, msync, newInstance, newInstance, newInstance, newInstanceLocal, open, open, openFile, openFile, openFileWithOptions, openFileWithOptions, primitiveCreate, primitiveMkdir, primitiveMkdir, printStatistics, processDeleteOnExit, rename, resolveLink, satisfyStoragePolicy, setDefaultUri, setDefaultUri, setQuota, setQuotaByStorageType, setStoragePolicy, unsetStoragePolicy
-
Field Details
-
UNDERSCORE_PREFIX
- See Also:
-
DOT_PREFIX
- See Also:
-
-
Constructor Details
-
DrillFileSystem
- Throws:
IOException
-
DrillFileSystem
public DrillFileSystem(org.apache.hadoop.conf.Configuration fsConf, OperatorStats operatorStats) throws IOException - Throws:
IOException
-
-
Method Details
-
setConf
public void setConf(org.apache.hadoop.conf.Configuration conf) This method should never be used onDrillFileSystemsinceDrillFileSystemis immutable.- Specified by:
setConfin interfaceorg.apache.hadoop.conf.Configurable- Overrides:
setConfin classorg.apache.hadoop.conf.Configured- Throws:
UnsupportedOperationException- when called.
-
getConf
public org.apache.hadoop.conf.Configuration getConf()Returns a copy ofConfigurationfor thisDrillFileSystem. Note: a copy of theConfigurationis returned in order to enforce immutability.- Specified by:
getConfin interfaceorg.apache.hadoop.conf.Configurable- Overrides:
getConfin classorg.apache.hadoop.conf.Configured- Returns:
- A copy of
Configurationfor thisDrillFileSystem.
-
getUnderlyingFs
public org.apache.hadoop.fs.FileSystem getUnderlyingFs()Returns a copy of the underlying file system. This class implementsFileSystem- Returns:
- The underlying
FileSystem
-
open
public org.apache.hadoop.fs.FSDataInputStream open(org.apache.hadoop.fs.Path f, int bufferSize) throws IOException If OperatorStats are provided return a instrumentedFSDataInputStream.- Specified by:
openin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
open
If OperatorStats are provided return a instrumentedFSDataInputStream.- Overrides:
openin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
initialize
This method should never be used onDrillFileSystemsinceDrillFileSystemis immutable.- Overrides:
initializein classorg.apache.hadoop.fs.FileSystem- Throws:
UnsupportedOperationException- when called.
-
getScheme
- Overrides:
getSchemein classorg.apache.hadoop.fs.FileSystem
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f) throws IOException - Overrides:
createin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, boolean overwrite) throws IOException - Overrides:
createin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, org.apache.hadoop.util.Progressable progress) throws IOException - Overrides:
createin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, short replication) throws IOException - Overrides:
createin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, short replication, org.apache.hadoop.util.Progressable progress) throws IOException - Overrides:
createin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize) throws IOException - Overrides:
createin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, org.apache.hadoop.util.Progressable progress) throws IOException - Overrides:
createin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, short replication, long blockSize) throws IOException - Overrides:
createin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) throws IOException - Overrides:
createin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getFileStatus
public org.apache.hadoop.fs.FileStatus getFileStatus(org.apache.hadoop.fs.Path f) throws IOException - Specified by:
getFileStatusin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
createSymlink
public void createSymlink(org.apache.hadoop.fs.Path target, org.apache.hadoop.fs.Path link, boolean createParent) throws IOException - Overrides:
createSymlinkin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getFileLinkStatus
public org.apache.hadoop.fs.FileStatus getFileLinkStatus(org.apache.hadoop.fs.Path f) throws IOException - Overrides:
getFileLinkStatusin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
supportsSymlinks
public boolean supportsSymlinks()- Overrides:
supportsSymlinksin classorg.apache.hadoop.fs.FileSystem
-
getLinkTarget
- Overrides:
getLinkTargetin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getFileChecksum
public org.apache.hadoop.fs.FileChecksum getFileChecksum(org.apache.hadoop.fs.Path f) throws IOException - Overrides:
getFileChecksumin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
setVerifyChecksum
public void setVerifyChecksum(boolean verifyChecksum) This method should never be used onDrillFileSystemsinceDrillFileSystemis immutable.- Overrides:
setVerifyChecksumin classorg.apache.hadoop.fs.FileSystem- Throws:
UnsupportedOperationException- when called.
-
setWriteChecksum
public void setWriteChecksum(boolean writeChecksum) This method should never be used onDrillFileSystemsinceDrillFileSystemis immutable.- Overrides:
setWriteChecksumin classorg.apache.hadoop.fs.FileSystem- Throws:
UnsupportedOperationException- when called.
-
getStatus
- Overrides:
getStatusin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getStatus
- Overrides:
getStatusin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
setPermission
public void setPermission(org.apache.hadoop.fs.Path p, org.apache.hadoop.fs.permission.FsPermission permission) throws IOException - Overrides:
setPermissionin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
setOwner
public void setOwner(org.apache.hadoop.fs.Path p, String username, String groupname) throws IOException - Overrides:
setOwnerin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
setTimes
- Overrides:
setTimesin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
createSnapshot
public org.apache.hadoop.fs.Path createSnapshot(org.apache.hadoop.fs.Path path, String snapshotName) throws IOException - Overrides:
createSnapshotin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
renameSnapshot
public void renameSnapshot(org.apache.hadoop.fs.Path path, String snapshotOldName, String snapshotNewName) throws IOException - Overrides:
renameSnapshotin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
deleteSnapshot
- Overrides:
deleteSnapshotin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
modifyAclEntries
public void modifyAclEntries(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) throws IOException - Overrides:
modifyAclEntriesin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
removeAclEntries
public void removeAclEntries(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) throws IOException - Overrides:
removeAclEntriesin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
removeDefaultAcl
- Overrides:
removeDefaultAclin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
removeAcl
- Overrides:
removeAclin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
setAcl
public void setAcl(org.apache.hadoop.fs.Path path, List<org.apache.hadoop.fs.permission.AclEntry> aclSpec) throws IOException - Overrides:
setAclin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getAclStatus
public org.apache.hadoop.fs.permission.AclStatus getAclStatus(org.apache.hadoop.fs.Path path) throws IOException - Overrides:
getAclStatusin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getWorkingDirectory
public org.apache.hadoop.fs.Path getWorkingDirectory()- Specified by:
getWorkingDirectoryin classorg.apache.hadoop.fs.FileSystem
-
append
public org.apache.hadoop.fs.FSDataOutputStream append(org.apache.hadoop.fs.Path f, int bufferSize, org.apache.hadoop.util.Progressable progress) throws IOException - Specified by:
appendin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
concat
public void concat(org.apache.hadoop.fs.Path trg, org.apache.hadoop.fs.Path[] psrcs) throws IOException - Overrides:
concatin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getReplication
Deprecated.- Overrides:
getReplicationin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
setReplication
- Overrides:
setReplicationin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
mkdirs
public boolean mkdirs(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission) throws IOException - Specified by:
mkdirsin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
copyFromLocalFile
public void copyFromLocalFile(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
copyFromLocalFilein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
moveFromLocalFile
public void moveFromLocalFile(org.apache.hadoop.fs.Path[] srcs, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
moveFromLocalFilein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
moveFromLocalFile
public void moveFromLocalFile(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
moveFromLocalFilein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
copyFromLocalFile
public void copyFromLocalFile(boolean delSrc, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
copyFromLocalFilein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
copyFromLocalFile
public void copyFromLocalFile(boolean delSrc, boolean overwrite, org.apache.hadoop.fs.Path[] srcs, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
copyFromLocalFilein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
copyFromLocalFile
public void copyFromLocalFile(boolean delSrc, boolean overwrite, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
copyFromLocalFilein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
copyToLocalFile
public void copyToLocalFile(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
copyToLocalFilein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
moveToLocalFile
public void moveToLocalFile(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
moveToLocalFilein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
copyToLocalFile
public void copyToLocalFile(boolean delSrc, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException - Overrides:
copyToLocalFilein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
copyToLocalFile
public void copyToLocalFile(boolean delSrc, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst, boolean useRawLocalFileSystem) throws IOException - Overrides:
copyToLocalFilein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
startLocalOutput
public org.apache.hadoop.fs.Path startLocalOutput(org.apache.hadoop.fs.Path fsOutputFile, org.apache.hadoop.fs.Path tmpLocalFile) throws IOException - Overrides:
startLocalOutputin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
completeLocalOutput
public void completeLocalOutput(org.apache.hadoop.fs.Path fsOutputFile, org.apache.hadoop.fs.Path tmpLocalFile) throws IOException - Overrides:
completeLocalOutputin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
close
- Specified by:
closein interfaceAutoCloseable- Specified by:
closein interfaceCloseable- Overrides:
closein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getUsed
- Overrides:
getUsedin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getBlockSize
Deprecated.- Overrides:
getBlockSizein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getDefaultBlockSize
Deprecated.- Overrides:
getDefaultBlockSizein classorg.apache.hadoop.fs.FileSystem
-
getDefaultBlockSize
public long getDefaultBlockSize(org.apache.hadoop.fs.Path f) - Overrides:
getDefaultBlockSizein classorg.apache.hadoop.fs.FileSystem
-
getDefaultReplication
Deprecated.- Overrides:
getDefaultReplicationin classorg.apache.hadoop.fs.FileSystem
-
getDefaultReplication
public short getDefaultReplication(org.apache.hadoop.fs.Path path) - Overrides:
getDefaultReplicationin classorg.apache.hadoop.fs.FileSystem
-
mkdirs
- Overrides:
mkdirsin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, EnumSet<org.apache.hadoop.fs.CreateFlag> flags, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress, org.apache.hadoop.fs.Options.ChecksumOpt checksumOpt) throws IOException - Overrides:
createin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
createNonRecursive
@Deprecated public org.apache.hadoop.fs.FSDataOutputStream createNonRecursive(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) throws IOException Deprecated.- Overrides:
createNonRecursivein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
createNonRecursive
@Deprecated public org.apache.hadoop.fs.FSDataOutputStream createNonRecursive(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) throws IOException Deprecated.- Overrides:
createNonRecursivein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
createNonRecursive
@Deprecated public org.apache.hadoop.fs.FSDataOutputStream createNonRecursive(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, EnumSet<org.apache.hadoop.fs.CreateFlag> flags, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) throws IOException Deprecated.- Overrides:
createNonRecursivein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
createNewFile
- Overrides:
createNewFilein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
append
public org.apache.hadoop.fs.FSDataOutputStream append(org.apache.hadoop.fs.Path f) throws IOException - Overrides:
appendin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
append
public org.apache.hadoop.fs.FSDataOutputStream append(org.apache.hadoop.fs.Path f, int bufferSize) throws IOException - Overrides:
appendin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) throws IOException - Specified by:
createin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
create
public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, EnumSet<org.apache.hadoop.fs.CreateFlag> flags, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress) throws IOException - Overrides:
createin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
listStatus
public org.apache.hadoop.fs.FileStatus[] listStatus(org.apache.hadoop.fs.Path f) throws FileNotFoundException, IOException - Specified by:
listStatusin classorg.apache.hadoop.fs.FileSystem- Throws:
FileNotFoundExceptionIOException
-
listCorruptFileBlocks
public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.Path> listCorruptFileBlocks(org.apache.hadoop.fs.Path path) throws IOException - Overrides:
listCorruptFileBlocksin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
listStatus
public org.apache.hadoop.fs.FileStatus[] listStatus(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.PathFilter filter) throws FileNotFoundException, IOException - Overrides:
listStatusin classorg.apache.hadoop.fs.FileSystem- Throws:
FileNotFoundExceptionIOException
-
listStatus
public org.apache.hadoop.fs.FileStatus[] listStatus(org.apache.hadoop.fs.Path[] files) throws FileNotFoundException, IOException - Overrides:
listStatusin classorg.apache.hadoop.fs.FileSystem- Throws:
FileNotFoundExceptionIOException
-
listStatus
public org.apache.hadoop.fs.FileStatus[] listStatus(org.apache.hadoop.fs.Path[] files, org.apache.hadoop.fs.PathFilter filter) throws FileNotFoundException, IOException - Overrides:
listStatusin classorg.apache.hadoop.fs.FileSystem- Throws:
FileNotFoundExceptionIOException
-
globStatus
public org.apache.hadoop.fs.FileStatus[] globStatus(org.apache.hadoop.fs.Path pathPattern) throws IOException - Overrides:
globStatusin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
globStatus
public org.apache.hadoop.fs.FileStatus[] globStatus(org.apache.hadoop.fs.Path pathPattern, org.apache.hadoop.fs.PathFilter filter) throws IOException - Overrides:
globStatusin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
listLocatedStatus
public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.LocatedFileStatus> listLocatedStatus(org.apache.hadoop.fs.Path f) throws FileNotFoundException, IOException - Overrides:
listLocatedStatusin classorg.apache.hadoop.fs.FileSystem- Throws:
FileNotFoundExceptionIOException
-
listFiles
public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.LocatedFileStatus> listFiles(org.apache.hadoop.fs.Path f, boolean recursive) throws FileNotFoundException, IOException - Overrides:
listFilesin classorg.apache.hadoop.fs.FileSystem- Throws:
FileNotFoundExceptionIOException
-
getHomeDirectory
public org.apache.hadoop.fs.Path getHomeDirectory()- Overrides:
getHomeDirectoryin classorg.apache.hadoop.fs.FileSystem
-
setWorkingDirectory
public void setWorkingDirectory(org.apache.hadoop.fs.Path new_dir) This method should never be used onDrillFileSystemsinceDrillFileSystemis immutable.- Specified by:
setWorkingDirectoryin classorg.apache.hadoop.fs.FileSystem- Throws:
UnsupportedOperationException- when called.
-
rename
public boolean rename(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst) throws IOException - Specified by:
renamein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
delete
Deprecated.- Overrides:
deletein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
delete
- Specified by:
deletein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
deleteOnExit
- Overrides:
deleteOnExitin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
cancelDeleteOnExit
public boolean cancelDeleteOnExit(org.apache.hadoop.fs.Path f) - Overrides:
cancelDeleteOnExitin classorg.apache.hadoop.fs.FileSystem
-
exists
- Overrides:
existsin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
isDirectory
- Overrides:
isDirectoryin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
isFile
- Overrides:
isFilein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getLength
Deprecated.- Overrides:
getLengthin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getContentSummary
public org.apache.hadoop.fs.ContentSummary getContentSummary(org.apache.hadoop.fs.Path f) throws IOException - Overrides:
getContentSummaryin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getUri
- Specified by:
getUriin classorg.apache.hadoop.fs.FileSystem
-
getCanonicalServiceName
- Specified by:
getCanonicalServiceNamein interfaceorg.apache.hadoop.security.token.DelegationTokenIssuer- Overrides:
getCanonicalServiceNamein classorg.apache.hadoop.fs.FileSystem
-
getName
Deprecated.- Overrides:
getNamein classorg.apache.hadoop.fs.FileSystem
-
makeQualified
public org.apache.hadoop.fs.Path makeQualified(org.apache.hadoop.fs.Path path) - Overrides:
makeQualifiedin classorg.apache.hadoop.fs.FileSystem
-
getDelegationToken
@Private public org.apache.hadoop.security.token.Token<?> getDelegationToken(String renewer) throws IOException - Specified by:
getDelegationTokenin interfaceorg.apache.hadoop.security.token.DelegationTokenIssuer- Overrides:
getDelegationTokenin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
addDelegationTokens
@LimitedPrivate({"HDFS","MapReduce"}) public org.apache.hadoop.security.token.Token<?>[] addDelegationTokens(String renewer, org.apache.hadoop.security.Credentials credentials) throws IOException - Specified by:
addDelegationTokensin interfaceorg.apache.hadoop.security.token.DelegationTokenIssuer- Throws:
IOException
-
getChildFileSystems
@LimitedPrivate("HDFS") public org.apache.hadoop.fs.FileSystem[] getChildFileSystems()- Overrides:
getChildFileSystemsin classorg.apache.hadoop.fs.FileSystem
-
getFileBlockLocations
public org.apache.hadoop.fs.BlockLocation[] getFileBlockLocations(org.apache.hadoop.fs.FileStatus file, long start, long len) throws IOException - Overrides:
getFileBlockLocationsin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getFileBlockLocations
public org.apache.hadoop.fs.BlockLocation[] getFileBlockLocations(org.apache.hadoop.fs.Path p, long start, long len) throws IOException - Overrides:
getFileBlockLocationsin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getServerDefaults
Deprecated.- Overrides:
getServerDefaultsin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getServerDefaults
public org.apache.hadoop.fs.FsServerDefaults getServerDefaults(org.apache.hadoop.fs.Path p) throws IOException - Overrides:
getServerDefaultsin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
resolvePath
- Overrides:
resolvePathin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
truncate
- Overrides:
truncatein classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
listStatusIterator
public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.FileStatus> listStatusIterator(org.apache.hadoop.fs.Path p) throws FileNotFoundException, IOException - Overrides:
listStatusIteratorin classorg.apache.hadoop.fs.FileSystem- Throws:
FileNotFoundExceptionIOException
-
access
public void access(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.permission.FsAction mode) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, IOException - Overrides:
accessin classorg.apache.hadoop.fs.FileSystem- Throws:
org.apache.hadoop.security.AccessControlExceptionFileNotFoundExceptionIOException
-
getFileChecksum
public org.apache.hadoop.fs.FileChecksum getFileChecksum(org.apache.hadoop.fs.Path f, long length) throws IOException - Overrides:
getFileChecksumin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
setXAttr
- Overrides:
setXAttrin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
setXAttr
public void setXAttr(org.apache.hadoop.fs.Path path, String name, byte[] value, EnumSet<org.apache.hadoop.fs.XAttrSetFlag> flag) throws IOException - Overrides:
setXAttrin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getXAttr
- Overrides:
getXAttrin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getXAttrs
- Overrides:
getXAttrsin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
getXAttrs
public Map<String,byte[]> getXAttrs(org.apache.hadoop.fs.Path path, List<String> names) throws IOException - Overrides:
getXAttrsin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
listXAttrs
- Overrides:
listXAttrsin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
removeXAttr
- Overrides:
removeXAttrin classorg.apache.hadoop.fs.FileSystem- Throws:
IOException
-
openPossiblyCompressedStream
Returns an InputStream from a Hadoop path. If the data is compressed, this method will return a compressed InputStream depending on the codec.- Parameters:
path- Input file path- Returns:
- InputStream of opened file path
- Throws:
IOException- If the file is unreachable, unavailable or otherwise unreadable
-
getCodec
public org.apache.hadoop.io.compress.CompressionCodec getCodec(org.apache.hadoop.fs.Path path) Returns theCompressionCodecfor a given file. This can be used to determine the type of compression (if any) which was used. Returns null if the file is not compressed.- Parameters:
path- The file of unknown compression- Returns:
- CompressionCodec used by the file. Null if the file is not compressed.
-
fileOpened
Description copied from interface:OpenFileTrackerAdd new file location andDrillFSDataInputStreamto list.- Specified by:
fileOpenedin interfaceOpenFileTracker
-
fileClosed
Description copied from interface:OpenFileTrackerRemove the givenDrillFSDataInputStreamfrom opened file list.- Specified by:
fileClosedin interfaceOpenFileTracker- Parameters:
fsDataInputStream-
-