org.apache.drill.exec.util

Class FileSystemUtil

java.lang.Object

org.apache.drill.exec.util.FileSystemUtil

public class FileSystemUtil
extends Object

Helper class that provides methods to list directories or file or both statuses. Can list statuses recursive and apply custom filters.

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.hadoop.fs.PathFilter	DUMMY_FILTER Filter that will accept all files and directories.
static String	RECURSIVE_FILE_LISTING_MAX_SIZE

Constructor Summary

Constructors

Constructor and Description
FileSystemUtil()

Method Summary

Modifier and Type	Method and Description
static List<org.apache.hadoop.fs.FileStatus>	listAll(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories and files present in given path applying custom filters if present.
static List<org.apache.hadoop.fs.FileStatus>	listAllSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories and files present in given path applying custom filters if present.
static List<org.apache.hadoop.fs.FileStatus>	listDirectories(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories present in given path applying custom filters if present.
static List<org.apache.hadoop.fs.FileStatus>	listDirectoriesSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories present in given path applying custom filters if present.
static List<org.apache.hadoop.fs.FileStatus>	listFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all files present in given path applying custom filters if present.
static List<org.apache.hadoop.fs.FileStatus>	listFilesSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all files present in given path applying custom filters if present.
static org.apache.hadoop.fs.PathFilter	mergeFilters(org.apache.hadoop.fs.PathFilter... filters) Will merge given array of filters into one.
static org.apache.hadoop.fs.PathFilter	mergeFilters(org.apache.hadoop.fs.PathFilter filter, org.apache.hadoop.fs.PathFilter[] filters) Merges given filter with array of filters.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

RECURSIVE_FILE_LISTING_MAX_SIZE

public static final String RECURSIVE_FILE_LISTING_MAX_SIZE

See Also:

Constant Field Values

DUMMY_FILTER

public static final org.apache.hadoop.fs.PathFilter DUMMY_FILTER

Filter that will accept all files and directories.

Constructor Detail

FileSystemUtil

public FileSystemUtil()

Method Detail

listDirectories

public static List<org.apache.hadoop.fs.FileStatus> listDirectories(org.apache.hadoop.fs.FileSystem fs,
                                                                    org.apache.hadoop.fs.Path path,
                                                                    boolean recursive,
                                                                    org.apache.hadoop.fs.PathFilter... filters)
                                                             throws IOException

Returns statuses of all directories present in given path applying custom filters if present. Will also include nested directories if recursive flag is set to true.

Parameters:

fs - current file system

path - path to directory

recursive - true if nested directories should be included

filters - list of custom filters (optional)

Returns:

list of matching directory statuses

Throws:

IOException

listDirectoriesSafe

public static List<org.apache.hadoop.fs.FileStatus> listDirectoriesSafe(org.apache.hadoop.fs.FileSystem fs,
                                                                        org.apache.hadoop.fs.Path path,
                                                                        boolean recursive,
                                                                        org.apache.hadoop.fs.PathFilter... filters)

Returns statuses of all directories present in given path applying custom filters if present. Will also include nested directories if recursive flag is set to true. Will ignore all exceptions during listing if any.

Parameters:

fs - current file system

path - path to directory

recursive - true if nested directories should be included

filters - list of custom filters (optional)

Returns:

list of matching directory statuses

listFiles

public static List<org.apache.hadoop.fs.FileStatus> listFiles(org.apache.hadoop.fs.FileSystem fs,
                                                              org.apache.hadoop.fs.Path path,
                                                              boolean recursive,
                                                              org.apache.hadoop.fs.PathFilter... filters)
                                                       throws IOException

Returns statuses of all files present in given path applying custom filters if present. Will also include files from nested directories if recursive flag is set to true.

Parameters:

fs - current file system

path - path to file or directory

recursive - true if files in nested directories should be included

filters - list of custom filters (optional)

Returns:

list of matching file statuses

Throws:

IOException

listFilesSafe

public static List<org.apache.hadoop.fs.FileStatus> listFilesSafe(org.apache.hadoop.fs.FileSystem fs,
                                                                  org.apache.hadoop.fs.Path path,
                                                                  boolean recursive,
                                                                  org.apache.hadoop.fs.PathFilter... filters)

Returns statuses of all files present in given path applying custom filters if present. Will also include files from nested directories if recursive flag is set to true.

Parameters:

fs - current file system

path - path to file or directory

recursive - true if files in nested directories should be included

filters - list of custom filters (optional)

Returns:

list of matching file statuses

listAll

public static List<org.apache.hadoop.fs.FileStatus> listAll(org.apache.hadoop.fs.FileSystem fs,
                                                            org.apache.hadoop.fs.Path path,
                                                            boolean recursive,
                                                            org.apache.hadoop.fs.PathFilter... filters)
                                                     throws IOException

Returns statuses of all directories and files present in given path applying custom filters if present. Will also include nested directories and their files if recursive flag is set to true.

Parameters:

fs - current file system

path - path to file or directory

recursive - true if nested directories and their files should be included

filters - list of custom filters (optional)

Returns:

list of matching directory and file statuses

Throws:

IOException

listAllSafe

public static List<org.apache.hadoop.fs.FileStatus> listAllSafe(org.apache.hadoop.fs.FileSystem fs,
                                                                org.apache.hadoop.fs.Path path,
                                                                boolean recursive,
                                                                org.apache.hadoop.fs.PathFilter... filters)

Returns statuses of all directories and files present in given path applying custom filters if present. Will also include nested directories and their files if recursive flag is set to true. Will ignore all exceptions during listing if any.

Parameters:

fs - current file system

path - path to file or directory

recursive - true if nested directories and their files should be included

filters - list of custom filters (optional)

Returns:

list of matching directory and file statuses

mergeFilters

public static org.apache.hadoop.fs.PathFilter mergeFilters(org.apache.hadoop.fs.PathFilter filter,
                                                           org.apache.hadoop.fs.PathFilter[] filters)

Merges given filter with array of filters. If array of filters is null or empty, will return given filter.

Parameters:

filter - given filter

filters - array of filters

Returns:

one filter that combines all given filters

mergeFilters

public static org.apache.hadoop.fs.PathFilter mergeFilters(org.apache.hadoop.fs.PathFilter... filters)

Will merge given array of filters into one. If given array of filters is empty, will return DUMMY_FILTER.

Parameters:

filters - array of filters

Returns:

one filter that combines all given filters