Package org.apache.drill.exec.util

Class FileSystemUtil

java.lang.Object
org.apache.drill.exec.util.FileSystemUtil
public class FileSystemUtil extends Object
Helper class that provides methods to list directories or file or both statuses. Can list statuses recursive and apply custom filters.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final org.apache.hadoop.fs.PathFilter
    DUMMY_FILTER
    Filter that will accept all files and directories.
    static final String
    RECURSIVE_LISTING_PROP_NAME
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    FileSystemUtil()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static List<org.apache.hadoop.fs.FileStatus>
    anyFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.PathFilter... filters)
    Returns the statuses of at least one file present in given path applying custom filters if present.
    static List<org.apache.hadoop.fs.FileStatus>
    listAll(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
    Returns statuses of all directories and files present in given path applying custom filters if present.
    static List<org.apache.hadoop.fs.FileStatus>
    listAllSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
    Returns statuses of all directories and files present in given path applying custom filters if present.
    static List<org.apache.hadoop.fs.FileStatus>
    listDirectories(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
    Returns statuses of all directories present in given path applying custom filters if present.
    static List<org.apache.hadoop.fs.FileStatus>
    listDirectoriesSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
    Returns statuses of all directories present in given path applying custom filters if present.
    static List<org.apache.hadoop.fs.FileStatus>
    listFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
    Returns statuses of all files present in given path applying custom filters if present.
    static List<org.apache.hadoop.fs.FileStatus>
    listFilesSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
    Returns statuses of all files present in given path applying custom filters if present.
    static org.apache.hadoop.fs.PathFilter
    mergeFilters(org.apache.hadoop.fs.PathFilter... filters)
    Will merge given array of filters into one.
    static org.apache.hadoop.fs.PathFilter
    mergeFilters(org.apache.hadoop.fs.PathFilter filter, org.apache.hadoop.fs.PathFilter[] filters)
    Merges given filter with array of filters.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • RECURSIVE_LISTING_PROP_NAME

      public static final String RECURSIVE_LISTING_PROP_NAME
      See Also:

    • DUMMY_FILTER

      public static final org.apache.hadoop.fs.PathFilter DUMMY_FILTER
      Filter that will accept all files and directories.

  • Constructor Details

    • FileSystemUtil

      public FileSystemUtil()

  • Method Details

    • listDirectories

      public static List<org.apache.hadoop.fs.FileStatus> listDirectories(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) throws IOException
      Returns statuses of all directories present in given path applying custom filters if present. Will also include nested directories if recursive flag is set to true.
      Parameters:
      fs - current file system
      path - path to directory
      recursive - true if nested directories should be included
      filters - list of custom filters (optional)
      Returns:
      list of matching directory statuses
      Throws:
      IOException

    • listDirectoriesSafe

      public static List<org.apache.hadoop.fs.FileStatus> listDirectoriesSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
      Returns statuses of all directories present in given path applying custom filters if present. Will also include nested directories if recursive flag is set to true. Will ignore all exceptions during listing if any.
      Parameters:
      fs - current file system
      path - path to directory
      recursive - true if nested directories should be included
      filters - list of custom filters (optional)
      Returns:
      list of matching directory statuses

    • listFiles

      public static List<org.apache.hadoop.fs.FileStatus> listFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) throws IOException
      Returns statuses of all files present in given path applying custom filters if present. Will also include files from nested directories if recursive flag is set to true.
      Parameters:
      fs - current file system
      path - path to file or directory
      recursive - true if files in nested directories should be included
      filters - list of custom filters (optional)
      Returns:
      list of matching file statuses
      Throws:
      IOException

    • listFilesSafe

      public static List<org.apache.hadoop.fs.FileStatus> listFilesSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
      Returns statuses of all files present in given path applying custom filters if present. Will also include files from nested directories if recursive flag is set to true.
      Parameters:
      fs - current file system
      path - path to file or directory
      recursive - true if files in nested directories should be included
      filters - list of custom filters (optional)
      Returns:
      list of matching file statuses

    • anyFile

      public static List<org.apache.hadoop.fs.FileStatus> anyFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.PathFilter... filters) throws IOException
      Returns the statuses of at least one file present in given path applying custom filters if present.
      Parameters:
      fs - current file system
      path - path to file or directory
      filters - list of custom filters (optional)
      Returns:
      list of at most one matching file status
      Throws:
      IOException

    • listAll

      public static List<org.apache.hadoop.fs.FileStatus> listAll(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) throws IOException
      Returns statuses of all directories and files present in given path applying custom filters if present. Will also include nested directories and their files if recursive flag is set to true.
      Parameters:
      fs - current file system
      path - path to file or directory
      recursive - true if nested directories and their files should be included
      filters - list of custom filters (optional)
      Returns:
      list of matching directory and file statuses
      Throws:
      IOException

    • listAllSafe

      public static List<org.apache.hadoop.fs.FileStatus> listAllSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
      Returns statuses of all directories and files present in given path applying custom filters if present. Will also include nested directories and their files if recursive flag is set to true. Will ignore all exceptions during listing if any.
      Parameters:
      fs - current file system
      path - path to file or directory
      recursive - true if nested directories and their files should be included
      filters - list of custom filters (optional)
      Returns:
      list of matching directory and file statuses

    • mergeFilters

      public static org.apache.hadoop.fs.PathFilter mergeFilters(org.apache.hadoop.fs.PathFilter filter, org.apache.hadoop.fs.PathFilter[] filters)
      Merges given filter with array of filters. If array of filters is null or empty, will return given filter.
      Parameters:
      filter - given filter
      filters - array of filters
      Returns:
      one filter that combines all given filters

    • mergeFilters

      public static org.apache.hadoop.fs.PathFilter mergeFilters(org.apache.hadoop.fs.PathFilter... filters)
      Will merge given array of filters into one. If given array of filters is empty, will return DUMMY_FILTER.
      Parameters:
      filters - array of filters
      Returns:
      one filter that combines all given filters