Package org.apache.drill.exec.util

Class DrillFileSystemUtil

java.lang.Object
org.apache.drill.exec.util.DrillFileSystemUtil
public class DrillFileSystemUtil extends Object
In Drill file system all directories and files that start with dot or underscore is ignored. This helper class that delegates all work to list directory and file statuses to FileSystemUtil class, only adding Drill file system filter first.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final org.apache.hadoop.fs.PathFilter
    DRILL_SYSTEM_FILTER
    Path filter that skips all files and folders that start with dot or underscore.

  • Constructor Summary

    Constructors
    Constructor
    Description
    DrillFileSystemUtil()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static List<org.apache.hadoop.fs.FileStatus>
    anyFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.PathFilter... filters)
    Returns the status of any file present in given path applying custom filters if present.
    static org.apache.hadoop.fs.Path
    createPathSafe(String path)
    Safely creates Hadoop Path for null and empty String paths
    static List<org.apache.hadoop.fs.FileStatus>
    listAll(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
    Returns statuses of all directories and files present in given path applying custom filters if present.
    static List<org.apache.hadoop.fs.FileStatus>
    listAllSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
    Returns statuses of all directories and files present in given path applying custom filters if present.
    static List<org.apache.hadoop.fs.FileStatus>
    listDirectories(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
    Returns statuses of all directories present in given path applying custom filters if present.
    static List<org.apache.hadoop.fs.FileStatus>
    listDirectoriesSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
    Returns statuses of all directories present in given path applying custom filters if present.
    static List<org.apache.hadoop.fs.FileStatus>
    listFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
    Returns statuses of all files present in given path applying custom filters if present.
    static List<org.apache.hadoop.fs.FileStatus>
    listFilesSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
    Returns statuses of all files present in given path applying custom filters if present.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • DRILL_SYSTEM_FILTER

      public static final org.apache.hadoop.fs.PathFilter DRILL_SYSTEM_FILTER
      Path filter that skips all files and folders that start with dot or underscore.

  • Constructor Details

    • DrillFileSystemUtil

      public DrillFileSystemUtil()

  • Method Details

    • listDirectories

      public static List<org.apache.hadoop.fs.FileStatus> listDirectories(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) throws IOException
      Returns statuses of all directories present in given path applying custom filters if present. Directories that start with dot or underscore are skipped. Will also include nested directories if recursive flag is set to true.
      Parameters:
      fs - current file system
      path - path to directory
      recursive - true if nested directories should be included
      filters - list of custom filters (optional)
      Returns:
      list of matching directory statuses
      Throws:
      IOException

    • listDirectoriesSafe

      public static List<org.apache.hadoop.fs.FileStatus> listDirectoriesSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
      Returns statuses of all directories present in given path applying custom filters if present. Directories that start with dot or underscore are skipped. Will include nested directories if recursive flag is set to true. Will ignore all exceptions during listing if any.
      Parameters:
      fs - current file system
      path - path to directory
      recursive - true if nested directories should be included
      filters - list of custom filters (optional)
      Returns:
      list of matching directory statuses

    • listFiles

      public static List<org.apache.hadoop.fs.FileStatus> listFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) throws IOException
      Returns statuses of all files present in given path applying custom filters if present. Files and nested directories that start with dot or underscore are skipped. Will also include files from nested directories if recursive flag is set to true.
      Parameters:
      fs - current file system
      path - path to file or directory
      recursive - true if files in nested directories should be included
      filters - list of custom filters (optional)
      Returns:
      list of matching file statuses
      Throws:
      IOException

    • listFilesSafe

      public static List<org.apache.hadoop.fs.FileStatus> listFilesSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
      Returns statuses of all files present in given path applying custom filters if present. Files and nested directories that start with dot or underscore are skipped. Will include files from nested directories if recursive flag is set to true. Will ignore all exceptions during listing if any.
      Parameters:
      fs - current file system
      path - path to file or directory
      recursive - true if files in nested directories should be included
      filters - list of custom filters (optional)
      Returns:
      list of matching file statuses

    • anyFile

      public static List<org.apache.hadoop.fs.FileStatus> anyFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.PathFilter... filters) throws IOException
      Returns the status of any file present in given path applying custom filters if present. Files and nested directories that start with dot or underscore are skipped. Will also include files from nested directories.
      Parameters:
      fs - current file system
      path - path to file or directory
      recursive - true if files in nested directories should be included
      filters - list of custom filters (optional)
      Returns:
      list of at most one matching file status
      Throws:
      IOException

    • listAll

      public static List<org.apache.hadoop.fs.FileStatus> listAll(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) throws IOException
      Returns statuses of all directories and files present in given path applying custom filters if present. Directories and files that start with dot or underscore are skipped. Will also include nested directories and their files if recursive flag is set to true.
      Parameters:
      fs - current file system
      path - path to file or directory
      recursive - true if nested directories and their files should be included
      filters - list of custom filters (optional)
      Returns:
      list of matching directory and file statuses
      Throws:
      IOException

    • listAllSafe

      public static List<org.apache.hadoop.fs.FileStatus> listAllSafe(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters)
      Returns statuses of all directories and files present in given path applying custom filters if present. Directories and files that start with dot or underscore are skipped. Will include nested directories and their files if recursive flag is set to true. Will ignore all exceptions during listing if any.
      Parameters:
      fs - current file system
      path - path to file or directory
      recursive - true if nested directories and their files should be included
      filters - list of custom filters (optional)
      Returns:
      list of matching directory and file statuses

    • createPathSafe

      public static org.apache.hadoop.fs.Path createPathSafe(String path)
      Safely creates Hadoop Path for null and empty String paths
      Parameters:
      path - String path, which can be null or empty
      Returns:
      Hadoop Path. Root - for empty or null path