Package org.apache.drill.exec.store.hive

Class HiveMetadataProvider

java.lang.Object

org.apache.drill.exec.store.hive.HiveMetadataProvider

public class HiveMetadataProvider
extends Object

Class which provides methods to get metadata of given Hive table selection. It tries to use the stats stored in MetaStore whenever available and delays the costly operation of loading of InputSplits until needed. When loaded, InputSplits are cached to speedup subsequent access.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	HiveMetadataProvider.HiveStats	Contains stats.
static class	HiveMetadataProvider.LogicalInputSplit	Contains group of input splits along with the partition.

Field Summary

Fields

Modifier and Type	Field	Description
static final int	RECORD_SIZE

Constructor Summary

Constructors

Constructor	Description
HiveMetadataProvider(String userName, HiveReadEntry hiveReadEntry, org.apache.hadoop.hive.conf.HiveConf hiveConf)

Method Summary

Modifier and Type	Method	Description
protected List<String>	getInputDirectories(HiveReadEntry hiveReadEntry)	Get the list of directories which contain the input files.
List<HiveMetadataProvider.LogicalInputSplit>	getInputSplits(HiveReadEntry hiveReadEntry)	Return HiveMetadataProvider.LogicalInputSplits for given HiveReadEntry.
HiveMetadataProvider.HiveStats	getStats(HiveReadEntry hiveReadEntry)	Return stats for table/partitions in given HiveReadEntry.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

RECORD_SIZE

public static final int RECORD_SIZE

See Also:

• Constant Field Values

Constructor Details

HiveMetadataProvider

public HiveMetadataProvider(String userName,
 HiveReadEntry hiveReadEntry,
 org.apache.hadoop.hive.conf.HiveConf hiveConf)

Method Details

getStats

public HiveMetadataProvider.HiveStats getStats(HiveReadEntry hiveReadEntry)
                                        throws IOException

Return stats for table/partitions in given HiveReadEntry. If valid stats are available in MetaStore, return it. Otherwise estimate using the size of the input data.

Parameters:

hiveReadEntry - Subset of the HiveReadEntry used when creating this cache object.

Returns:

hive statistics holder

Throws:

IOException - if was unable to retrieve table statistics

getInputSplits

public List<HiveMetadataProvider.LogicalInputSplit> getInputSplits(HiveReadEntry hiveReadEntry)

Return HiveMetadataProvider.LogicalInputSplits for given HiveReadEntry. First splits are looked up in cache, if not found go through InputFormat.getSplits(JobConf, int) to find the splits.

Parameters:

hiveReadEntry - Subset of the HiveReadEntry used when creating this object.

Returns:

list of logically grouped input splits

getInputDirectories

protected List<String> getInputDirectories(HiveReadEntry hiveReadEntry)

Get the list of directories which contain the input files. This list is useful for explain plan purposes.

Parameters:

hiveReadEntry - HiveReadEntry containing the input table and/or partitions.