org.apache.drill.exec.expr.fn

Class FunctionImplementationRegistry

java.lang.Object

org.apache.drill.exec.expr.fn.FunctionImplementationRegistry

All Implemented Interfaces:

AutoCloseable, FunctionLookupContext

public class FunctionImplementationRegistry
extends Object
implements FunctionLookupContext, AutoCloseable

Registry for functions. Notably, in addition to Drill its functions (in LocalFunctionRegistry), other PluggableFunctionRegistry (e.g., HiveFunctionRegistry) is also registered in this class

Constructor Summary

Constructors

Constructor and Description
FunctionImplementationRegistry(DrillConfig config)
FunctionImplementationRegistry(DrillConfig config, ScanResult classpathScan)
FunctionImplementationRegistry(DrillConfig config, ScanResult classpathScan, OptionManager optionManager)
FunctionImplementationRegistry(DrillConfig config, ScanResult classpathScan, OptionSet optionManager)

Method Summary

Modifier and Type	Method and Description
void	close() If deleteTmpDir is set to true, deletes generated temporary directory.
DrillFuncHolder	findDrillFunction(FunctionResolver functionResolver, FunctionCall functionCall) First attempts to find the Drill function implementation that matches the name, arg types and return type.
DrillFuncHolder	findExactMatchingDrillFunction(String name, List<TypeProtos.MajorType> argTypes, TypeProtos.MajorType returnType) Finds the Drill function implementation that matches the name, arg types and return type.
AbstractFuncHolder	findNonDrillFunction(FunctionCall functionCall) Find function implementation for given functionCall in non-Drill function registries such as Hive UDF registry.
Map<String,List<FunctionHolder>>	getAllJarsWithFunctionsHolders() Retrieve all functions, mapped by source jars (after syncing)
LocalFunctionRegistry	getLocalFunctionRegistry()
RemoteFunctionRegistry	getRemoteFunctionRegistry()
boolean	isFunctionComplexOutput(String name)
void	register(DrillOperatorTable operatorTable) Register functions in given operator table.
boolean	syncWithRemoteRegistry(int version) Purpose of this method is to synchronize remote and local function registries if needed and to inform if function registry was changed after given version.
List<String>	validate(org.apache.hadoop.fs.Path path) Using given local path to jar creates unique class loader for this jar.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FunctionImplementationRegistry

public FunctionImplementationRegistry(DrillConfig config)

FunctionImplementationRegistry

public FunctionImplementationRegistry(DrillConfig config,
                                      ScanResult classpathScan)

FunctionImplementationRegistry

public FunctionImplementationRegistry(DrillConfig config,
                                      ScanResult classpathScan,
                                      OptionManager optionManager)

FunctionImplementationRegistry

public FunctionImplementationRegistry(DrillConfig config,
                                      ScanResult classpathScan,
                                      OptionSet optionManager)

Method Detail

register

public void register(DrillOperatorTable operatorTable)

Register functions in given operator table.

Parameters:

operatorTable - operator table

findDrillFunction

public DrillFuncHolder findDrillFunction(FunctionResolver functionResolver,
                                         FunctionCall functionCall)

First attempts to find the Drill function implementation that matches the name, arg types and return type. If exact function implementation was not found, syncs local function registry with remote function registry if needed and tries to find function implementation one more time but this time using given functionResolver.

Specified by:

findDrillFunction in interface FunctionLookupContext

Parameters:

functionResolver - function resolver

functionCall - function call

Returns:

best matching function holder

findExactMatchingDrillFunction

public DrillFuncHolder findExactMatchingDrillFunction(String name,
                                                      List<TypeProtos.MajorType> argTypes,
                                                      TypeProtos.MajorType returnType)

Finds the Drill function implementation that matches the name, arg types and return type.

Parameters:

name - function name

argTypes - input parameters types

returnType - function return type

Returns:

exactly matching function holder

findNonDrillFunction

public AbstractFuncHolder findNonDrillFunction(FunctionCall functionCall)

Find function implementation for given functionCall in non-Drill function registries such as Hive UDF registry. Note: Order of searching is same as order of PluggableFunctionRegistry implementations found on classpath.

Specified by:

findNonDrillFunction in interface FunctionLookupContext

Parameters:

functionCall - function call

Returns:

drill function holder

isFunctionComplexOutput

public boolean isFunctionComplexOutput(String name)

getLocalFunctionRegistry

public LocalFunctionRegistry getLocalFunctionRegistry()

getRemoteFunctionRegistry

public RemoteFunctionRegistry getRemoteFunctionRegistry()

validate

public List<String> validate(org.apache.hadoop.fs.Path path)
                      throws IOException

Using given local path to jar creates unique class loader for this jar. Class loader is closed to release opened connection to jar when validation is finished. Scan jar content to receive list of all scanned classes and starts validation process against local function registry. Checks if received list of validated function is not empty.

Parameters:

path - local path to jar we need to validate

Returns:

list of validated function signatures

Throws:

IOException

syncWithRemoteRegistry

public boolean syncWithRemoteRegistry(int version)

Purpose of this method is to synchronize remote and local function registries if needed and to inform if function registry was changed after given version.

To make synchronization as much light-weigh as possible, first only versions of both registries are checked without any locking. If synchronization is needed, enters synchronized block to prevent others loading the same jars. The need of synchronization is checked again (double-check lock) before comparing jars. If any missing jars are found, they are downloaded to local udf area, each is wrapped into JarScan. Once jar download is finished, all missing jars are registered in one batch. In case if any errors during jars download / registration, these errors are logged.

During registration local function registry is updated with remote function registry version it is synced with. When at least one jar of the missing jars failed to download / register, local function registry version are not updated but jars that where successfully downloaded / registered are added to local function registry.

If synchronization between remote and local function registry was not needed, checks if given registry version matches latest sync version to inform if function registry was changed after given version.

Parameters:

version - remote function registry local function registry was based on

Returns:

true if remote and local function registries were synchronized after given version

getAllJarsWithFunctionsHolders

public Map<String,List<FunctionHolder>> getAllJarsWithFunctionsHolders()

Retrieve all functions, mapped by source jars (after syncing)

Returns:

Map of source jars and their functionHolders

close

public void close()

If deleteTmpDir is set to true, deletes generated temporary directory. Otherwise cleans up localUdfDir.

Specified by:

close in interface AutoCloseable