org.apache.drill.exec.physical.impl.common

Interface HashTable

All Known Implementing Classes:

HashTableTemplate

public interface HashTable

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	HashTable.PutStatus

Field Summary

Fields

Modifier and Type	Field and Description
static int	BATCH_MASK
static int	BATCH_SIZE The batch size used for internal batch holders
static float	DEFAULT_LOAD_FACTOR The default load factor of a hash table.
static int	MAXIMUM_CAPACITY The maximum capacity of the hash table (in terms of number of buckets).
static TemplateClassDefinition<HashTable>	TEMPLATE_DEFINITION

Method Summary

Modifier and Type	Method and Description
void	clear() Frees all the direct memory consumed by the HashTable.
long	getActualSize() The amount of direct memory consumed by the hash table.
int	getBuildHashCode(int incomingRowIdx) Computes the hash code for the record at the given index in the build side batch.
int	getProbeHashCode(int incomingRowIdx) Computes the hash code for the record at the given index in the probe side batch.
void	getStats(HashTableStats stats)
int	getTargetBatchRowCount()
boolean	isEmpty()
String	makeDebugString() Returns a message containing memory usage statistics.
org.apache.commons.lang3.tuple.Pair<VectorContainer,Integer>	nextBatch()
boolean	outputKeys(int batchIdx, VectorContainer outContainer, int numRecords) Retrieves the key columns and transfers them to the output container.
int	probeForKey(int incomingRowIdx, int hashCode)
HashTable.PutStatus	put(int incomingRowIdx, IndexPointer htIdxHolder, int hashCode, int batchSize)
void	reset() Clears all the memory used by the HashTable and re-initializes it.
void	setTargetBatchRowCount(int batchRowCount)
void	setup(HashTableConfig htConfig, BufferAllocator allocator, VectorContainer incomingBuild, RecordBatch incomingProbe, RecordBatch outgoing, VectorContainer htContainerOrig, FragmentContext context, ClassGenerator<?> cg) setup(org.apache.drill.exec.physical.impl.common.HashTableConfig, org.apache.drill.exec.memory.BufferAllocator, org.apache.drill.exec.record.VectorContainer, org.apache.drill.exec.record.RecordBatch, org.apache.drill.exec.record.RecordBatch, org.apache.drill.exec.record.VectorContainer, org.apache.drill.exec.ops.FragmentContext, org.apache.drill.exec.expr.ClassGenerator<?>) must be called before anything can be done to the HashTable.
int	size()
void	updateBatches() Updates the incoming (build and probe side) value vectors references in the HashTableTemplate.BatchHolders.
void	updateIncoming(VectorContainer newIncoming, RecordBatch newIncomingProbe) Changes the incoming probe and build side batches, and then updates all the value vector references in the HashTableTemplate.BatchHolders.
void	updateInitialCapacity(int initialCapacity) Update the initial capacity for the hash table.

Field Detail

TEMPLATE_DEFINITION

static final TemplateClassDefinition<HashTable> TEMPLATE_DEFINITION

MAXIMUM_CAPACITY

static final int MAXIMUM_CAPACITY

The maximum capacity of the hash table (in terms of number of buckets).

See Also:

Constant Field Values

DEFAULT_LOAD_FACTOR

static final float DEFAULT_LOAD_FACTOR

The default load factor of a hash table.

See Also:

Constant Field Values

BATCH_SIZE

static final int BATCH_SIZE

The batch size used for internal batch holders

See Also:

Constant Field Values

BATCH_MASK

static final int BATCH_MASK

See Also:

Constant Field Values

Method Detail

setup

void setup(HashTableConfig htConfig,
           BufferAllocator allocator,
           VectorContainer incomingBuild,
           RecordBatch incomingProbe,
           RecordBatch outgoing,
           VectorContainer htContainerOrig,
           FragmentContext context,
           ClassGenerator<?> cg)

setup(org.apache.drill.exec.physical.impl.common.HashTableConfig, org.apache.drill.exec.memory.BufferAllocator, org.apache.drill.exec.record.VectorContainer, org.apache.drill.exec.record.RecordBatch, org.apache.drill.exec.record.RecordBatch, org.apache.drill.exec.record.VectorContainer, org.apache.drill.exec.ops.FragmentContext, org.apache.drill.exec.expr.ClassGenerator<?>) must be called before anything can be done to the HashTable.

Parameters:

htConfig -

allocator -

incomingBuild -

incomingProbe -

outgoing -

htContainerOrig -

context -

cg -

updateBatches

void updateBatches()
            throws SchemaChangeException

Updates the incoming (build and probe side) value vectors references in the HashTableTemplate.BatchHolders. This is useful on OK_NEW_SCHEMA (need to verify).

Throws:

SchemaChangeException

getBuildHashCode

int getBuildHashCode(int incomingRowIdx)
              throws SchemaChangeException

Computes the hash code for the record at the given index in the build side batch.

Parameters:

incomingRowIdx - The index of the build side record of interest.

Returns:

The hash code for the record at the given index in the build side batch.

Throws:

SchemaChangeException

getProbeHashCode

int getProbeHashCode(int incomingRowIdx)
              throws SchemaChangeException

Computes the hash code for the record at the given index in the probe side batch.

Parameters:

incomingRowIdx - The index of the probe side record of interest.

Returns:

The hash code for the record at the given index in the probe side batch.

Throws:

SchemaChangeException

put

HashTable.PutStatus put(int incomingRowIdx,
                        IndexPointer htIdxHolder,
                        int hashCode,
                        int batchSize)
                 throws SchemaChangeException,
                        RetryAfterSpillException

Throws:

SchemaChangeException

RetryAfterSpillException

probeForKey

int probeForKey(int incomingRowIdx,
                int hashCode)
         throws SchemaChangeException

Parameters:

incomingRowIdx - The index of the key in the probe batch.

hashCode - The hashCode of the key.

Returns:

Returns -1 if the data in the probe batch at the given incomingRowIdx is not in the hash table. Otherwise returns the composite index of the key in the hash table (index of BatchHolder and record in Batch Holder).

Throws:

SchemaChangeException

getStats

void getStats(HashTableStats stats)

size

int size()

isEmpty

boolean isEmpty()

clear

void clear()

Frees all the direct memory consumed by the HashTable.

updateInitialCapacity

void updateInitialCapacity(int initialCapacity)

Update the initial capacity for the hash table. This method will be removed after the key vectors are removed from the hash table. It is used to allocate HashTableTemplate.BatchHolders of appropriate size when the final size of the HashTable is known. Warning! Only call this method before you have inserted elements into the HashTable.

Parameters:

initialCapacity - The new initial capacity to use.

updateIncoming

void updateIncoming(VectorContainer newIncoming,
                    RecordBatch newIncomingProbe)

Changes the incoming probe and build side batches, and then updates all the value vector references in the HashTableTemplate.BatchHolders.

Parameters:

newIncoming - The new build side batch.

newIncomingProbe - The new probe side batch.

reset

void reset()

Clears all the memory used by the HashTable and re-initializes it.

outputKeys

boolean outputKeys(int batchIdx,
                   VectorContainer outContainer,
                   int numRecords)

Retrieves the key columns and transfers them to the output container. Note this operation removes the key columns from the HashTable.

Parameters:

batchIdx - The index of a HashTableTemplate.BatchHolder in the HashTable.

outContainer - The destination container for the key columns.

numRecords - The number of key recorts to transfer.

Returns:

makeDebugString

String makeDebugString()

Returns a message containing memory usage statistics. Intended to be used for printing debugging or error messages.

Returns:

A debug string.

getActualSize

long getActualSize()

The amount of direct memory consumed by the hash table.

Returns:

setTargetBatchRowCount

void setTargetBatchRowCount(int batchRowCount)

getTargetBatchRowCount

int getTargetBatchRowCount()

nextBatch

org.apache.commons.lang3.tuple.Pair<VectorContainer,Integer> nextBatch()