org.apache.drill.exec.vector

Class VarCharVector

java.lang.Object

org.apache.drill.exec.vector.BaseValueVector

org.apache.drill.exec.vector.BaseDataValueVector

org.apache.drill.exec.vector.VarCharVector

All Implemented Interfaces:

Closeable, AutoCloseable, Iterable<ValueVector>, ValueVector, VariableWidthVector

public final class VarCharVector
extends BaseDataValueVector
implements VariableWidthVector

VarCharVector implements a vector of variable width values. Elements in the vector are accessed by position from the logical start of the vector. A fixed width offsetVector is used to convert an element's position to it's offset from the start of the (0-based) DrillBuf. Size is inferred from adjacent elements.

• The width of each element is 4 byte(s). Note that the actual width is variable, this width is used as a guess for certain calculations.
• The equivalent Java primitive is 'int'

NB: this class is automatically generated from VariableLengthVectors.java and ValueVectorTypes.tdd using FreeMarker.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	VarCharVector.Accessor
static class	VarCharVector.BufferedMutator Helper class to buffer container mutation as a means to optimize native memory copy operations.
class	VarCharVector.Mutator Overview

Nested classes/interfaces inherited from class org.apache.drill.exec.vector.BaseValueVector

BaseValueVector.BaseAccessor, BaseValueVector.BaseMutator

Nested classes/interfaces inherited from interface org.apache.drill.exec.vector.VariableWidthVector

VariableWidthVector.VariableWidthAccessor, VariableWidthVector.VariableWidthMutator

Field Summary

Fields inherited from class org.apache.drill.exec.vector.BaseDataValueVector

data, emptyByteArray

Fields inherited from class org.apache.drill.exec.vector.BaseValueVector

allocator, field, INITIAL_VALUE_ALLOCATION, MAX_ALLOCATION_SIZE

Fields inherited from interface org.apache.drill.exec.vector.VariableWidthVector

DEFAULT_RECORD_BYTE_COUNT, MIN_BYTE_COUNT, offsetsField

Fields inherited from interface org.apache.drill.exec.vector.ValueVector

BITS_VECTOR_NAME, MAX_BUFFER_SIZE, MAX_ROW_COUNT, MIN_ROW_COUNT, OFFSETS_VECTOR_NAME, VALUES_VECTOR_NAME

Constructor Summary

Constructors

Constructor and Description
VarCharVector(MaterializedField field, BufferAllocator allocator)

Method Summary

Modifier and Type	Method and Description
void	allocateNew() Allocate new buffers.
void	allocateNew(int totalBytes, int valueCount) Allocate a new memory space for this vector.
boolean	allocateNewSafe() Allocates new buffers.
void	clear() Release the underlying DrillBuf and reset the ValueVector to empty.
void	collectLedgers(Set<AllocationManager.BufferLedger> ledgers) Add the ledgers underlying the buffers underlying the components of the vector to the set provided.
void	copyEntry(int toIndex, ValueVector from, int fromIndex)
protected void	copyFrom(int fromIndex, int thisIndex, VarCharVector from)
void	copyFromSafe(int fromIndex, int thisIndex, VarCharVector from)
void	decrementAllocationMonitor()
void	exchange(ValueVector other) Exchange state with another value vector of the same type.
VarCharVector.Accessor	getAccessor() Returns an accessor that is used to read from this vector instance.
int	getAllocatedSize() Returns the total size of buffers allocated by this vector.
DrillBuf[]	getBuffers(boolean clear) Return the underlying buffers associated with this vector.
int	getBufferSize() Returns the number of bytes that is used by this vector instance.
int	getBufferSizeFor(int valueCount) Returns the number of bytes that is used by this vector if it holds the given number of values.
int	getByteCapacity() Provide the maximum amount of variable width bytes that can be stored int his vector.
int	getCurrentSizeInBytes() Return the number of bytes contained in the current var len byte vector.
UserBitShared.SerializedField	getMetadata() Get the metadata for this field.
VarCharVector.Mutator	getMutator() Returns an mutator that is used to write to this vector instance.
long	getOffsetAddr()
UInt4Vector	getOffsetVector()
int	getPayloadByteCount(int valueCount) Return the number of value bytes consumed by actual data.
FieldReader	getReader() Returns a field reader that supports reading values from this vector.
TransferPair	getTransferPair(BufferAllocator allocator) Returns a transfer pair, creating a new target vector of the same type.
TransferPair	getTransferPair(String ref, BufferAllocator allocator)
int	getValueCapacity() Returns the maximum number of values that can be stored in this vector instance.
int	getVarByteLength() Return the number of bytes contained in the current var len byte vector.
void	load(UserBitShared.SerializedField metadata, DrillBuf buffer) Load the data provided in the buffer.
TransferPair	makeTransferPair(ValueVector to) Returns a new transfer pair that is used to transfer underlying buffers into the target vector.
void	reAlloc()
DrillBuf	reallocRaw(int newAllocationSize) Core of vector allocation.
void	reset() This method has a similar effect of allocateNew() without actually clearing and reallocating the value vector.
void	setInitialCapacity(int valueCount) Set the initial record capacity
void	splitAndTransferTo(int startIndex, int length, VarCharVector target)
void	toNullable(ValueVector nullableVector) Convert a non-nullable vector to nullable by shuffling the data from one to the other.
void	transferTo(VarCharVector target)

Methods inherited from class org.apache.drill.exec.vector.BaseDataValueVector

close, getBuffer

Methods inherited from class org.apache.drill.exec.vector.BaseValueVector

checkBufRefs, fillBitsVector, getAllocator, getField, getField, getMetadataBuilder, iterator, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.drill.exec.vector.ValueVector

close, getAllocator, getField

Methods inherited from interface java.lang.Iterable

forEach, iterator, spliterator

Constructor Detail

VarCharVector

public VarCharVector(MaterializedField field,
                     BufferAllocator allocator)

Method Detail

getReader

public FieldReader getReader()

Description copied from interface: ValueVector

Returns a field reader that supports reading values from this vector.

Specified by:

getReader in interface ValueVector

getBufferSize

public int getBufferSize()

Description copied from interface: ValueVector

Returns the number of bytes that is used by this vector instance. This is a bit of a misnomer. Returns the number of bytes used by data in this instance.

Specified by:

getBufferSize in interface ValueVector

Overrides:

getBufferSize in class BaseDataValueVector

getAllocatedSize

public int getAllocatedSize()

Description copied from interface: ValueVector

Returns the total size of buffers allocated by this vector. Has meaning only when vectors are directly allocated and each vector has its own buffer. Does not have meaning for vectors deserialized from the network or disk in which multiple vectors share the same vector.

Specified by:

getAllocatedSize in interface ValueVector

Overrides:

getAllocatedSize in class BaseDataValueVector

Returns:

allocated buffer size, in bytes

getBufferSizeFor

public int getBufferSizeFor(int valueCount)

Description copied from interface: ValueVector

Returns the number of bytes that is used by this vector if it holds the given number of values. The result will be the same as if Mutator.setValueCount() were called, followed by calling getBufferSize(), but without any of the closing side-effects that setValueCount() implies wrt finishing off the population of a vector. Some operations might wish to use this to determine how much memory has been used by a vector so far, even though it is not finished being populated.

Specified by:

getBufferSizeFor in interface ValueVector

Parameters:

valueCount - the number of values to assume this vector contains

Returns:

the buffer size if this vector is holding valueCount values

getValueCapacity

public int getValueCapacity()

Description copied from interface: ValueVector

Returns the maximum number of values that can be stored in this vector instance.

Specified by:

getValueCapacity in interface ValueVector

getByteCapacity

public int getByteCapacity()

Description copied from interface: VariableWidthVector

Provide the maximum amount of variable width bytes that can be stored int his vector.

Specified by:

getByteCapacity in interface VariableWidthVector

Returns:

getCurrentSizeInBytes

public int getCurrentSizeInBytes()

Return the number of bytes contained in the current var len byte vector. TODO: Remove getVarByteLength with it's implementation after all client's are moved to using getCurrentSizeInBytes. It's kept as is to preserve backward compatibility

Specified by:

getCurrentSizeInBytes in interface VariableWidthVector

Returns:

getVarByteLength

public int getVarByteLength()

Return the number of bytes contained in the current var len byte vector.

Returns:

getMetadata

public UserBitShared.SerializedField getMetadata()

Description copied from interface: ValueVector

Get the metadata for this field. Used in serialization

Specified by:

getMetadata in interface ValueVector

Overrides:

getMetadata in class BaseValueVector

Returns:

FieldMetadata for this field.

load

public void load(UserBitShared.SerializedField metadata,
                 DrillBuf buffer)

Description copied from interface: ValueVector

Load the data provided in the buffer. Typically used when deserializing from the wire.

Specified by:

load in interface ValueVector

Parameters:

metadata - Metadata used to decode the incoming buffer.

buffer - The buffer that contains the ValueVector.

clear

public void clear()

Description copied from interface: ValueVector

Release the underlying DrillBuf and reset the ValueVector to empty.

Specified by:

clear in interface ValueVector

Overrides:

clear in class BaseDataValueVector

getBuffers

public DrillBuf[] getBuffers(boolean clear)

Description copied from interface: ValueVector

Return the underlying buffers associated with this vector. Note that this doesn't impact the reference counts for this buffer so it only should be used for in-context access. Also note that this buffer changes regularly thus external classes shouldn't hold a reference to it (unless they change it).

Specified by:

getBuffers in interface ValueVector

Overrides:

getBuffers in class BaseDataValueVector

Parameters:

clear - Whether to clear vector before returning; the buffers will still be refcounted; but the returned array will be the only reference to them

Returns:

The underlying buffers that is used by this vector instance.

getOffsetAddr

public long getOffsetAddr()

getOffsetVector

public UInt4Vector getOffsetVector()

Specified by:

getOffsetVector in interface VariableWidthVector

getTransferPair

public TransferPair getTransferPair(BufferAllocator allocator)

Description copied from interface: ValueVector

Returns a transfer pair, creating a new target vector of the same type.

Specified by:

getTransferPair in interface ValueVector

Overrides:

getTransferPair in class BaseValueVector

getTransferPair

public TransferPair getTransferPair(String ref,
                                    BufferAllocator allocator)

Specified by:

getTransferPair in interface ValueVector

makeTransferPair

public TransferPair makeTransferPair(ValueVector to)

Description copied from interface: ValueVector

Returns a new transfer pair that is used to transfer underlying buffers into the target vector.

Specified by:

makeTransferPair in interface ValueVector

transferTo

public void transferTo(VarCharVector target)

splitAndTransferTo

public void splitAndTransferTo(int startIndex,
                               int length,
                               VarCharVector target)

copyFrom

protected void copyFrom(int fromIndex,
                        int thisIndex,
                        VarCharVector from)

copyFromSafe

public void copyFromSafe(int fromIndex,
                         int thisIndex,
                         VarCharVector from)

copyEntry

public void copyEntry(int toIndex,
                      ValueVector from,
                      int fromIndex)

Specified by:

copyEntry in interface ValueVector

collectLedgers

public void collectLedgers(Set<AllocationManager.BufferLedger> ledgers)

Description copied from interface: ValueVector

Add the ledgers underlying the buffers underlying the components of the vector to the set provided. Used to determine actual memory allocation.

Specified by:

collectLedgers in interface ValueVector

Overrides:

collectLedgers in class BaseDataValueVector

Parameters:

ledgers - set of ledgers to which to add ledgers for this vector

getPayloadByteCount

public int getPayloadByteCount(int valueCount)

Description copied from interface: ValueVector

Return the number of value bytes consumed by actual data.

Specified by:

getPayloadByteCount in interface ValueVector

setInitialCapacity

public void setInitialCapacity(int valueCount)

Description copied from interface: ValueVector

Set the initial record capacity

Specified by:

setInitialCapacity in interface ValueVector

allocateNew

public void allocateNew()

Description copied from interface: ValueVector

Allocate new buffers. ValueVector implements logic to determine how much to allocate.

Specified by:

allocateNew in interface ValueVector

allocateNewSafe

public boolean allocateNewSafe()

Description copied from interface: ValueVector

Allocates new buffers. ValueVector implements logic to determine how much to allocate.

Specified by:

allocateNewSafe in interface ValueVector

Returns:

Returns true if allocation was successful.

allocateNew

public void allocateNew(int totalBytes,
                        int valueCount)

Description copied from interface: VariableWidthVector

Allocate a new memory space for this vector. Must be called prior to using the ValueVector.

Specified by:

allocateNew in interface VariableWidthVector

Parameters:

totalBytes - Desired size of the underlying data buffer.

valueCount - Number of values in the vector.

reset

public void reset()

Description copied from class: BaseDataValueVector

This method has a similar effect of allocateNew() without actually clearing and reallocating the value vector. The purpose is to move the value vector to a "mutate" state

Overrides:

reset in class BaseDataValueVector

reAlloc

public void reAlloc()

reallocRaw

public DrillBuf reallocRaw(int newAllocationSize)

Description copied from class: BaseDataValueVector

Core of vector allocation. Given a new size (which must be a power of two), allocate the new buffer, copy the current values, and leave the unused parts garbage-filled.

Specified by:

reallocRaw in class BaseDataValueVector

Parameters:

newAllocationSize - new buffer size as a power of two

Returns:

the new buffer

decrementAllocationMonitor

public void decrementAllocationMonitor()

getAccessor

public VarCharVector.Accessor getAccessor()

Description copied from interface: ValueVector

Returns an accessor that is used to read from this vector instance.

Specified by:

getAccessor in interface ValueVector

Specified by:

getAccessor in interface VariableWidthVector

getMutator

public VarCharVector.Mutator getMutator()

Description copied from interface: ValueVector

Returns an mutator that is used to write to this vector instance.

Specified by:

getMutator in interface ValueVector

Specified by:

getMutator in interface VariableWidthVector

exchange

public void exchange(ValueVector other)

Description copied from interface: ValueVector

Exchange state with another value vector of the same type. Used to implement look-ahead writers.

Specified by:

exchange in interface ValueVector

Overrides:

exchange in class BaseDataValueVector

toNullable

public void toNullable(ValueVector nullableVector)

Description copied from interface: ValueVector

Convert a non-nullable vector to nullable by shuffling the data from one to the other. Avoids the need to generate copy code just to change mode. If this vector is non-nullable, accepts a nullable dual (same minor type, different mode.) If the vector is non-nullable, or non-scalar, then throws an exception.

Specified by:

toNullable in interface ValueVector

Overrides:

toNullable in class BaseValueVector

Parameters:

nullableVector - nullable vector of the same minor type as this vector