ValueVector (Drill : 1.20.3 API)

All Superinterfaces:

AutoCloseable, Closeable, Iterable<ValueVector>

All Known Subinterfaces:

FixedWidthVector, NullableVector, RepeatedValueVector, VariableWidthVector

All Known Implementing Classes:

AbstractContainerVector, AbstractMapVector, AbstractRepeatedMapVector, BaseDataValueVector, BaseRepeatedValueVector, BaseValueVector, BigIntVector, BitVector, DateVector, Decimal18Vector, Decimal28DenseVector, Decimal28SparseVector, Decimal38DenseVector, Decimal38SparseVector, Decimal9Vector, DictVector, Float4Vector, Float8Vector, IntervalDayVector, IntervalVector, IntervalYearVector, IntVector, ListVector, MapVector, NullableBigIntVector, NullableBitVector, NullableDateVector, NullableDecimal18Vector, NullableDecimal28DenseVector, NullableDecimal28SparseVector, NullableDecimal38DenseVector, NullableDecimal38SparseVector, NullableDecimal9Vector, NullableFloat4Vector, NullableFloat8Vector, NullableIntervalDayVector, NullableIntervalVector, NullableIntervalYearVector, NullableIntVector, NullableSmallIntVector, NullableTimeStampVector, NullableTimeVector, NullableTinyIntVector, NullableUInt1Vector, NullableUInt2Vector, NullableUInt4Vector, NullableUInt8Vector, NullableVar16CharVector, NullableVarBinaryVector, NullableVarCharVector, NullableVarDecimalVector, ObjectVector, RepeatedBigIntVector, RepeatedBitVector, RepeatedDateVector, RepeatedDecimal18Vector, RepeatedDecimal28DenseVector, RepeatedDecimal28SparseVector, RepeatedDecimal38DenseVector, RepeatedDecimal38SparseVector, RepeatedDecimal9Vector, RepeatedDictVector, RepeatedFloat4Vector, RepeatedFloat8Vector, RepeatedIntervalDayVector, RepeatedIntervalVector, RepeatedIntervalYearVector, RepeatedIntVector, RepeatedListVector, RepeatedListVector.DelegateRepeatedVector, RepeatedMapVector, RepeatedSmallIntVector, RepeatedTimeStampVector, RepeatedTimeVector, RepeatedTinyIntVector, RepeatedUInt1Vector, RepeatedUInt2Vector, RepeatedUInt4Vector, RepeatedUInt8Vector, RepeatedVar16CharVector, RepeatedVarBinaryVector, RepeatedVarCharVector, RepeatedVarDecimalVector, SmallIntVector, TimeStampVector, TimeVector, TinyIntVector, UInt1Vector, UInt2Vector, UInt4Vector, UInt8Vector, UnionVector, UntypedNullVector, Var16CharVector, VarBinaryVector, VarCharVector, VarDecimalVector, VectorAccessibleComplexWriter, ZeroVector
```
public interface ValueVector
extends Closeable, Iterable<ValueVector>
```
An abstraction that is used to store a sequence of values in an individual column. A value vector stores underlying data in-memory in a columnar fashion that is compact and efficient. The column whose data is stored, is referred by getField(). A vector when instantiated, relies on a dead buffer. It is important that vector is allocated before attempting to read or write. There are a few "rules" around vectors:
- Values need to be written in order (e.g. index 0, 1, 2, 5).
- Null vectors start with all values as null before writing anything.
- For variable width types, the offset vector should be all zeros before writing.
- You must call setValueCount before a vector can be read.
- You should never write to a vector once it has been read.
- Vectors may not grow larger than the number of bytes specified in MAX_BUFFER_SIZE to prevent memory fragmentation. Use the setBounded() methods in the mutator to enforce this rule.
Please note that the current implementation doesn't enforce those rules, hence we may find few places that deviate from these rules (e.g. offset vectors in Variable Length and Repeated vector) This interface "should" strive to guarantee this order of operation:
allocate > mutate > setvaluecount > access > clear (or allocate to start the process over).

Nested Class Summary

Nested Classes
Modifier and Type	Interface and Description
`static interface`	`ValueVector.Accessor` Reads from this vector instance.
`static interface`	`ValueVector.Mutator` Writes into this vector instance.

Field Summary

Fields
Modifier and Type	Field and Description
`static String`	`BITS_VECTOR_NAME`
`static int`	`MAX_BUFFER_SIZE` Maximum allowed size of the buffer backing a value vector.
`static int`	`MAX_ROW_COUNT` Maximum allowed row count in a vector.
`static int`	`MIN_ROW_COUNT`
`static String`	`OFFSETS_VECTOR_NAME`
`static String`	`VALUES_VECTOR_NAME` Deprecated.

Method Summary

All Methods Instance Methods Abstract Methods
Modifier and Type	Method and Description
`void`	`allocateNew()` Allocate new buffers.
`boolean`	`allocateNewSafe()` Allocates new buffers.
`void`	`clear()` Release the underlying DrillBuf and reset the ValueVector to empty.
`void`	`close()` Alternative to clear().
`void`	`collectLedgers(Set<AllocationManager.BufferLedger> ledgers)` Add the ledgers underlying the buffers underlying the components of the vector to the set provided.
`void`	`copyEntry(int toIndex, ValueVector from, int fromIndex)`
`void`	`exchange(ValueVector other)` Exchange state with another value vector of the same type.
`ValueVector.Accessor`	`getAccessor()` Returns an `accessor` that is used to read from this vector instance.
`int`	`getAllocatedSize()` Returns the total size of buffers allocated by this vector.
`BufferAllocator`	`getAllocator()`
`DrillBuf[]`	`getBuffers(boolean clear)` Return the underlying buffers associated with this vector.
`int`	`getBufferSize()` Returns the number of bytes that is used by this vector instance.
`int`	`getBufferSizeFor(int valueCount)` Returns the number of bytes that is used by this vector if it holds the given number of values.
`MaterializedField`	`getField()` Get information about how this field is materialized.
`UserBitShared.SerializedField`	`getMetadata()` Get the metadata for this field.
`ValueVector.Mutator`	`getMutator()` Returns an `mutator` that is used to write to this vector instance.
`int`	`getPayloadByteCount(int valueCount)` Return the number of value bytes consumed by actual data.
`FieldReader`	`getReader()` Returns a `field reader` that supports reading values from this vector.
`TransferPair`	`getTransferPair(BufferAllocator allocator)` Returns a `transfer pair`, creating a new target vector of the same type.
`TransferPair`	`getTransferPair(String ref, BufferAllocator allocator)`
`int`	`getValueCapacity()` Returns the maximum number of values that can be stored in this vector instance.
`void`	`load(UserBitShared.SerializedField metadata, DrillBuf buffer)` Load the data provided in the buffer.
`TransferPair`	`makeTransferPair(ValueVector target)` Returns a new `transfer pair` that is used to transfer underlying buffers into the target vector.
`void`	`setInitialCapacity(int numRecords)` Set the initial record capacity
`void`	`toNullable(ValueVector nullableVector)` Convert a non-nullable vector to nullable by shuffling the data from one to the other.

Methods inherited from interface java.lang.Iterable
forEach, iterator, spliterator

- Field Detail
  - MAX_BUFFER_SIZE
```
static final int MAX_BUFFER_SIZE
```
    Maximum allowed size of the buffer backing a value vector. Set to the Netty chunk size to prevent memory fragmentation.
  - MAX_ROW_COUNT
```
static final int MAX_ROW_COUNT
```
    Maximum allowed row count in a vector. Repeated vectors may have more items, but can have no more than this number or arrays. Limited by 2-byte length in SV2: 65536 = 2¹⁶.
    
    See Also:
    
    Constant Field Values
  - MIN_ROW_COUNT
```
static final int MIN_ROW_COUNT
```
    See Also:
    
    Constant Field Values
  - BITS_VECTOR_NAME
```
static final String BITS_VECTOR_NAME
```
    See Also:
    
    Constant Field Values
  - OFFSETS_VECTOR_NAME
```
static final String OFFSETS_VECTOR_NAME
```
    See Also:
    
    Constant Field Values
  - VALUES_VECTOR_NAME
```
@Deprecated
static final String VALUES_VECTOR_NAME
```
    Deprecated.
    
    See Also:
    
    Constant Field Values
- Method Detail
  - allocateNew
```
void allocateNew()
          throws OutOfMemoryException
```
    Allocate new buffers. ValueVector implements logic to determine how much to allocate.
    
    Throws:
    
    OutOfMemoryException - Thrown if no memory can be allocated.
  - allocateNewSafe
```
boolean allocateNewSafe()
```
    Allocates new buffers. ValueVector implements logic to determine how much to allocate.
    
    Returns:
    
    Returns true if allocation was successful.
  - getAllocator
```
BufferAllocator getAllocator()
```
  - setInitialCapacity
```
void setInitialCapacity(int numRecords)
```
    Set the initial record capacity
    
    Parameters:
    
    numRecords -
  - getValueCapacity
```
int getValueCapacity()
```
    Returns the maximum number of values that can be stored in this vector instance.
  - close
```
void close()
```
    Alternative to clear(). Allows use as an AutoCloseable in try-with-resources.
    
    Specified by:
    
    close in interface AutoCloseable
    
    Specified by:
    
    close in interface Closeable
  - clear
```
void clear()
```
    Release the underlying DrillBuf and reset the ValueVector to empty.
  - getField
```
MaterializedField getField()
```
    Get information about how this field is materialized.
  - getTransferPair
```
TransferPair getTransferPair(BufferAllocator allocator)
```
    Returns a transfer pair, creating a new target vector of the same type.
  - getTransferPair
```
TransferPair getTransferPair(String ref,
                             BufferAllocator allocator)
```
  - makeTransferPair
```
TransferPair makeTransferPair(ValueVector target)
```
    Returns a new transfer pair that is used to transfer underlying buffers into the target vector.
  - getAccessor
```
ValueVector.Accessor getAccessor()
```
    Returns an accessor that is used to read from this vector instance.
  - getMutator
```
ValueVector.Mutator getMutator()
```
    Returns an mutator that is used to write to this vector instance.
  - getReader
```
FieldReader getReader()
```
    Returns a field reader that supports reading values from this vector.
  - getMetadata
```
UserBitShared.SerializedField getMetadata()
```
    Get the metadata for this field. Used in serialization
    
    Returns:
    
    FieldMetadata for this field.
  - getBufferSize
```
int getBufferSize()
```
    Returns the number of bytes that is used by this vector instance. This is a bit of a misnomer. Returns the number of bytes used by data in this instance.
  - getAllocatedSize
```
int getAllocatedSize()
```
    Returns the total size of buffers allocated by this vector. Has meaning only when vectors are directly allocated and each vector has its own buffer. Does not have meaning for vectors deserialized from the network or disk in which multiple vectors share the same vector.
    
    Returns:
    
    allocated buffer size, in bytes
  - getBufferSizeFor
```
int getBufferSizeFor(int valueCount)
```
    Returns the number of bytes that is used by this vector if it holds the given number of values. The result will be the same as if Mutator.setValueCount() were called, followed by calling getBufferSize(), but without any of the closing side-effects that setValueCount() implies wrt finishing off the population of a vector. Some operations might wish to use this to determine how much memory has been used by a vector so far, even though it is not finished being populated.
    
    Parameters:
    
    valueCount - the number of values to assume this vector contains
    
    Returns:
    
    the buffer size if this vector is holding valueCount values
  - getBuffers
```
DrillBuf[] getBuffers(boolean clear)
```
    Return the underlying buffers associated with this vector. Note that this doesn't impact the reference counts for this buffer so it only should be used for in-context access. Also note that this buffer changes regularly thus external classes shouldn't hold a reference to it (unless they change it).
    
    Parameters:
    
    clear - Whether to clear vector before returning; the buffers will still be refcounted; but the returned array will be the only reference to them
    
    Returns:
    
    The underlying buffers that is used by this vector instance.
  - load
```
void load(UserBitShared.SerializedField metadata,
          DrillBuf buffer)
```
    Load the data provided in the buffer. Typically used when deserializing from the wire.
    
    Parameters:
    
    metadata - Metadata used to decode the incoming buffer.
    
    buffer - The buffer that contains the ValueVector.
  - copyEntry
```
void copyEntry(int toIndex,
               ValueVector from,
               int fromIndex)
```
  - collectLedgers
```
void collectLedgers(Set<AllocationManager.BufferLedger> ledgers)
```
    Add the ledgers underlying the buffers underlying the components of the vector to the set provided. Used to determine actual memory allocation.
    
    Parameters:
    
    ledgers - set of ledgers to which to add ledgers for this vector
  - getPayloadByteCount
```
int getPayloadByteCount(int valueCount)
```
    Return the number of value bytes consumed by actual data.
  - exchange
```
void exchange(ValueVector other)
```
    Exchange state with another value vector of the same type. Used to implement look-ahead writers.
  - toNullable
```
void toNullable(ValueVector nullableVector)
```
    Convert a non-nullable vector to nullable by shuffling the data from one to the other. Avoids the need to generate copy code just to change mode. If this vector is non-nullable, accepts a nullable dual (same minor type, different mode.) If the vector is non-nullable, or non-scalar, then throws an exception.
    
    Parameters:
    
    nullableVector - nullable vector of the same minor type as this vector

Interface ValueVector

Nested Class Summary

Field Summary

Method Summary

Methods inherited from interface java.lang.Iterable

Field Detail

MAX_BUFFER_SIZE

MAX_ROW_COUNT

MIN_ROW_COUNT

BITS_VECTOR_NAME

OFFSETS_VECTOR_NAME

VALUES_VECTOR_NAME

Method Detail

allocateNew

allocateNewSafe

getAllocator

setInitialCapacity

getValueCapacity

close

clear

getField

getTransferPair

getTransferPair

makeTransferPair

getAccessor

getMutator

getReader

getMetadata

getBufferSize

getAllocatedSize

getBufferSizeFor

getBuffers

load

copyEntry

collectLedgers

getPayloadByteCount

exchange

toNullable