How to create Apache Arrow vectors for writing to a Table in TypeScript

Question

I am trying to write a method that takes a column of data, data: any[] and packs it into an Arrow-typed Array Buffer to put into an Arrow table.

For example, if I have Type.Float64, I can do this:

Float64Vector.from(toFloat64Array(data));

But for Int64Vector , TimeSecondVector , or TimestampVector , I'm getting errors:

TimestampVector.from(data)
error: Type 'AbstractVector<any>' is missing the following properties from type 'BaseVector<any>': offset, VectorName, values, typeIds, and 7 more.ts(2740)

DecimalVector.from(data), TimeSecondVector.from(data), and BinaryVector.from(data)
Type 'AbstractVector<any>' is not assignable to type 'BaseVector<any>'.ts(2322)

I have been combing the documentation and I cannot find anything that explains how to use this. I just want to know how I can build these types of Vectors:

TimeSecondVector
TimestampVector

Answer 1

It turns out Apache Arrow provides Builders ! Great doc about builders

Also, do not trust this:

Float64Vector.from(toFloat64Array(data));

for any of the data types. Builders seem to be the way to go, until have to start dealing with Uint32Arrays (Decimal and Binary? That's in another question).

export const createTimeSecondVector = (data: number[]): TimeSecondVector => {
  const builder = TimeSecondBuilder.new({
    type: new TimeSecond(),
    nullValues: [null, undefined]
  });
  data.forEach(d => {
    builder.append(d);
  });
  return builder.finish().toVector();
};

export const createTimestampVector = (data: number[]): TimestampVector => {
  const builder = TimestampBuilder.new({
    type: new Timestamp(TimeUnit.SECOND, 'America/Denver'),
    nullValues: [null, undefined]
  });
  data.forEach(d => {
    builder.append(d);
  });
  return builder.finish().toVector();
};