从 java 中的箭头文件读取 float16 数据类型列的正确方法是什么？

Question

I am trying to establish an IPC pipeline. I have a python program which is writing an.arrow file and saving it in memory which gets picked up by the Java application. This application reads the file schema and does relevant operations. Currently I am having trouble with a column that is of float16 datatype . So my python program writes this column something like this:

# sample
float16column= pa.array(([np.float16(0) for _ in range(5)]), type=type_float16)
item_table = pa.table([float16column], ['samplefloat16columnname'])
local = fs.LocalFileSystem()
with local.open_output_stream("output.arrow") as file:
    with pa.RecordBatchFileWriter(file, table.schema) as writer:
        writer.write_table(table)

Now when I try to read this file from java application (and mind you this is the only column throwing error) using the below program

    public void read(String path) throws IOException {
    File arrowFile = new File(path);
    FileInputStream fileInputStream = new FileInputStream(arrowFile);
    SeekableReadChannel seekableReadChannel = new SeekableReadChannel(fileInputStream.getChannel());
    ArrowFileReader arrowFileReader = new ArrowFileReader(seekableReadChannel,
            new RootAllocator(Integer.MAX_VALUE));
    List<ArrowBlock> arrowBlocks = arrowFileReader.getRecordBlocks();
    for (int i = 0; i < arrowBlocks.size(); i++) {
        ArrowBlock rbBlock = arrowBlocks.get(i);
        if (!arrowFileReader.loadRecordBatch(rbBlock)) { // load the batch
            throw new IOException("Expected to read record batch");
        }
        // do something with the loaded batch

    }
}

I see this error:

Exception in thread "main" java.lang.UnsupportedOperationException: NYI: FloatingPoint(HALF)

Now I am not very proficient in java, but I am guessing this may have something to do with incompatible data types of both. Does anyone else know the correct of way of doing this?

ps: reading the same arrow file using python seems to be working fine

Answer 1

The smallest floating point type supported in Java is float, which is 4 bytes and is represented in Arrow Java as float4.