How to save 16bit depth Image to file in Arcore (java)

Question

I want to save the depth info from the arcore to the storage. Here is the example from the developer guide.

  public int getMillimetersDepth(Image depthImage, int x, int y) {
  // The depth image has a single plane, which stores depth for each
  // pixel as 16-bit unsigned integers.
  Image.Plane plane = depthImage.getPlanes()[0];
  int byteIndex = x * plane.getPixelStride() + y * plane.getRowStride();
  ByteBuffer buffer = plane.getBuffer().order(ByteOrder.nativeOrder());
  short depthSample = buffer.getShort(byteIndex);
  return depthSample;
}

So I want to save this bytebuffer into a local file, but my output txt file is not readable. How could I fix this? Here is what I have

Image depthImage = frame.acquireDepthImage();
Image.Plane plane = depthImage.getPlanes()[0];
int format = depthImage.getFormat();
ByteBuffer buffer = plane.getBuffer().order(ByteOrder.nativeOrder());
byte[] data = new byte[buffer.remaining()];
buffer.get(data);
File mypath=new File(super.getExternalFilesDir("depDir"),Long.toString(lastPointCloudTimestamp)+".txt");
FileChannel fc = new FileOutputStream(mypath).getChannel();
fc.write(buffer);
fc.close();
depthImage.close();

I tried to decode them with

String s = new String(data, "UTF-8");
System.out.println(StandardCharsets.UTF-8.decode(buffer).toString());

but the output is still strange like this

    .03579:<=>@ABCDFGHJKMNOPRQSUWY]_b

Answer 1

In order to obtain the depth data provided by the ARCore session, you need to write bytes into your local file. A Buffer object is a container , it countains a finite sequence of elements of a specific primitive type (here bytes for a ByteBuffer). So what you need to write inside your file is your data variable that corresponds to the information previously stored in the buffer (according to buffer.get(data) ).

It works fine for me, I managed to draw the provided depth map within a python code (but the idea behind the following code can be easily adapted to a java code):

depthData = np.fromfile('depthdata.txt', dtype = np.uint16) 
H = 120 
W = 160
def extractDepth(x):
    depthConfidence = (x >> 13) & 0x7 
    if (depthConfidence > 6): return 0 
    return x & 0x1FFF 
depthMap = np.array([extractDepth(x) for x in depthData]).reshape(H,W)
depthMap = cv.rotate(depthMap, cv.ROTATE_90_CLOCKWISE)

For further details, read the information concerning the format of the depthmap (DEPTH16) here: https://developer.android.com/reference/android/graphics/ImageFormat#DEPTH16 You must also be aware that the depthmap resolution is set to 160x120 pixels and is oriented according to a landscape format.

Make also sure to surround your code by a try/catch code bloc in case of a IOException error.