EasyAR access Camera Frames as OpenCV Mat

Question

I'm using EasyAR to develop an app on android using C++ & I'm trying to use opencv with it, what I'm trying to achieve is: get the easyAR frames that it got from the camera as Mat and do some processing using opencv then return the frames to view.

Why do all that? simply I'm only after the EasyAR camera frame crossplatform access (I think it's really fast, I just built the sample HelloAR)

in the Sample HelloAR, there is a line auto frame = streamer->peek();

1. is there is a way to convert this to be used in openCV ?

2. is there is an alternative way to access camera frames from c++ in both IOS & Android (Min API 16)?

your help is appreciated, thank you.

here is the samples link, I'm using HelloAR http://s3-us-west-2.amazonaws.com/easyar/sdk/EasyAR_SDK_2.0.0_Basic_Samples_Android_2017-05-29.tar.xz

Answer 1

Okay, I managed to get a solution for this so simply frame (class Frame in EasyAR) contains a vector of images (probably different images for the same frame), accessing that vector returns an Image object with a method called data (a byte array) and that can be used to initialize a Mat in opencv

here is the code to clarify for anyone searching for the same

unsigned char* imageBuffer = static_cast<unsigned char*>(frame->images().at(0)->data());

int height = frame->images()[0]->height(); // height of the image
int width = frame->images()[0]->width(); // width of image
// Obtained Frame is YUV21 by default, so convert that to RGBA
cv::Mat _yuv(height+height/2, width, CV_8UC1, imageBuffer);
cv::cvtColor(_yuv, _yuv, CV_YUV2RGBA_NV21);