Passing a base64 string encoded image/ byte image as an image for processsing in Firebase ML Vision in Flutter

Question

I want to OCR text from a base64 encoded image. I know the image works because I can display it using

Image.memory(base64Decode(captchaEncodedImgFetched))

Now, the problem is I need to pass this image to Firebase ML Vision for processing.

The library firebase_ml_vision has an example for using a image from file

final File imageFile = getImageFile();
final FirebaseVisionImage visionImage = FirebaseVisionImage.fromFile(imageFile);`

However I have a base64 encoded image.

I tried the following

final FirebaseVisionImage visionImage = FirebaseVisionImage.fromBytes(
base64Decode(captchaEncodedImgFetched));

But it seems to need FirebaseVisionImageMetadata() as a argument, but I know nothing about byte images.

This class needs a lot more arguments which I don't understand. For example, it needs a size: Size(width, height) argument. Isn't the image supposed to have a size already? Why do I need to specify it again?

For now I set it to Size(200, 50) . Then there are the other arugments and I don't know what to pass to them. For exmaple the planeData and rawFormat .

Here are the docs for these:

https://pub.dev/documentation/firebase_ml_vision/latest/firebase_ml_vision/FirebaseVisionImageMetadata-class.html

https://pub.dev/documentation/firebase_ml_vision/latest/firebase_ml_vision/FirebaseVisionImagePlaneMetadata-class.html

https://pub.dev/documentation/firebase_ml_vision/latest/

Answer 1

FirebaseVisionImage.fromBytes needs FirebaseVisionImageMetadata which intern needs FirebaseVisionImagePlaneMetadata . Example below:

// Below example uses metadata values based on an RGBA-encoded 1080x1080 image
final planeMetadata = FirebaseVisionImagePlaneMetadata(
    width: 1080,
    height: 1080,
    bytesPerRow: 1080 * 4,
);

final imageMetadata = FirebaseVisionImageMetadata(
    size: Size(1080, 1080),
    planeData: planeMetadata,
    rawFormat: 'RGBA', 
);

final visionImage = FirebaseVisionImage.fromBytes(decoded, metadata);

The simpler workaround though at a cost of performance is to write the bytes to the disk and read the image from there, as:

File imgFile = File('myimage.png');
imageFile.writeAsBytesSync(decoded.ToList());

final visionImage = FirebaseVisionImage.fromFile(imageFile);