创建检测卡片值的 ML 模型

Question

This is a more generic question about training an ML-Model to detect cards.

The cards are a kid's game, 4 different colors, numbers and symbols. I don't need to detect the color, just the value (aka symbol) of the cards.

I tried to take pictures with my iPhone of every card, used RectLabel to draw the rectangles around the symbols in the upper left corner (the cards have an upside down-symbol in the lower right corner, too, I didn't mark these as they'll be hidden during detection). I cropped the images so only the card is visible, no surroundings.

Then I uploaded my images to app.roboflow.ai and let them do their magic (using Auto-Orient, Resize to 416x416, Grayscale, Auto-Adjust Contrast, Rotation, Shear, Blur and Noise).

That gave me another set of images which I used to train my model with CreateML from Apple.

However, when I use that model in my app (I'm using the Breakfast Finder Demo from Apple), the cards values aren't detected - well, sometimes it works, but only at a certain distance from the phone and the labels are either upside down or sideways.

My guess is this is because my images aren't taken the way they should be?

Any hints on how I'd have to set this whole thing up so my model gets trained well?

Answer 1

My bet would be on this being the problem:

I cropped the images so only the card is visible, no surroundings

You want your training images to be as similar as possible to the images your model will see in the wild. If it's trained only on images of cards with no surroundings and then you show it images of cards with things around them it won't know what to do.

This UNO scoring example is extremely similar to your problem and might provide some ideas and guidance.