Dynamically Crop image with JS

Question

I'd like to extract a conversation for a game with JS (plain) and get just the text bubbles. This isn't that hard just to crop and image, but sadly the image's dimensions and ratios is going to change. Here's the image before and after

Original (messages removed):

And then the auto cropped:

How do I crop this? All the text is preserved but nothing else is in it but the text (I need this because I'm converting it to text data via tesseract and it gets confused on some other things in the image). My goal is to have this entirely client side so no PHP could be used. The image ratio and dimensions are also going to change. It's always going to be landscape but some are practically square while others are really long and short. Is this possible. Could I use some sort of AI model to capture it? Any help is greatly appreciated. Thanks!

Edit

This game is Brawl Stars avaivible on iOS and the Play Store. Tapping system would be very hard, it's made by Supercell which in my experience has their apps fairly well locked down. All apis are foucues on battle results not the conversations.

Answer 1

Before I answer I should note a few things. Firstly, this is a pretty terrible way to do things, hopefully there is some javascript you can hook into, or maybe it even displays things to the DOM. I don't know the game, so I can't tell you, but you should definately look for alternate solutions than just scraping the visuals. Secondly: without knowing what libraries or languages or code that you're using, it's a bit tricky to provide example code, but...

The best way to do this would probably be first looking for easily identifiable corners of the screen, and using those to get the area that you're looking for. Something like that top left corner where it meets the top bar. Maybe get the bottom right from the speech bubble or just use the literal bottom right hand corner.

Alternatively you could try to figure out how it scales and edit your code appropriately, but that isn't a fun approach...