What is the fastest way to detect a certain color on a specific pixel and send a click once detected?

Question

Not sure if this code has already been uploaded since I haven't able to find one. I have more experience reading code than writing it so any help is appreciated.

Answer 1

First you have to know colour theory and how to detect whether two colours are the "same" or "similar" (different filters will have different outcomes too).. then you'll have to know what image formats you wish to parse into raw RGB pixels.

Finally, you'll compare the colours in the image to the colours you want to find..

First.. Colour similarity is measured with Euclidean distance or "Pythagorean distance" given as:

distance = squareRoot(of: abs(r1^2 - r2^2) + abs(g1^2 - g2^2) + abs(b1^2 - b2^2))

where r1g1b1 is the first colours and r2g2b2 is the second colour in RGB space.. Then you compare that distance to some tolerance.. IE some threshold that you're okay with as an angle of error..

For example the colour white.. 16777215 as a UInt is the same as 255, 255, 255 in RGB..

Then there is 16777214 as a UInt is the same as 255, 255, 254..

the two colours are white to the eye and extremely similar to the human eye but to the computer they are different! Therefore you use the formula and you compare the distance to some tolerance let's say "10".. why? because if the distance is 0, the colours are an exact match. If not, then you know they are not perfect, but they may be similar depending on how much of a threshold you're willing to accept.. That is what the distance represents. Otherwise compare for exact matches by comparing the bytes.

https://en.wikipedia.org/wiki/Color_difference

Now, you don't have to do it in RGB space.. you can use HSL and XYZ and CIELAB, etc.. at the end of the day, it will be fairly similar results.

I have written code that does exactly that:https://github.com/Brandon-T/CMML/blob/master/src/finder.c#L252

and https://github.com/Brandon-T/CMML/blob/master/src/finder.c#L30

#include <stdio.h>
#include <stdlib.h>
#include "bitmap.h"
#include "finder.h"

void TestOne(CTSInfo* info)
{
    rgb32 colour = {144, 240, 255, 0};
    PointArray pts;
    initPointArray(&pts);

    if (findColours(info, &pts, &colour, 0, 0, info->targetImage->width, info->targetImage->height))
    {
        Point *p = pts.p;
        int I = 0;
        for (; I < pts.size; ++I, ++p)
            printf("Colour found at: (%d, %d)\n", p->x, p->y);
    }

    freePointArray(&pts);
}

void TestTwo(CTSInfo* info, bitmap* bmp_to_find)
{
    int x, y;

    if (findImageToleranceIn(info, bmp_to_find, &x, &y, 0, 0, info->targetImage->width, info->targetImage->height, 0))
    {
        printf("Image found at: (%d, %d)\n", x, y);
    }
}

int main()
{
    CTSInfo info = {0};
    bitmap bmp_to_find = {0};
    bitmap bmp_target = {0};

    defaultCTS(&info);
    bitmap_from_file(&bmp_to_find, "C:/Users/Brandon/Desktop/find.bmp");
    bitmap_from_file(&bmp_target, "C:/Users/Brandon/Desktop/target.bmp");
    info.targetImage = &bmp_target;



    TestOne(&info);
    printf("\n");

    TestTwo(&info, &bmp_to_find);
    printf("\n");


    freebmp(&bmp_to_find);
    freebmp(&bmp_target);
    return 0;
}

If you are planning on doing this in pure C++ you might have a harder time as you'll either have to learn X11 or WinAPI to get screenshots from windows/games to detect colours within.. The best program I know that does this is located at: https://villavu.com/forum and it's called "Simba". It's written in Pascal and is used to bot Runescape with colours. It's quite successful and I've used it myself.

Someone there wrote a tutorial on CTS (Colour Tolerance Space): https://villavu.com/forum/showthread.php?t=74908