Automating mouse clicks on screen

Question

I am looking for a way of creating a program that will perform a mouse click where it finds a certain color on the screen.

For example if there is a red box on the screen, I would want the program to click on the red box in the center of it.

How could I accomplish this in C#?

Answer 1

As you only wanted a general way, I didn't really make it perfect, but here is the idea:

Have a method for taking a screen shot:

public Bitmap ScreenShot()
{
    var screenShot = new Bitmap(Screen.PrimaryScreen.Bounds.Width,
                                Screen.PrimaryScreen.Bounds.Height,
                                PixelFormat.Format32bppArgb);

    using (var g = Graphics.FromImage(screenShot))
    {
        g.CopyFromScreen(0, 0, 0, 0, Screen.PrimaryScreen.Bounds.Size);
    }

    return screenShot;
}

And a method to find a certain color in a bitmap: Note that this implementation can be DRASTICALLY improved using unsafe code and LockBits (read here and here ).

public Point? GetFirstPixel(Bitmap bitmap, Color color)
{
    for (var y = 0; y < bitmap.Height; y++)
    {
        for (var x = 0; x < bitmap.Width; x++)
        {
            if (bitmap.GetPixel(x, y).Equals(color))
            {
                return new Point(x, y);
            }
        }
    }

    return null;
}

Another method you'll need is one for clicking a certain point:

[DllImport("user32.dll",
           CharSet=CharSet.Auto,
           CallingConvention=CallingConvention.StdCall)]
private static extern void mouse_event(long dwFlags,
                                      long dx,
                                      long dy,
                                      long cButtons,
                                      long dwExtraInfo);

private const int MOUSEEVENTF_LEFTDOWN = 0x02;
private const int MOUSEEVENTF_LEFTUP = 0x04;

public void Click(Point pt)
{
    Cursor.Position = pt;
    mouse_event(MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP, pt.X, pt.Y, 0, 0);
}

And finally, one to wrap it all up:

public bool ClickOnFirstPixel(Color color)
{
    var pt = GetFirstPixel(ScreenShot(), color);

    if (pt.HasValue)
    {
        Click(pt.Value);
    }

    // return whether found pixel and clicked it
    return pt.HasValue;
}

Then, the usage would be:

if (ClickOnFirstPixel(Color.Red))
{
    Console.WriteLine("Found a red pixel and clicked it!");
}
else
{
    Console.WriteLine("Didn't find a red pixel, didn't click.");
}