How to programmatically (C# or Java) launch an app in Windows and invoke click in it's window?

Question

There is a simple application that works in Windows. It has very simple interface: squre window with buttons in fixed coordinates.

I need to write a program that makes use of this application: to launch it and to click one of buttons (let's say invoke a click at (150,200)).

Is there any way to do it in Java or .NET?

Answer 1

The Java based solution is to launch the app. in a Process and use the Robot to interact with it.

The best solution on this thread was by @HFoE but deleted by a moderator. For reference, it basically came down to..

If you want to control another Windows application, use a tool that was built specifically for this such as AutoIt V3 .

Since "Don't do it" seems to be considered a valid answer when an alternative is supplied (by general opinion on Meta), I cannot understand why the answer was deleted.

Answer 2

As Hovercraft Full Of Eels if you can - use autoit - it's much easier. If AutoIt is not an option then you will need to use winAPI functions in order to do it.

For example to call mouseclick at coordinates:

[DllImport("user32.dll")]
static extern bool SetCursorPos(int x, int y);

[DllImport("user32.dll")]
static extern bool GetCursorPos(ref Point lpPoint);

[DllImport("user32.dll")]
public static extern void mouse_event(int dwFlags, int dx, int dy, int cButtons, int dwExtraInfo);

public void LeftMouseClick(int xpos, int ypos) //Make a click at specified coords and return mouse back
{
    Point retPoint = new Point();
    GetCursorPos(ref retPoint); // set retPoint as mouse current coords
    SetCursorPos(xpos, ypos); //set mouse cursor position
    mouse_event(MOUSEEVENTF_LEFTDOWN, xpos, ypos, 0, 0); 
    mouse_event(MOUSEEVENTF_LEFTUP, xpos, ypos, 0, 0); //click made
    SetCursorPos(retPoint.X, retPoint.Y); //return mouse position to coords
}

But be aware, that to make click inside a window it needs to be at front of you - you cannot click to a minimized app for example.

If you want to try - you can find all needed functions(how to run a programm, get needed window by hwnd and so on) at PInvoke

Answer 3

For .Net you can pretty much use AutomationElement which I prefer. There's a bit of learning time, but it shouldn't take much. You can start your app with ProcessStartInfo .

If you have VS2010 Pro or Ultimate you can use the CodedUITests to generate a couple of button pushes.

As @Hovercraft Full Of Eels suggested - Autoit, Python could do the same

Answer 4

Yes - in C#...

1. Use the Process class to start the process (there are plenty of resources on the web on how to do this.
2. Wait until the process has started (either just wait for a fixed amount of time which is probably going to be long enough, or you could try and do something fancy like IPC or monitoring for a window being created)
3. To simulate the click take a look at How to simulate Mouse Click in C#? which uses a P/Invoke call to the mouse_event function.

However note that there are several things that can go wrong with this

• Someone might move the window, or place another window on top of that window in the time it takes to launch the application
• On a slower PC it may take longer to load the application (this risk can be mitigated by doing things like monitoring open windows and waiting for the expected application window to appear)

Answer 5

在.net中，您可以从System.Diagnostics进行Process.Start来启动应用程序，甚至可以传递参数，并且可以使用P / Invoke来模拟鼠标事件，因此在SO上已经有了答案

Answer 6

Here is my working test app to play with clicking in windows. We just start some app and hope to click it in right place) It would be nice to have some solution for capturing windows this way =)

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Threading;

namespace ConsoleApplication8
{
    class Program
    {
        static void Main(string[] args)
        {
            var startInfo = new ProcessStartInfo(@"C:\Users\Bodia\Documents\visual studio 2010\Projects\ConsoleApplication8\WindowsFormsApplication1\bin\Debug\WindowsFormsApplication1.exe");
            startInfo.WindowStyle = ProcessWindowStyle.Maximized;

            Console.WriteLine(1);

            var process = Process.Start(startInfo);
            Console.WriteLine(2);

            Thread.Sleep(400);
            Console.WriteLine(3);

            LeftMouseClick(1000, 200);
            Console.WriteLine(4);
        }

        static void CursorFun()
        {
            Point cursorPos = new Point();
            GetCursorPos(ref cursorPos);

            cursorPos.X += 100;
            Thread.Sleep(1000);
            SetCursorPos(cursorPos.X, cursorPos.Y);

            cursorPos.X += 100;
            Thread.Sleep(1000);
            SetCursorPos(cursorPos.X, cursorPos.Y);

            cursorPos.X += 100;
            Thread.Sleep(1000);
            SetCursorPos(cursorPos.X, cursorPos.Y);

            cursorPos.X += 100;
            Thread.Sleep(1000);
            SetCursorPos(cursorPos.X, cursorPos.Y);

        }

        [DllImport("user32.dll")]
        static extern bool SetCursorPos(int x, int y);

        [DllImport("user32.dll")]
        static extern bool GetCursorPos(ref Point lpPoint);

        [DllImport("user32.dll")]
        public static extern void mouse_event(int dwFlags, int dx, int dy, int cButtons, int dwExtraInfo);

        public static void LeftMouseClick(int xpos, int ypos) //Make a click at specified coords and return mouse back
        {
            Point retPoint = new Point();
            GetCursorPos(ref retPoint); // set retPoint as mouse current coords
            SetCursorPos(xpos, ypos); //set mouse cursor position
            mouse_event(MOUSEEVENTF_LEFTDOWN, xpos, ypos, 0, 0);
            mouse_event(MOUSEEVENTF_LEFTUP, xpos, ypos, 0, 0); //click made
            SetCursorPos(retPoint.X, retPoint.Y); //return mouse position to coords
        }
        struct Point
        {
            public int X;
            public int Y;
        }

        private const int MOUSEEVENTF_LEFTDOWN = 0x02;
        private const int MOUSEEVENTF_LEFTUP = 0x04;
        private const int MOUSEEVENTF_RIGHTDOWN = 0x08;
        private const int MOUSEEVENTF_RIGHTUP = 0x10;
    }
}