使用带有设置延迟的 mouse_event 平滑鼠标移动 C++

Question

I'm coding a mouse macro. It needs to meet certain points on the screen in a set delay between each point. For exammple, it must move (x 14, y 30) in 132ms. The issue I'm having is mouse_event jumps to that exact position so I need to include some sort of smoothing method so that it moves smoothly to each point. (the smoother the movement is the better the macro). Currently I am using this method of smoothing each movement.

This works well but it has its limitations for example if it needs to move 10 pixels left and the smoothing is set to 20 it will continue to jump.

Does anyone know of a more accurate method of smoothing mouse movements? (requirements accurate, smooth)

void Smoothing(int smoothing, int delay, int x, int y) {
    for (int i = 0; i < smoothing; i++) {
        mouse_event(1, x / smoothing, y / smoothing, 0, 0);
        AccurateSleep(delay / smoothing);
    }
    mouse_event(1, x % smoothing, y % smoothing, 0, 0);
    Sleep(delay % smoothing);
}

Answer 1

Linear Interpolation was my first thought when I read the question (as well as mentioned in the other answer ).

A general formular for interpolation is:

x = (1 - t) · x ₀ + t · x ₁

x ... interpolated value
x ₀ ... start value
x ₁ ... destination value
t ... interpolation parameter in range [0, 1]

I even intended to write this as answer when I realized some facts that might form possible constraints (which the OP unfortunately didn't mention explicitly).

1. All operations are about integral values. So, doing integer arithmetic may be preferred.
2. The mouse_event() as well as the AccurateSleep() is called with delta values. This might be dictated by the API used by OP.

So, I thought twice and made the following MCVE to resemble OPs problem:

#include <iostream>

static int xMouse = 0, yMouse = 0, t = 0;

void mouse_event(int _1, int dx, int dy, int _4, int _5)
{
  xMouse += dx; yMouse += dy;
  std::cout << "mouse_event(" << _1 << ", " << dx << ", " << dy << ", " << _4 << ", " << _5 << "): "
    << xMouse << ", " << yMouse << '\n';
}

void AccurateSleep(int delay)
{
  t += delay;
  std::cout << "AccurateSleep(" << delay << "): " << t << '\n';

}

void Sleep(int delay)
{
  t += delay;
  std::cout << "Sleep(" << delay << "): " << t << '\n';
}

void Smoothing(int smoothing, int delay, int x, int y)
{
    for (int i = 0; i < smoothing; i++) {
        mouse_event(1, x / smoothing, y / smoothing, 0, 0);
        AccurateSleep(delay / smoothing);
    }
    mouse_event(1, x % smoothing, y % smoothing, 0, 0);
    Sleep(delay % smoothing);
}

#define PRINT_AND_DO(...) std::cout << #__VA_ARGS__ << ";\n"; __VA_ARGS__ 

int main()
{
  PRINT_AND_DO(xMouse = 0; yMouse = 0; t = 0);
  PRINT_AND_DO(Smoothing(10, 132, 14, 30));
  PRINT_AND_DO(xMouse = 0; yMouse = 0; t = 0);
  PRINT_AND_DO(Smoothing(20, 15, 10, 0));
}

Output:

xMouse = 0; yMouse = 0; t = 0;
Smoothing(10, 132, 14, 30);
mouse_event(1, 1, 3, 0, 0): 1, 3
AccurateSleep(13): 13
mouse_event(1, 1, 3, 0, 0): 2, 6
AccurateSleep(13): 26
mouse_event(1, 1, 3, 0, 0): 3, 9
AccurateSleep(13): 39
mouse_event(1, 1, 3, 0, 0): 4, 12
AccurateSleep(13): 52
mouse_event(1, 1, 3, 0, 0): 5, 15
AccurateSleep(13): 65
mouse_event(1, 1, 3, 0, 0): 6, 18
AccurateSleep(13): 78
mouse_event(1, 1, 3, 0, 0): 7, 21
AccurateSleep(13): 91
mouse_event(1, 1, 3, 0, 0): 8, 24
AccurateSleep(13): 104
mouse_event(1, 1, 3, 0, 0): 9, 27
AccurateSleep(13): 117
mouse_event(1, 1, 3, 0, 0): 10, 30
AccurateSleep(13): 130
mouse_event(1, 4, 0, 0, 0): 14, 30
Sleep(2): 132

xMouse = 0; yMouse = 0; t = 0;
Smoothing(20, 15, 10, 0);
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 10, 0, 0, 0): 10, 0
Sleep(15): 15

Then I modified Smoothing() implementing the above mentioned interpolation formula with some adjustments to the specific situation:

1. For t , i / smoothing (with i in range [1, smoothing]) is used.
2. While the loop does the interpolation for each i , the values of previous iteration are kept and used to compute delta values for the function calls of mouse_event() and AccurateSleep() .
3. Of course, the order of operations is important as this is integer arithmetic. Hence, xI = i * x / smoothing is not equivalent to xI = i / smoothing * x . (Ie commutativity is not provided by these integral operations.)

The modified Smoothing() :

void Smoothing(int smoothing, int delay, int x, int y)
{
  int x_ = 0, y_ = 0, t_ = 0;
  for (int i = 1; i <= smoothing; ++i) {
    // i / smoothing provides the interpolation paramter in [0, 1]
    int xI = i * x / smoothing;
    int yI = i * y / smoothing;
    int tI = i * delay / smoothing;
    mouse_event(1, xI - x_, yI - y_, 0, 0);
    AccurateSleep(tI - t_);
    x_ = xI; y_ = yI; t_ = tI;
  }
}

Output:

xMouse = 0; yMouse = 0; t = 0;
Smoothing(10, 132, 14, 30);
mouse_event(1, 1, 3, 0, 0): 1, 3
AccurateSleep(13): 13
mouse_event(1, 1, 3, 0, 0): 2, 6
AccurateSleep(13): 26
mouse_event(1, 2, 3, 0, 0): 4, 9
AccurateSleep(13): 39
mouse_event(1, 1, 3, 0, 0): 5, 12
AccurateSleep(13): 52
mouse_event(1, 2, 3, 0, 0): 7, 15
AccurateSleep(14): 66
mouse_event(1, 1, 3, 0, 0): 8, 18
AccurateSleep(13): 79
mouse_event(1, 1, 3, 0, 0): 9, 21
AccurateSleep(13): 92
mouse_event(1, 2, 3, 0, 0): 11, 24
AccurateSleep(13): 105
mouse_event(1, 1, 3, 0, 0): 12, 27
AccurateSleep(13): 118
mouse_event(1, 2, 3, 0, 0): 14, 30
AccurateSleep(14): 132

xMouse = 0; yMouse = 0; t = 0;
Smoothing(20, 15, 10, 0);
mouse_event(1, 0, 0, 0, 0): 0, 0
AccurateSleep(0): 0
mouse_event(1, 1, 0, 0, 0): 1, 0
AccurateSleep(1): 1
mouse_event(1, 0, 0, 0, 0): 1, 0
AccurateSleep(1): 2
mouse_event(1, 1, 0, 0, 0): 2, 0
AccurateSleep(1): 3
mouse_event(1, 0, 0, 0, 0): 2, 0
AccurateSleep(0): 3
mouse_event(1, 1, 0, 0, 0): 3, 0
AccurateSleep(1): 4
mouse_event(1, 0, 0, 0, 0): 3, 0
AccurateSleep(1): 5
mouse_event(1, 1, 0, 0, 0): 4, 0
AccurateSleep(1): 6
mouse_event(1, 0, 0, 0, 0): 4, 0
AccurateSleep(0): 6
mouse_event(1, 1, 0, 0, 0): 5, 0
AccurateSleep(1): 7
mouse_event(1, 0, 0, 0, 0): 5, 0
AccurateSleep(1): 8
mouse_event(1, 1, 0, 0, 0): 6, 0
AccurateSleep(1): 9
mouse_event(1, 0, 0, 0, 0): 6, 0
AccurateSleep(0): 9
mouse_event(1, 1, 0, 0, 0): 7, 0
AccurateSleep(1): 10
mouse_event(1, 0, 0, 0, 0): 7, 0
AccurateSleep(1): 11
mouse_event(1, 1, 0, 0, 0): 8, 0
AccurateSleep(1): 12
mouse_event(1, 0, 0, 0, 0): 8, 0
AccurateSleep(0): 12
mouse_event(1, 1, 0, 0, 0): 9, 0
AccurateSleep(1): 13
mouse_event(1, 0, 0, 0, 0): 9, 0
AccurateSleep(1): 14
mouse_event(1, 1, 0, 0, 0): 10, 0
AccurateSleep(1): 15

Live Demo on coliru

Note:

The last iteration is done with i == smoothing so that i / smoothing results in 1. Hence, the last interpolation step yields the exact values – no post-correction is necessary like in OPs original approach.

Answer 2

View the points as vectors and interpolate between them. This is often referred to as "lerping" sort for linear interpolation. You can find many resources that may help if you search up linear interpolation. Here's an answer that may help understand what it is.

Since I have extra time on my hands I've typed up an example of a program that does it as well.

#include <iostream>
#include <chrono>

struct Vec2d {
    double x;
    double y;
    Vec2d(double x, double y) : x(x), y(y) {};
};

Vec2d lerp(Vec2d const& a, Vec2d const& b, double t)  {
    double x((1.0 - t) * a.x + t * b.x);
    double y((1.0 - t) * a.y + t * b.y);
    return Vec2d(x, y);
}

int main(int argc, char* argv[]) {
    Vec2d p1(10, 10);
    Vec2d p2(20, 40);

    double maxTime(100); //max time 100 milliseconds
    double elapsedTime(0);
    std::chrono::time_point<std::chrono::system_clock> start(std::chrono::system_clock::now());
    std::chrono::time_point<std::chrono::system_clock> end(start);
    while(elapsedTime < maxTime) {
        elapsedTime += std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count();
        start = end;

        //This is where the lerping happens
        double t(elapsedTime / maxTime);
        Vec2d p3(lerp(p1, p2, t));

        //Show what's happening.
        std::cout << "p3: " << p3.x << ", " << p3.y << std::endl;
        end = std::chrono::system_clock::now();
    }

    return 0;
}

Short Explaination: t isa value from 0 to 1. When t == 0.0 lerp will return a "copy" of p1 . When t == 1.0 lerp will return a "copy" of p2 . When t == 0.5 lerp will return (p1 + p2) / 2 (the mid point between them).

You'll also need to add code to continually update the position of the mouse. To do this you'll need to keep track of how much time has elapsed and calculate the value of t based on the amount of time required to travel from p1 to p2 and the actual time that has elapsed. The above code does this with the use of a while loop and std::chrono to keep track of elapsed time. This will implementation however will be dependent on how you intend to trigger these "updates".

Hope this helped.