Move mouse cursor with accelerometer and gyroscope

Question

Hope you can help me!

I want to controle my pc cursor with the values from an gyroscope and an accelerometer. So basically when I move the sensor, the cursor should be moved. I used an MPU-6050 chip and an ESP32.

How I get the position of the cursor:

public static void main (String[] args){

        for(int i = 0; i<= 1000000; i++) {

        PointerInfo info = MouseInfo.getPointerInfo();

        Point location = info.getLocation();

        System.out.println("x="+ location.x + " y=" + location.y);

}

How I will get the values from the sensor:

 public SensorData(JsonObject data) {
    //accerlation accelerometer
    ax = data.get("ax").asDouble()/ASENSETIFITY;
    ay = data.get("ay").asDouble()/ASENSETIFITY;
    az = data.get("az").asDouble()/ASENSETIFITY;

    //temperature 
    temp = data.get("t").asDouble()/340.00+36.53;

    //gyroscope
    gx = data.get("gx").asDouble()/GSENSETIFITY;
    gy = data.get("gy").asDouble()/GSENSETIFITY;
    gz = data.get("gz").asDouble()/GSENSETIFITY;

}

public String toString() {
    return "ax: " + Double.toString(ax) +", ay: " + Double.toString(ay)
        +", az: " + Double.toString(az) +", temp: " + Double.toString(temp)
        +", gx: " + Double.toString(gx) +", gy: " + Double.toString(gy)
        +", gz: " + Double.toString(gz);

}

What I got so far:

package mouse;

import java.awt.AWTException;

import java.awt.Robot;
import Connection.SensorData;

public class MouseMoving {
Robot robot;

public MouseMoving(){
    try {
        robot = new Robot();
    } catch (AWTException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
}


    public static void verarbeite(SensorData data){

    System.out.print("Deine Maus befindet sich da:");
    System.out.println(data);

    //robot.mouseMove(1,1);
}


}   

My Question:

How can I now controle the cursor? Do you have an idea?

I thought, that you need the last position of the mouse cursor, then get the current position of the sensor and change the mouse position. But how can I write this whith my values?

I found this website, where 3 objects where rotated on the pc when the sensor will be moved. But this isn't written in Java and I don't understand his solution.

Also I found this and this question. Maybe it will work like that. But I haven't been programming that long and don't understand it.

Answer 1

If I understand your question well, you are trying to move the mouse cursor with Java. To achieve that, you can use the Robot class, which hase a MouseMove(int x, int y) method.

By default, it positions the cursor on the primary screen. If you have a multiple screen setup, you will have to pass a GraphicsDevice object to indicate which screen should be used.

Example, moving your cursor 10 pixels to the right

PointerInfo pointerInfo = MouseInfo.getPointerInfo();
Point mouseLocation = pointerInfo.getLocation();
GraphicsDevice graphicsDevice = pointerInfo.getDevice();
new Robot(graphicsDevice).MouseMove(mouseLocation.getX()+10, mouseLocation.getY());