Java WebSockets和awt.robot鼠标移动

Question

The main goal of my application is to be able to control my computer from my local network for Netflix viewing and such.

I created a websocket that works really well. Currently I'm deferring the actual mouse movements to an application called nircmd.

I sometimes get small hickups using nircmd. So I would like to use an awt.robot instead, but when I try using awt.robot to move my mouse I get the following error: java.awt.AWTException: headless environment

I went into the Administrator console in glass fish and found a property: -Djava.awt.headless=true Setting this to false produces the same error. Which boggles me.

Is it possible to give glass fish access to at least my mouse cursor and keyboard?

My server code is:

@OnMessage
public void onMessage(String message, Session sesh){
    Robot rob = null;
    try {
        rob = new Robot();
        rob.mouseMove(100, 100);
    } catch (AWTException ex) {
        Logger.getLogger(WebSocketServer.class.getName()).log(Level.SEVERE, null, ex);
    }

The client is written in javascript.

Answer 1

Please try to disable headless environment by code:

static {
        System.setProperty("java.awt.headless", "false");
}

---

Later edit:

I found and old project of mine in which i did this exact thing from an android device using the accelerometer, change it around a bit and it will fit you just fine:

Server class:

public class Server {

    static ServerSocket server;
    static Socket s;
    private static InputStreamReader inputStreamReader;
    private static BufferedReader bufferedReader;

    public static void main(String[] args) throws IOException, SQLException,
            ClassNotFoundException, AWTException {

        server = new ServerSocket(4444);
        System.out.println("Waiting for clients to connect...");
        MouseRobot mr = new MouseRobot();

        while (true) {
            s = server.accept();
            InetAddress clientAddress = s.getInetAddress();
            System.out.println("Incoming connection from " + clientAddress);

            inputStreamReader = new InputStreamReader(s.getInputStream());
            bufferedReader = new BufferedReader(inputStreamReader); //get client msg       
            String line;
            while ((line = bufferedReader.readLine()) != null) {
                System.out.println(line); 
                if (line.charAt(0) == 'X') {
                    mr.moveMouseX(Double.parseDouble(line.substring(1, line.length())));
                }
                else if (line.charAt(0) == 'Y') {
                    mr.moveMouseY(Double.parseDouble(line.substring(1, line.length())));
                }
            }
        }
    }   
}

Mouse robot class:

public class MouseRobot {

    Robot robot;
    double width;
    double height;
    float gravity = 9.8f;
    float maxAcc = 19.6f;
    int lastX = 0;
    int lastY = 0;

    public MouseRobot() throws AWTException {
        robot = new Robot();
        robot.setAutoDelay(0);
        robot.setAutoWaitForIdle(false);
        Dimension screenSize = Toolkit.getDefaultToolkit().getScreenSize();
        width = screenSize.getWidth();
        height = screenSize.getHeight();
    }

    public void moveMouseX(double x) {
        x += gravity;
        robot.mouseMove((int) (width - x * width/maxAcc), lastY);
        lastX = (int) (width - x * width/maxAcc);
    }

    public void moveMouseY(double y) {
        y += gravity;
        robot.mouseMove(lastX, (int)(height - y * height/maxAcc));
        lastY = (int)(height - y * height/maxAcc);
    }

}

The Server class first wait for a connection, after the connection is successful it starts listening for messages. Each message that begins with X moves the mouse on the X axis, the same goes for Y.

In the mouse class you could just remove all the unnecessary variables (gravity, maxAcc ...) and simply move the mouse to X / Y

Sorry for the late reply: here you go:

MainActivity:

public class MainActivity extends Activity implements SensorEventListener{

    private SensorManager senSensorManager;
    private Sensor senAccelerometer;

    static EditText ipEditText;
    static TextView statusTextView;
    static Button connectBtn;
    static EditText messagesEditText;
    static TextView xTextView;
    static TextView yTextView;

    static int status = -1;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        //UI
        ipEditText = (EditText) findViewById(R.id.ipEditText);
        statusTextView = (TextView) findViewById(R.id.statusTextView);
        connectBtn = (Button) findViewById(R.id.connectBtn);
        messagesEditText = (EditText) findViewById(R.id.messagesEditText);
        xTextView = (TextView) findViewById(R.id.xTextView);
        yTextView = (TextView) findViewById(R.id.yTextView);

        //Sensors
        senSensorManager = (SensorManager) getSystemService(Context.SENSOR_SERVICE);
        senAccelerometer = senSensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER);
        senSensorManager.registerListener(this, senAccelerometer, SensorManager.SENSOR_DELAY_FASTEST);

        connectBtn.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                setStatus(0);
                status = ServerClient.connectToServer(ipEditText.getText().toString());
                setStatus(status);
                if (status == 1) {
                    messagesEditText.setText("Successfully connected to: " + ipEditText.getText()
                        + "\n-----------");
                }
            }
        });
    }

    @Override
    public void onSensorChanged(SensorEvent sensorEvent) {
        if (status != 1)
            return;
        Sensor mySensor = sensorEvent.sensor;
        if (mySensor.getType() == Sensor.TYPE_ACCELEROMETER) {
            ServerClient.sendMessage("x" + sensorEvent.values[0]);
            ServerClient.sendMessage("y" + sensorEvent.values[1]);
            messagesEditText.setText(messagesEditText.getText().toString());
            xTextView.setText("x: " + sensorEvent.values[0]);
            yTextView.setText("y: " + sensorEvent.values[1]);
            //ServerClient.sendMessage("z = " + sensorEvent.values[2]);
        }
    }

    @Override
    public void onAccuracyChanged(Sensor sensor, int accuracy) {

    }

    public static void setStatus(int status) {
        switch (status) {
            case -1:
                statusTextView.setText("Disconnected");
                statusTextView.setTextColor(Color.RED);
            case 0:
                statusTextView.setText("Connecting");
                statusTextView.setTextColor(Color.YELLOW);
            case 1:
                statusTextView.setText("Connected");
                statusTextView.setTextColor(Color.GREEN);
        }
    }
}

ServerClient class:

public class ServerClient {

    static Socket socket = null;
    static PrintWriter printWriter = null;

    public static int connectToServer(final String serverIP) {
        Thread connectionThread = new Thread(new Runnable() {
            @Override
            public void run() {
                try {
                    try {
                        Log.d("srv", "trying to build socket");
                        socket = new Socket(serverIP, 4444); //connect to server
                        Log.d("srv", "socket made");

                    } catch (UnknownHostException e) {
                        e.printStackTrace();
                        Log.d("srv", "UnknownHostException");
                    } catch (IOException e) {
                        e.printStackTrace();
                        Log.d("srv", "IOException");
                    }
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        });
        connectionThread.start();
        try {
            connectionThread.join();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

        if(socket != null && socket.isConnected() && !socket.isClosed())
            return 1;
        else return -1;
    }

    public static void sendMessage(String messsage) {
        try {
            //dataOutputStream.writeUTF(messsage);
            printWriter = new PrintWriter(socket.getOutputStream(), true);
            printWriter.write(messsage + "\r\n");  //write the message to output stream
            printWriter.flush();
            Log.d("srv", "Sent: " + messsage);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}