Selenium Webdriver 将鼠标移动到 Point

Question

I am currently trying to move the cursor to a point ( org.openqa.selenium.Point ) that has been set by checking for an occurrence of a marker on a live chart from which I can get no details but can find the X and Y coordinates of.

How can I move to the mouse to hover over said point to open the underlying JavaScript menu?

Current code

//finds marker on the current web page

Point image = page.findImage("C:\\Pictures\\marker.png") ;

//move mouse to this x,y location 

driver.getMouse().mouseMove((Coordinates) image);

This doesnt work as Point cannot be cast to org.openqa.selenium.interactions.internal.Coordinates .

Answer 1

IMHO you should pay your attention to Robot.class

Still if you want to move the mouse pointer physically, you need to take different approach using Robot class

  Point coordinates = driver.findElement(By.id("ctl00_portalmaster_txtUserName")).getLocation();
  Robot robot = new Robot();
  robot.mouseMove(coordinates.getX(),coordinates.getY()+120);

Webdriver provide document coordinates, where as Robot class is based on Screen coordinates, so I have added +120 to compensate the browser header.
Screen Coordinates : These are coordinates measured from the top left corner of the user's computer screen. You'd rarely get coordinates (0,0) because that is usually outside the browser window. About the only time you'd want these coordinates is if you want to position a newly created browser window at the point where the user clicked. In all browsers these are in event.screenX and event.screenY .
Window Coordinates : These are coordinates measured from the top left corner of the browser's content area. If the window is scrolled, vertically or horizontally, this will be different from the top left corner of the document. This is rarely what you want. In all browsers these are in event.clientX and event.clientY.
Document Coordinates : These are coordinates measured from the top left corner of the HTML Document. These are the coordinates that you most frequently want, since that is the coordinate system in which the document is defined.

More details you can get here

Hope this be helpful to you.

Answer 2

Why use java.awt.Robot when org.openqa.selenium.interactions.Actions.class would probably work fine? Just sayin.

Actions builder = new Actions(driver);

builder.keyDown(Keys.CONTROL)
   .click(someElement)
   .moveByOffset( 10, 25 );
   .click(someOtherElement)
   .keyUp(Keys.CONTROL).build().perform();

Answer 3

I am using JavaScript but some of the principles are common I am sure.

The code I am using is as follows:

    var s = new webdriver.ActionSequence(d);
    d.findElement(By.className('fc-time')).then(function(result){
        s.mouseMove(result,l).click().perform();
    });

the driver = d . The location = l is simply {x:300,y:500) - it is just an offset.

What I found during my testing was that I could not make it work without using the method to find an existing element first, using that at a basis from where to locate my click.

I suspect the figures in the locate are a bit more difficult to predict than I thought.

It is an old post but this response may help other newcomers like me.

Answer 4

the solution is implementing anonymous class in this manner:

        import org.openqa.selenium.Point;
        import org.openqa.selenium.interactions.HasInputDevices;
        import org.openqa.selenium.interactions.Mouse;
        import org.openqa.selenium.interactions.internal.Coordinates;

        .....

        final Point image = page.findImage("C:\\Pictures\\marker.png") ;

        Mouse mouse = ((HasInputDevices) driver).getMouse();

        Coordinates imageCoordinates =  new Coordinates() {

              public Point onScreen() {
                throw new UnsupportedOperationException("Not supported yet.");
              }

              public Point inViewPort() {
                Response response = execute(DriverCommand.GET_ELEMENT_LOCATION_ONCE_SCROLLED_INTO_VIEW,
        ImmutableMap.of("id", getId()));

    @SuppressWarnings("unchecked")
    Map<String, Number> mapped = (Map<String, Number>) response.getValue();

    return new Point(mapped.get("x").intValue(), mapped.get("y").intValue());
              }

              public Point onPage() {
                return image;
              }

              public Object getAuxiliary() {
                // extract the selenium imageElement id (imageElement.toString() and parse out the "{sdafbsdkjfh}" format id) and return it
              }
            };

        mouse.mouseMove(imageCoordinates);

Answer 5

If you are using a RemoteWebDriver, you can cast WebElement into RemoteWebElement. You can then call getCoordinates() on that object to get the coordinates.

        WebElement el = driver.findElementById("elementId");
        Coordinates c = ((RemoteWebElement)el).getCoordinates();
        driver.getMouse().mouseMove(c);

Answer 6

Got it working with

Actions builder = new Actions(driver);
WebElement el = some element;
builder.keyDown(Keys.CONTROL)
.moveByOffset( 10, 25 )
.clickAndHold(el)
.build().perform();

Answer 7

Using MoveToElement you will be able to find or click in whatever point you want, you have just to define the first parameter, it can be the session(winappdriver) or driver(in other ways) which is created when you instance WindowsDriver. Otherwise you can set as first parameter a grid (my case), a list, a panel or whatever you want.

Note: The top-left of your first parameter element will be the position X = 0 and Y = 0

   Actions actions = new Actions(this.session);
    int xPosition = this.session.FindElementsByAccessibilityId("GraphicView")[0].Size.Width - 530;
    int yPosition = this.session.FindElementsByAccessibilityId("GraphicView")[0].Size.Height- 150;
    actions.MoveToElement(this.xecuteClientSession.FindElementsByAccessibilityId("GraphicView")[0], xPosition, yPosition).ContextClick().Build().Perform();

Answer 8

You can do:

JavascriptExecutor js = (JavascriptExecutor) driver;  
js.executeScript("document.elementFromPoint(25,25)");

And then you will get the element. you can add .click() to click on the element.

The executeScript allow you to get the document element and use elementFromPoint built-in javascript function to click by X,Y Cord

Answer 9

Robot robot = new Robot();
robot.mouseMove(coordinates.x,coordinates.y+80);

Rotbot is good solution. It works for me.