如何使用Java在Sikuli中找到图像的精确匹配

Question

Am new to Sikuli and trying to Automate Citirx Application. Need Help

Am trying to select a user role in a screen, The screen has multiple roles and hence i need to scroll down the screen and search for a particular Role and click the Role.

I have Captured image of a Particular Role that i need to select and used below Code. In the second Image i have highlighted the Role i need to select in Red

Below is the Code an Trying:

Creating a Method:

 public static boolean clipExist(Screen screen, String clip )
 {
        Match m = screen.exists(clip);
         if(m != null)
      {
              return true;
        }
       else
       {
          return false;
       }
   }

Using the Method:

        while(! clipExist(screen, "C:\\Users\\Satish_D1\\workspace\\Sikuli Demo\\Images\\DownArrow.PNG"))       
    {           
       screen.wheel(1 , 3);     
       if(clipExist(screen, "C:\\Users\\Satish_D1\\workspace\\Sikuli Demo\\Images\\Roles\\UK\\ENTP\\GEDIS_SALES_SUPPORT_ORL_CPF2.0_UK_ENTP.PNG"))
       {
        screen.doubleClick("C:\\Users\\Satish_D1\\workspace\\Sikuli Demo\\Images\\Roles\\UK\\ENTP\\GEDIS_SALES_SUPPORT_ORL_CPF2.0_UK_ENTP.PNG",0);
        break;
       }
      }

Answer 1

The image recognision uses per default a similarity of 0.7 (see description of Patterns in SikuliX Documentation ). That means SikuliX looks for 'pretty similar' images. You can specify the similarity for the pattern recognision thanks to the method similar , or in your case use the method exact . In your method clipExist , you should replace the name of the image:

    Match m = screen.exists(clip);

by:

    Match m = screen.exists(Pattern(clip).exact())

Answer 2

It seems SikuliX 1.1 experience some problem with finding the text on a screen, but recognition works. You might want to scan the entire text screen by screen and split the lines. Next compare each line with the required role and save the degree of similarity. Select the line with the biggest similarity. In Python/Jython exists a special function for that in difflib module. similarity = difflib.SequenceMatcher(None, string_a, string_b)

Answer 3

Here are the alternatives that you can do.

First alternative: capture scrollbar

1. Capture the down arrow in the scrollbar
2. Capture the image when you reach the end of scrollbar. The image contains the scroll progress and the down arrow of the scrollbar
3. Click down arrow until you find image of (2)

This method have drawback ie when the number of items are dynamic, the visual appearance of (2) will be different especially the scroll progress. However, this can be tricked by capturing only the lower part of scroll progress and the arrow. Please note that your mouse may make difficulty in (3) because you may not find (2) when it is covered by mouse. To handle this, every time you click down arrow, you may hover your mouse a bit before checking for (2). This is the complete script:

down_arrow = "downarrow.png"
complete_scroll = "completescroll.png"

while not exists(complete_scroll):
    click(down_arrow)
    hover(Location(300, 200))

Second alternative, use keyboard (down key)

Click anywhere in the items to be scrolled and do some type(Key.DOWN) for the number of item you have. In case you have dynamic number of item, you may do type(Key.DOWN) for any number that always bigger than your number of items. Here is the script to do

inside_item = "inside.png"

for n in range(10000):
    type(Key.DOWN)

Hope it helps

Answer 4

I used 's' as a screen class reference. Hence, once we get an image then we will set the region for the same followed by the required image where you want to click

public static void main(String args[])
{
    Match m = s.find("IMAGE");
    Region r = new Region(m.x+11, m.y+22,12,12);
    r.click(); 
    s.find("ENTPIMAGE.PNG");
    r.click("ENTPIMAGE.PNG");
}