Object Search and Static Image Testing

T-Plan Robot Enterprise v2.2 introduces two new features which target testing of imaging systems such as the GIS (Geographic Information System) and CAD (Computer Aided Engineering) ones.

The first one is Static Image Client which allows automation against images loaded from the file system. The client behaves the same way as a live desktop client such as the RFB/VNCVirtual Network Computing (VNC) is a graphical desktop sharing system that uses the RFB protocol to remotely control another computer. VNC is platform-independent. Multiple clients may connect to a VNC server at the same time. one. Image files may be loaded interactively by the Login dialog or in an automated manner through the "-c" CLICommand Line Interface option or by calling the Connect command with an argument in form of "file://<path_to_the_image>". Static image testing may be successfully applied in a number of scenarios:

• Testing of images, such as screen shots, photographs, maps, paintings and outputs of various image producing systems,
  
• Testing of systems which update an image file (for example a status screen shot). Since the client periodically checks the image file size and time stamp and reloads the image whenever a change is detected, it may be used to test systems which generate output into an image file (or image files).
  
• Debugging of failed image comparisons from live desktop testing. If you modify your script to save the VNC desktop image to a PNG file whenever image comparison fails unexpectedly, for example using the Screenshot command, you may open the file later on and reapply the image comparison to find out what was wrong. You may eventually easily recreate the image template from the image and verify the new functionality using the standard GUI tools.
  

Testing of images has a few specific aspects. Unlike live desktops, static images don't consume key events and the Press and Type/Typeline actions are disabled. If the image is truly static meaning that it is not being updated by an outside process, it also doesn't make sense to wait for an update event through the Waitfor command.

The second feature is Object Search image comparison method. It was designed to locate objects of a particular color or a range of colors. Though this approach is not suitable for classic GUI components, it performs well on low color outputs from imaging systems such as maps, drawings, schemes, etc. The Object Search is typically applied to images loaded through the Static Image Client or to live VNC desktops displaying the target image.

The method accepts an optional template which must contain exactly one object of the specified color. When specified the method filters out the list of objects located to leave just those which are similar to the template image shape up to the ratio specified by the passrate parameter. If the rotations parameter is also specified and is greater than 1, the object list will be matched against the list of template shapes rotated the specified number of times. This allows to build test tasks like "search for all green rectangles which might be in any level of rotation".

The object color is specified by the color parameter. The range of accepted object colors is specified by the toleranceparameter which is similar to the Tolerant Image Search one. It is a number between 0 and 255 which indicates how much the Red, Green and Blue components of a pixel may differ at a maximum to consider the color be equivalent to the object color specified by the color parameter. Be aware that the higher the tolerance value, the higher the probability of false shape detections is. In most scenarios the value should be in the [0, 100] range depending on how much the object is changing, for example as a result of blurring caused by rotation. If the parameter is not specified, it defaults to zero and the algorithm looks for solid color objects only. For a complete specification of algorithm parameters see the Object Search specification.

As an example let's consider a map-like image with black paths and green triangle objects:

Our goal is to find out the number and location of green triangles. This is a perfect task for the static image client combined with the object search algorithm. The script code as well as a video follow. You may optionally download the complete test suite and try it out on your own environment. Since image testing is platform independent, the example will work fine on all operating systems supported by T-Plan Robot Enterprise.

# Make the script look for template images in the same dir as the script Var _TEMPLATE_DIR="{_SCRIPT_DIR}" # Connect to the map image. Connect file://{_SCRIPT_DIR}/map.png # Locate all triangles and highlight them on the screen Compareto "triangle.png" method="object" passrate="85" rotations="40" color="01FF19" tolerance="100" draw="true" # Pause the script to allow to review the results. # Resuming will finish the script and clear up the drawings. # Should you need to work with the objects further on, their number # and coordinates are available through variables populated by the "object" method. Pause

Image testing example on Ubuntu Linux