T-Plan Home
T-Plan Robot Enterprise 4.3.1 Doc Collection

Local Desktop Automation

Last update: 7 December 2015

1. Introduction
2. Setting Up
3. Connection
4. Display Handling
5. Keyboard Handling
6. Starting Local Applications

1. Introduction

T-Plan Robot Enterprise version 3.2 introduced support of automation of applications displayed on the local desktop. This means that Robot and one or more Applications Under Test (AUT) run on the same local desktop. The automation is of course not limited to the AUT and it may perform any desktop action that a regular user can do.

Local desktop


2. Setting Up

Local desktop automation is supported over two connection types:
  1. The Local Desktop connection relies on the local Java SE installation. There are no other dependencies. To connect simply select the connection in the Login Dialog and hit Connect.
  2. The local desktop can be also automated over a locally running VNC Server:
The Local Desktop connection relying on Java is the preferred one. It is in general more reliable than the VNC server and it can type non-English characters (see the keyboard support). The Local Desktop connection is also typically faster in terms of screen polling. The VNC server may provide better screen performance if it is attached directly to the video driver, such as for example UltraVNC with the Mirror Driver or TightVNC with the DFMirage driver.

Compatibility between the Local Desktop and VNC Server connections is not explicitly guaranteed. A test script designed against a local or remote VNC connection may or may not run correctly with the Local Desktop. The following factors apply:

3. Connection

To connect to the local desktop from a test script use the Connect command (TPR scripts) or the connect() method (Java test scripts). The argument URL must be in form of either "java://localhost" (Local Desktop connection type) or "rfb://localhost" (VNC Server).

For example, the following command/method call will connect you to the local desktop using the Local Desktop connection:

TPR test scripts:     Connect java://localhost
Java test scripts:    connect("java://localhost");

To connect to the local desktop using a VNC server running on the standard port of 5900:

TPR test scripts:     Connect rfb://localhost
Java test scripts:    connect("rfb://localhost");

The URL may be also used together with the -c/--connect CLI option to establish the connection on the Robot start up. On a successful connection Robot starts up with the Workflow chart displayed instead of the desktop viewer and the application title will indicate the connection (java://localhost or rfb://localhost:5900).

IMPORTANT: Do not change the OS keyboard layout while using the Local Desktop connection. It maps the layout when the connection is established and any later change may cause the test script press invalid keys. To remap the keyboard disconnect and reconnect. This may be performed even from a test script by a sequence of Disconnect and Connect commands (TPR format) or the disconnect()  and connect() methods (Java format).

4. Display Handling

On single display environments the local desktop automation process differs from the remote one. When Robot needs to work with the local screen it must hide the GUI first. In the test script development phase it also applies to situations when the user chooses to perform image comparison, to take a screen shot, to define a screen location or a rectangle, to create a new template image or to perform Component Capture. Similarly when a script starts to execute the GUI gets hidden and it is not restored until the scripts finishes. To pause the script execution and restore the GUI move the mouse. The behavior can be changed through the Local Machine panel of the Preferences dialog. The options allow to set the mouse sensitivity for the script pausing purposes and the UI hiding behavior.

Robot v4.1.1 introduced full support of multi display environments. It is possible to automate a particular screen or all of them. When there are two or more displays the Login window displays a screen selector and a drop down with the options:

Local Desktop Logon

Screens are numbered starting from 1 where the first one is typically the primary one.

The most efficient set up is to automate one display and put the Robot GUI on the other one. This eliminates the need to hide the GUI on script executions and graphical operations.

To connect to a particular screen from a test script or from the CLI add the screen number to the connection URL as follows. When the number is omitted or is 0 the connection will automate all screens. This form is also suitable for the -c/--connect CLI connection. You may copy the URL of any of your recent connections from the Tools->CLI Wizard window. Another alternative is to give your connection a name in the Desktop->Connection Manager window and then use it instead of the URL.

TPR test scripts:     Connect java://localhost?screen=1
Java test scripts:    connect("java://localhost?screen=1");

IMPORTANT: For performance reasons the screen configuration is loaded just once on the Robot startup. To ensure correct operation do not change the screen layout or add/remove displays while Robot is running.

Since Robot 4.3.1 properties of the connected displays are exposed to test scripts through variables as follows:

// Search for a component on all screens
Compareto "mount.png" method= "search2"

// Exit if not found
if ({_EXIT_CODE} > 0) {
    Exit "1"

// Test the display the component is showing on
if ({_DISPLAY_NO} == 1) {
    // Component on the first display -> do something
} else {
    // Component on another screen -> do something else

For a more generic description also see the table of implicit variables in the Var command specification.

5. Keyboard Handling

When the connection is being established it pops up a window showing the keyboard mapping progress. To ensure correct operation do not work with the desktop while it is displayed.

Version 4.1 introduced the stored keyboard maps (not applicable to the local VNC connection). Once the keyboard is mapped it gets saved to a file. The subsequent connections will reuse it as long as there is no change in the keyboard layout. This approach is faster and it avoids app focus issues on test suites executing a sequence of shorter test scripts.

The keyboard maps are by default stored to the user home folder. Should you need to manage the map(s) manually look for files with the tprlkm_ prefix and the .keymap extension. There's one file per each input language and keyboard layout. For example, the German keyboard layout will be stored as <home>/tprlkm_de_DE.keymap.
Robot will also accept any keyboard map file placed in its installation directory (the folder containing the robot.jar file). These maps must follow the above file name convention. They are preferred to the ones placed in the user home folder. This feature allows to create custom Robot based solutions.

Usage of stored maps is controlled through the localDesktop.useStoredKeymaps preference in the user configuration file. You may set it either in the Login window or in the Local Desktop panel of the Preferences window. Should you need to modify the option programmatically please read the Setting Preferences documentation.

6. Starting Local Applications

Version 3.5.2 extended the Exec command with the nowait and kill parameters to simplify automation of local applications over the Local Desktop connection. The basic pattern is that the script starts a local application, tests it and closes it at the script end. Example:
// Start myapp.exe and don't wait until it finishes

// Test the application

// Kill the application ('1' is ordinary number of the process started with "Exec nowait=true")