<a
                  href=http://www.t-plan.com>T-Plan Home</a>
T-Plan Robot Enterprise Doc Home
1 August 2016

Continuous Integration of T-Plan Robot Enterprise


Contents

1. Introduction
2. CI Installation
3. Create Project
4. Integrate Project
5. Design Automation
6. Integrate Automation
7. Advanced Tips
8. Conclusion


1. Introduction

This document describes how to set up a Continuous Integration job that builds an application and executes T-Plan Robot Enterprise automated testing. The guide uses Jenkins as the tool of choice. Integration with Hudson is not covered but the steps are very similar.



2. CI Installation

Download Jenkins from:
http://jenkins-ci.org
The native release is the easiest way to go. In this guide we will install native Windows release of Jenkins 1.563 on a Windows 8.1 system.

The native installer sets up Jenkins to start as a Windows service. This is not suitable for integration with Robot relying on the Local Desktop or iOS Mirror connections because the service won't have sufficient privileges to start local UI applications. For the needs of our example integration we will have to disable the Jenkins service:
  1. Open Services:
  2. Double click the Jenkins service, stop it and set the Start Type to "Disabled"
TIP: For details on how to integrate Jenkins running as a service with T-Plan Robot see the Advanced Tips chapter.

To start Jenkins manually open a command prompt, change to the Jenkins install directory (typically C:\Program Files\Jenkins or C:\Program Files (x86)\Jenkins) and execute:

jre\bin\java -jar jenkins.war
To open the Jenkins web interface open http://localhost:8080 in a web browser.
Jenkins home


3. Create Project

For the purpose of this guide we will use a simple Java application developed with Netbeans. It displays a simple message window with a Close button. As the application is a Java one it will work fine on any Java supported platform such as MS Windows, Mac OS X, Linux, Unix or Solaris OS. You can download the application source code and its project from the following link:
Demo Java application
Download the demo Java application (~20 kB)
You don't need to have NetBeans installed for this example. You must only install Ant and Java Development Kit (JDK) if you don't have it. These are needed for compilation and building of the sample Java project.

Install alternative #1: Use the Jenkins tools
  1. Click "Manage Jenkins" in the Jenkins home
  2. Click Configure
  3. Use the "Add JDK" and "Add Ant" buttons to install/configure JDK and Ant
Install alternative #2: Manual installation and configuration
  1. Download JDK and install it: http://www.oracle.com/technetwork/java/javase/downloads/index.html
  2. Download the Ant binary distribution (ZIP) from http://ant.apache.org and unzip it to the local hard drive, for example to "C:\Program Files\Ant"
  3. Put the "C:\Program Files\Ant\bin" folder to the system path and configure the JAVA_HOME variable:
  4. <previous_path>;C:\Program Files\Ant\apache-ant-1.9.4\bin
Windows system variable configuration
TIP: If you have a NetBeans installation it contains an Ant instance you may reuse. Some NetBeans releases also contain a JDK. You must only configure the variables or add the locations through the Jenkins' Configure page.


4. Integrate Project

The next step is to integrate the project of your application with CI (Jenkins). For simplicity we presume that the application is not under a version control (CVS, SVN etc).

  1. Click "New Item" in the Jenkins home
  2. Give the item a name, such as "DemoApp", choose "Build a free-style software project" and click OK
New Jenkins item
  1. Leave the Source Code Management option on "None". You may change it to CVS, SVN or any other supported system as needed.
  2. Click "Add build step". As Netbeans use Ant select the "Invoke Ant" option. Add two targets separated by a space:
clean jar
  1. Click Save
Build step configuration

The project is now set. To initialize the local project workspace we have to build it:

  1. Go to Jenkins home, click DemoApp and select "Build Now" from the left menu.
  2. In the "Build History" select the link to the build log.
  3. Select "Console Output" from the left menu.
Congratulations, your demo Java project is integrated with Jenkins now! The further configuration is up to your needs now. You may set up Jenkins to build the project on a regular schedule. If the project is under version control you may also trigger builds after every source code change.

The next step is to design a simple T-Plan Robot automated test and integrate it into the build process.



5. Design Automation

Before we design the automated testing we have to choose the test environment configuration. T-Plan Robot supports a number of architectures that involve one or more machines and/or mobile devices. For an overview see the Supported Configurations tutorial topic.

For the purpose of this guide we will choose the simplest single-machine scenario. We will test the application on the Jenkins machine using the Local Desktop connection. The T-Plan Robot project with the test code can be downloaded here:

Test project
Download the T-Plan Robot test project (4 kb)

The test project is cross platform and it will work on any Java supported platform (MS Windows, Mac OS X, Unix, Linux, Solaris OS). Unzip the project archive and open the folder it contains using File->Open Project under the T-Plan Robot GUI. The project contains one test script (tpr/Jenkins.tpr) and one template image of the icon used by the Close button of the test application.

Jenkins.tpr
1.    // Set the variable below to the location where you unzipped the downloaded demo
2.    // Java application. This will allow you to execute this test script outside of Jenkins.
3.    Var APP_HOME= "C:\demoapp"
4.   
5.    // This will ensure that the application path will be set to the Jenkins one if
6.    // the script gets called from there. The WORKSPACE system variable is set by Jenkins.
7.    if (exists _ENV_WORKSPACE) {
8.        // Use the Jenkins workspace instead
9.        Var APP_HOME= "{_ENV_WORKSPACE}"
10.    }
11.   
12.    // Set the report and template directories to the project ones
13.    Var _REPORT_DIR= "{_PROJECT_REPORT_UNIQUE_DIR}"
14.    Var _TEMPLATE_DIR= "{_PROJECT_TEMPLATE_DIR}"
15.   
16.    // Create an XML report
17.    Report "results.xml" desc="Testing of the Jenkins demo application, build {_ENV_BUILD_TAG} ({_ENV_BUILD_ID}) located at {_ENV_WORKSPACE}"
18.   
19.    // Connect to the local desktop
20.    Connect java://localhost
21.   
22.    // Declare the script number and name
23.    Script "1" name="Continuous Integration demo"
24.   
25.    // The application start command
26.    Var CMD= "java -jar {APP_HOME}{_FS}dist{_FS}demoapp.jar"
27.   
28.    // Start the application. We will open the Windows Run box and type in the command
29.    Exec "{CMD}" nowait = true
30.   
31.    // Wait no longer than 20 seconds for the application
32.    // to start. We verify it by looking for the "Close" icon.
33.    Waitfor match template="close.png" method="search2" passrate=100% timeout=20s
34.    if ({_EXIT_CODE} > 0) {
35.        Exec kill =1
36.        Exit "1" desc="Failed to start the demo application (command={CMD})!"
37.    }
38.   
39.    // Record successful app start and take a screen shot
40.    Step "Started the demo application" actual="OK"
41.    Screenshot "demoapp.png"
42.   
43.    // Click the button
44.    Mouse "click" to="x:{_COMPARETO_CLICK_X},y:{_COMPARETO_CLICK_Y}"
45.   
46.    // Wait no longer than 5 seconds for the "Close" icon to disappear.
47.    // This will verify that the button closes the window correctly.
48.    Waitfor mismatch template="close.png" method="search2" passrate=100% timeout=5s
49.    if ({_EXIT_CODE} > 0) {
50.        // Kill the application manually and terminate the script
51.        Exec kill =1
52.        Exit "2" desc="Failed to close the demo app through the Close button!"
53.    }
54.    Step "Closed the demo application through the Close button." actual="OK"

The script tests a very simple scenario:

  1. Start the demo application.
  2. Verify that the application started properly. This is achieved though searching the screen for the Close button icon.
  3. Click the Close button.
  4. Verify that the application shut down properly. The script ensures that the Close button is no longer visible.

If you set the variable APP_HOME variable (line #3) to the location where you unzipped the demo application you can execute the script right away. Requirements:

In order to produce a reasonable test report the script creates a few result objects. It creates an XML report at line 17. It gives the script a name and a number (line 23) and it creates two step results (lines 40 and 54) and a screen shot (line 41). To view the resulting report see the Test Results tree node of the Robot project or open the online copy here.

In the next step we will integrate the test script into Continuous Integration.



6. Integrate Automation

Integration with CI is enabled thanks to the T-Plan Robot command line interface (CLI). The most important options:

CLI Option
Description
-r/--run <path>
Execute the test script or schedule specified by <path>.
-c/--connect
-p/--password
Connect to a test environment. The password is optional and should be used only where required, such as for example for VNC Server connections. Connection can be also established directly from the script through the Connect command (TPR scripts) or connect() method call (Java scripts).
-n/--nodisplay
Execute in the CLI mode without displaying the GUI. It is recommended for production environments because this mode consumes less system resources (CPU, RAM). This option is required for remotely started executions (telnet, rsh/ssh).
-v/--variable <name=value>
Override a variable value. This option may be specified multiple times to override multiple variables. It is typically used to parametrize the script execution or to change the behavior through setting of the standard (predefined) variables. For example, to make the script save the report to C:\reports specify:
-v "_REPORT_DIR=C:\reports"
To create the start command suitable for CI integration use the CLI Wizard tool. To start it select Tools->CLI Wizard in Robot:
%WORKSPACE%\reports\%BUILD_NUMBER%
The WORKSPACE system variable will be set by Jenkins to the application workspace path. The BUILD_NUMBER one will contain the unique build number. For example, if the workspace is C:\Users\JohnDoe\.jenkins\jobs\DemoApp\workspace and the build number is 3 the report will be stored to
C:\Users\JohnDoe\.jenkins\jobs\DemoApp\workspace\reports\3
This will make the report available at the following URL. You may also navigate to the report from Jenkins through the Workspace link in the project's (job's) menu.
http://localhost:8080/job/DemoApp/ws/reports/3/results.xml
This allows you to send the test result URL by email after every build. This can be achieved through the Email-ext (Email Extension) Jenkins plugin. To enable valid URLs you will also have to go to Manage Jenkins -> Configure and set the Jenkins Location to a valid host name. For other report options see the Advanced Tips chapter.
CLI Wizard

To integrate the automation to the build process simply add a new build step:

  1. Go to Jenkins home, click DemoApp and select "Configure" from the left menu.
  2. Click "Add Build Step" in the Build section and choose "Execute Windows batch command"
  3. Paste the CLI command into the Command field.
The automation build step
  1. Save the changes and select Build Now. Jenkins will now build the application AND run the automation.
  2. The Jenkins build will succeed only if the build and automation finish successfully. It will fail if the build or automation fails. The test results will be stored to a unique build-specific folder in the Jenkins workspace and can be viewed in the web browser.
TIP: To debug the test script under Jenkins remove the -r and -n options from the command temporarily. This will make Jenkins start the Robot GUI without starting the automation. You may then execute the script manually in Robot and debug it. Jenkins will not complete the build until you close Robot.


7. Advanced Tips

Automation under Jenkins running as a service

Jenkins running as a service is typically restricted in two areas:

  1. The process doesn't have permissions to start GUI applications. This prevents it from integration with Robot automation based on the Local Desktop and iOS Mirror connections. Other "remote" connections such as VNC Server (except for Local Desktop over VNC) and Android Over ADB can be integrated with Jenkins in the service mode provided that you start Robot with the -n/--nodisplay CLI option.
  2. The process usually runs under a user account with limited file system permissions. Impacts on Robot:
Automated processes wrapped into a custom Java code (example) will require an extra step. Due to a limitation in Robot 3.5.2 and earlier it is necessary to set the cliMode field of the ApplicationSupport class as follows before the class is instantiated for the first time. When it's omitted the ApplicationSupport constructor will throw a java.lang.ExceptionInInitializerError. This will be fixed in release 4.0.
import com.tplan.robot.*;
import java.lang.reflect.Field;

...

try {
    Field field = ApplicationSupport.class.getDeclaredField("cliMode");
    try {
        field.setAccessible(true);
        field.set(null, true);
    } finally {
        field.setAccessible(false);
    }
} catch (Exception e) {
    e.printStackTrace();
}
ApplicationSupport a = new ApplicationSupport();
...

                

Linking reports to the Jenkins UI

The latest Robot report can be optionally linked to the project's menu and header page:
Custom link to the latest Robot report
The steps:
  1. Install the HTML Publisher Plugin
  2. The plugin unfortunately can't handle report paths containing Jenkins build variables such as %BUILD_NUMBER%. It can only link to static locations in the project workspace. To work around this we have to make the Robot script create an HTML copy of the report in a stable directory. To achieve it add the following command to the Jenkins.tpr script:
Report "..\latest\results.html" desc="Testing of the Jenkins demo application, build {_ENV_BUILD_TAG} ({_ENV_BUILD_ID}) located at {_ENV_WORKSPACE}"
  1. Select Configure in the DemoApp menu. Select "Add post-build action" and choose "Publish HTML reports". Configure the parameters as follows:
HTML directory to archive: reports\latest
Index page[s]: results.html
Report title: Latest T-Plan Robot Report
  1. Save the changes and execute the build to make the link appear in the menu.


8. Conclusion

With the steps above allow you will be able to integrate T-Plan Robot automated testing with the Continuous Integration tool of your choice. The principles of the CLI integration can be applied to integration with virtually any other framework.

The document does not cover other test environment configurations involving multiple test machines. These will be discussed later.

Should you have any question please contact the T-Plan Support team.