T-Plan Robot Enterprise Doc Home

1 August 2016

Continuous Integration of T-Plan Robot Enterprise

1. Introduction

2. CI Installation

http://jenkins-ci.org

1. Open Services:
• Windows 8: Go to the Start screen and type "view local services"
• Windows 7 and older: Open the Control Panel, search for Services and open "View local services" from the "Administrative Tools" group
  
2. Double click the Jenkins service, stop it and set the Start Type to "Disabled"

To start Jenkins manually open a command prompt, change to the Jenkins install directory (typically C:\Program Files\Jenkins or C:\Program Files (x86)\Jenkins) and execute:

jre\bin\java -jar jenkins.war

3. Create Project

Download the demo Java application (~20 kB)

1. Download JDK and install it: http://www.oracle.com/technetwork/java/javase/downloads/index.html
   
2. Download the Ant binary distribution (ZIP) from http://ant.apache.org and unzip it to the local hard drive, for example to "C:\Program Files\Ant"
3. Put the "C:\Program Files\Ant\bin" folder to the system path and configure the JAVA_HOME variable:
   
• Windows 8: go to the Start screen and type "system"
• Windows 7 and older: open Control Panel and select System
  
• Select Advanced System Settings -> Environment Variables, double click the Path variable from the System variables table and append a semicolon ; and the path to the variable value:

<previous_path>;C:\Program Files\Ant\apache-ant-1.9.4\bin


• Add a new System variable, the name is JAVA_HOME and the value is the JDK install path (for example C:\Program Files\Java\jdk1.7.0_51)

• Restart Jenkins to pick up the system changes. If you started Jenkins from a command prompt close it and start a new one.
  
• To verify the result open a command prompt (Windows+R, "cmd") and execute "ant.bat". The system must recognize the command. If there's a message like "tools.jar not found" you haven't set the JAVA_HOME variable properly.
  

4. Integrate Project

The next step is to integrate the project of your application with CI (Jenkins). For simplicity we presume that the application is not under a version control (CVS, SVN etc).

1. Click "New Item" in the Jenkins home
   
2. Give the item a name, such as "DemoApp", choose "Build a free-style software project" and click OK. 

3. Leave the Source Code Management option on "None". You may change it to CVS, SVN or any other supported system as needed.
   
4. Click "Add build step". As Netbeans use Ant select the "Invoke Ant" option. Add two targets separated by a space:

clean jar

5. Click Save

The project is now set. To initialize the local project workspace we have to build it:

1. Go to Jenkins home, click DemoApp and select "Build Now" from the left menu.
2. In the "Build History" select the link to the build log.
3. Select "Console Output" from the left menu.
   
• If the project is under a version control it should show success because Jenkins pulls the project from the server.
  
• As our example does not use any version control the build will fail with a message that Ant can not find the build.xml file. Download the sample project and unzip it into the location mentioned in the log (typically <user_home>\.jenkins\jobs\DemoApp\workspace). Then go back to the project page and click "Build Now" again. This time it should succeed.

5. Design Automation

Before we design the automated testing we have to choose the test environment configuration. T-Plan Robot supports a number of architectures that involve one or more machines and/or mobile devices. For an overview see the Supported Configurations tutorial topic.

For the purpose of this guide we will choose the simplest single-machine scenario. We will test the application on the Jenkins machine using the Local Desktop connection. The T-Plan Robot project with the test code can be downloaded here:

Download the T-Plan Robot test project (4 kb)

The test project is cross platform and it will work on any Java supported platform (MS Windows, Mac OS X, Unix, Linux, Solaris OS). Unzip the project archive and open the folder it contains using File->Open Project under the T-Plan Robot GUI. The project contains one test script (tpr/Jenkins.tpr) and one template image of the icon used by the Close button of the test application.

Jenkins.tpr

1.    // Set the variable below to the location where you unzipped the downloaded demo 2.    // Java application. This will allow you to execute this test script outside of Jenkins. 3.    Var APP_HOME= "C:\demoapp" 4.    5.    // This will ensure that the application path will be set to the Jenkins one if 6.    // the script gets called from there. The WORKSPACE system variable is set by Jenkins. 7.    if (exists _ENV_WORKSPACE) { 8.        // Use the Jenkins workspace instead 9.        Var APP_HOME= "{_ENV_WORKSPACE}" 10.    } 11.    12.    // Set the report and template directories to the project ones 13.    Var _REPORT_DIR= "{_PROJECT_REPORT_UNIQUE_DIR}" 14.    Var _TEMPLATE_DIR= "{_PROJECT_TEMPLATE_DIR}" 15.    16.    // Create an XML report 17.    Report "results.xml" desc="Testing of the Jenkins demo application, build {_ENV_BUILD_TAG} ({_ENV_BUILD_ID}) located at {_ENV_WORKSPACE}" 18.    19.    // Connect to the local desktop 20.    Connect java://localhost 21.    22.    // Declare the script number and name 23.    Script "1" name="Continuous Integration demo" 24.    25.    // The application start command 26.    Var CMD= "java -jar {APP_HOME}{_FS}dist{_FS}demoapp.jar" 27.    28.    // Start the application. We will open the Windows Run box and type in the command 29.    Exec "{CMD}" nowait = true 30.    31.    // Wait no longer than 20 seconds for the application 32.    // to start. We verify it by looking for the "Close" icon. 33.    Waitfor match template="close.png" method="search2" passrate=100% timeout=20s 34.    if ({_EXIT_CODE} > 0) { 35.        Exec kill =1 36.        Exit "1" desc="Failed to start the demo application (command={CMD})!" 37.    } 38.    39.    // Record successful app start and take a screen shot 40.    Step "Started the demo application" actual="OK" 41.    Screenshot "demoapp.png" 42.    43.    // Click the button 44.    Mouse "click" to="x:{_COMPARETO_CLICK_X},y:{_COMPARETO_CLICK_Y}" 45.    46.    // Wait no longer than 5 seconds for the "Close" icon to disappear. 47.    // This will verify that the button closes the window correctly. 48.    Waitfor mismatch template="close.png" method="search2" passrate=100% timeout=5s 49.    if ({_EXIT_CODE} > 0) { 50.        // Kill the application manually and terminate the script 51.        Exec kill =1 52.        Exit "2" desc="Failed to close the demo app through the Close button!" 53.    } 54.    Step "Closed the demo application through the Close button." actual="OK"

The script tests a very simple scenario:

1. Start the demo application.
2. Verify that the application started properly. This is achieved though searching the screen for the Close button icon.
3. Click the Close button.
4. Verify that the application shut down properly. The script ensures that the Close button is no longer visible.
   

If you set the variable APP_HOME variable (line #3) to the location where you unzipped the demo application you can execute the script right away. Requirements:

• Java installed and put on the system path (presume you've already done it under 3. Create Project)
  
• T-Plan Robot 3.5.2 or higher (or at least 3.5.2Beta from 19 May 2014 or newer) which introduced support of the "Exec kill" command. The script will also execute on older releases but it will not kill the application if it for some reason fails to close. The Jenkins build will then hang because it will wait for all processes to complete.
  

In order to produce a reasonable test report the script creates a few result objects. It creates an XML report at line 17. It gives the script a name and a number (line 23) and it creates two step results (lines 40 and 54) and a screen shot (line 41). To view the resulting report see the Test Results tree node of the Robot project or open the online copy here.

In the next step we will integrate the test script into Continuous Integration.

6. Integrate Automation

Integration with CI is enabled thanks to the T-Plan Robot command line interface (CLI). The most important options:

CLI Option	Description
-r/--run <path>	Execute the test script or schedule specified by <path>.
-c/--connect -p/--password	Connect to a test environment. The password is optional and should be used only where required, such as for example for VNC Server connections. Connection can be also established directly from the script through the Connect command (TPR scripts) or connect() method call (Java scripts).
-n/--nodisplay	Execute in the CLI mode without displaying the GUI. It is recommended for production environments because this mode consumes less system resources (CPU, RAM). This option is required for remotely started executions (telnet, rsh/ssh).
-v/--variable <name=value>	Override a variable value. This option may be specified multiple times to override multiple variables. It is typically used to parametrize the script execution or to change the behavior through setting of the standard (predefined) variables. For example, to make the script save the report to C:\reports specify: -v "_REPORT_DIR=C:\reports"

• "Test script" -> choose Jenkins.tpr from the demo project
• "File paths" -> absolute
• OPTIONAL: Set "Connect to" to "java://localhost". This is not really needed because our script calls the Connect command to start the connection on its own.
  
• OPTIONAL: Set on "Execute the script in the CLI mode"
• OPTIONAL: We will further instruct Robot to store the report (test results) to a unique directory under the Jenkins workspace. To do so we will set the _REPORT_DIR Robot standard variable to a value like:

%WORKSPACE%\reports\%BUILD_NUMBER%

The WORKSPACE system variable will be set by Jenkins to the application workspace path. The BUILD_NUMBER one will contain the unique build number. For example, if the workspace is C:\Users\JohnDoe\.jenkins\jobs\DemoApp\workspace and the build number is 3 the report will be stored to


C:\Users\JohnDoe\.jenkins\jobs\DemoApp\workspace\reports\3


This will make the report available at the following URL. You may also navigate to the report from Jenkins through the Workspace link in the project's (job's) menu.

http://localhost:8080/job/DemoApp/ws/reports/3/results.xml

This allows you to send the test result URL by email after every build. This can be achieved through the Email-ext (Email Extension) Jenkins plugin. To enable valid URLs you will also have to go to Manage Jenkins -> Configure and set the Jenkins Location to a valid host name. For other report options see the Advanced Tips chapter.

• Once you are done with the CLI Wizard click the Copy To Clipboard button.

To integrate the automation to the build process simply add a new build step:

1. Go to Jenkins home, click DemoApp and select "Configure" from the left menu.
2. Click "Add Build Step" in the Build section and choose "Execute Windows batch command"
3. Paste the CLI command into the Command field.

4. Save the changes and select Build Now. Jenkins will now build the application AND run the automation.
5. The Jenkins build will succeed only if the build and automation finish successfully. It will fail if the build or automation fails. The test results will be stored to a unique build-specific folder in the Jenkins workspace and can be viewed in the web browser.

7. Advanced Tips

Automation under Jenkins running as a service

Jenkins running as a service is typically restricted in two areas:

1. The process doesn't have permissions to start GUI applications. This prevents it from integration with Robot automation based on the Local Desktop and iOS Mirror connections. Other "remote" connections such as VNC Server (except for Local Desktop over VNC) and Android Over ADB can be integrated with Jenkins in the service mode provided that you start Robot with the -n/--nodisplay CLI option.
2. The process usually runs under a user account with limited file system permissions. Impacts on Robot:
• If there's no valid user home folder the Robot process will fail to save changes to the user configuration file. You will see a few exceptions in the log but the program will continue using the default factory settings. As Robot stores the license key location to the configuration file you may see the process fail for a missing license. To work around this save the key file to the Robot's installation directory from where it will be picked up automatically. Alternatively add a CLI option like:
  
  
  -o "config.licenseKeys=<license_key_path>"
  
  The -o/--option option may be specified repeatedly to customize preferences from the configuration file. Robot version 4.0 will address these limitations with the new --cfg and --licensekey options.
• If the process has no permissions to create folders in the default system temporary location the application log service (Help->Log Viewer in the Robot's GUI) will be disabled. All logs will be printed out to the terminal (command prompt) only.
• If the process has no permissions to create files in the system temporary location the Robot process will crash or not work properly. It is necessary to grant the process at least these permissions.
  

import com.tplan.robot.*;
import java.lang.reflect.Field;

...

try {
    Field field = ApplicationSupport.class.getDeclaredField("cliMode");
    try {
        field.setAccessible(true);
        field.set(null, true);
    } finally {
        field.setAccessible(false);
    }
} catch (Exception e) {
    e.printStackTrace();
}
ApplicationSupport a = new ApplicationSupport();
...
                

Linking reports to the Jenkins UI

1. Install the HTML Publisher Plugin
2. The plugin unfortunately can't handle report paths containing Jenkins build variables such as %BUILD_NUMBER%. It can only link to static locations in the project workspace. To work around this we have to make the Robot script create an HTML copy of the report in a stable directory. To achieve it add the following command to the Jenkins.tpr script:

Report "..\latest\results.html" desc="Testing of the Jenkins demo application, build {_ENV_BUILD_TAG} ({_ENV_BUILD_ID}) located at {_ENV_WORKSPACE}"

3. Select Configure in the DemoApp menu. Select "Add post-build action" and choose "Publish HTML reports". Configure the parameters as follows:

HTML directory to archive:	reports\latest
Index page[s]:	results.html
Report title:	Latest T-Plan Robot Report

4. Save the changes and execute the build to make the link appear in the menu.
   

8. Conclusion

With the steps above allow you will be able to integrate T-Plan Robot automated testing with the Continuous Integration tool of your choice. The principles of the CLI integration can be applied to integration with virtually any other framework.

The document does not cover other test environment configurations involving multiple test machines. These will be discussed later.

Should you have any question please contact the T-Plan Support team.