Automating Android Over The ADB

Last update: 18 November 2013

Contents:
1. Introduction
2. Setting Up
3. Troubleshooting
4. Automation
   4.1 Connecting To The Device
   4.2 Dealing With Screen Rotation
   4.3 Touch Screen Automation
   4.4 Keyboard Automation

1. Introduction

T-Plan Robot Enterprise version 3.1 and higher supports automation of Android devices over the ADB (Android Debug Bridge). ADB is a tool delivered as part of the Android SDK which provides connectivity to Android devices attached to the USB.

Compared to automation over VNC (namely VMLite VNC and DroidVNC):

Automation over ADB	Automation over VNC
+ The device doesn't have to be rooted	- DroidVNC: The device must be rooted + VMLite VNC: The device doesn't have to be rooted
+ No network required (only USB)	- DroidVNC: Network connection required (Wi-Fi) + VMLite VNC: Network or USB connection required
+ Good support across devices and Android OS versions	+ Good support across devices and Android OS versions
+ No SW installation on the device required	- The device must have the VNC server installed & started
+ Good support of the Android native keys	- Support of the Android native keys is subject to the server vendor
- Slower screen refresh rate for large screen devices due to lack of image compression in the Android SDK	+ Good screen refresh rate for large screen devices (depends on the preferred image encoding)
+ Support of both automatic and manual screen rotation	- Manual screen rotation only supported
+ Support of pinch & zoom gestures (Robot 3.5+/Android 4.3+)	- No support of pinch & zoom

The nature of the ADB connection makes this approach suitable for non-intrusive automated testing of Android applications across multiple device types, preferably with smaller screen resolutions.

2. Setting Up

Though the Android SDK is supported on Windows, Linux and Mac OS X, it is recommended to use a MS Windows host for the automation. Automation on Linux (namely Ubuntu Linux which we used for testing) requires to run Robot from the root account and connection to the device is unstable, especially under a high load during test script execution. Though Robot can in most cases repair the connection automatically it leads to delays and lower reliability.

Set up instructions:

Download and install the Android SDK revision 18 or higher on your computer. If you have and older version of the SDK upgrade it to the latest one using the <SDK_path>/tools/android tool.
Enable the "USB Debugging" option on your Android device:

Android 4.1 and lower: Go to Menu->Settings->Applications->Development and set on the "USB Debugging" option.

Android 4.2 and higher:

Go to Menu->Settings

Select the About menu in the More or General group. On some devices it is located under Settings->About->Software->Information->More.

Scroll to Build Number and tap it 7 times until you see a message saying “You are now a developer” or ”Developer mode has been enabled”.

Go back to the Settings menu, select the Developer Options and turn on the slider at the top to enable the options. Then set on USB Debugging.

Connect the Android device through the USB cable to the computer. It is not recommended to connect the device after the connection is initiated from Robot because it leads to intermittent connection crashes.
MS Windows only: Add the folder containing adb.exe to the user or system path:

Open Windows Explorer (Windows+E)
Right click Computer and select Properties
Select "Advanced System Settings" and click the "Environment Variables" button
Select the PATH user variable, click Edit and insert the path to the folder containing adb.exe to the beginning of the list. To separate the path from the original value use semicolon, ';'. The new value should look like:

<SDK_path>\platform-tools;<originalPaths>
Save with OK.

Start T-Plan Robot Enterprise. If your system is a Linux/Unix one you will have to start it from the root account (sudo may not be sufficient).
In the Login Window, set the "Connection Type" to "Android over ADB" and provide the path to the Android SDK. This path will be saved to the user preferences and it will get populated automatically onwards.
If you have two or more Android devices connected to the computer, set the "Device Serial No." field to the serial number of your device. You may find it out through clicking of the "..." button or by executing "adb devices" from the platform-tools/ folder of your Android SDK. Alternatively leave the field on "default" to connect to the first detected device.
Hit Connect to connect to the device.

3. Troubleshooting

To resolve issues with the Android connection take advantage of settings available under the "Android over ADB" panel of the Edit->Preferences window:

Should you experience any issues please set on the Debug mode to produce logs. These can be viewed in the console (command prompt) or the log file through the Help->Log Viewer window.
When the device screen image is distorted or displays in wrong colors change the decoding method:

The default "Android SDK" decoder relies on the Android SDK to decode the image data into the Java format. This approach is known to fail for some devices, for example Samsung Galaxy GT-I5500 which displays with a greenish screen.
The "Robot" one decodes the data on its own. This method is slightly slower but seems to work for most failing devices.

On unresolvable screen issues set on the "Create the screen image dump" option, make an attempt to connect to create the dump and then send the log file through the Help->Log Viewer window to T-Plan support. Then set off the option. The dump will allow us to reconstruct the image from the raw pixel data and inspect why the decoder fails.

If Robot fails to connect to the target device for no obvious reason and the log or the Login Window provides no reasonable explanation:

Disconnect the device from the USB cable and connect it again.
Restart the ADB server manually:

Open a console/command prompt and switch to the platform-tools folder under the Android SDK installation path.
Run: adb kill-server
Verify that the device is connected to ADB: adb devices

Limitations of the keyboard and touch screen support are listed in the Automation chapter. Should you experience any outstanding issues please contact the T-Plan support and attach the log file.

4. Automation

4.1 Connecting To The Device

To connect to an Android device from a test script use the Connect command (TPR scripts) or the connect() method (Java test scripts). The argument URL must be in form of either "adb://default" or "adb://<device_serial_No>". The Android SDK path must be configured prior to running the script either by providing the path in the Login Window and connecting to the device once or by setting it in the "Android over ADB" panel of the Preferences window.

For example, the following command/method call will connect you to the first detected device:

TPR test scripts: Connect adb://default
Java test scripts: connect("adb://default");

To connect to a device with the serial number of MB104PY10519:

TPR test scripts: Connect adb://MB104PY10519
Java test scripts: connect("adb://MB104PY10519");

4.2 Dealing With Screen Rotation

The ADB connection supports two screen rotation modes. To change the mode go to the "Android over ADB" panel of the Preferences window.

1. Auto screen rotation (default)

The auto mode will read the screen orientation from the device on every screen refresh and rotate it accordingly. As this has a significant impact on the screen performance it is recommended to switch to the manual mode or to design the test suite against a constant screen orientation where high performance is required.

2. Manual screen rotation

In this mode the screen must be rotated in the Robot view manually (or from the script) through pressing of the F4 key. This behavior is compatible with the Droid VNC server.

To ensure that the touch screen events are properly placed and the image comparison works as expected always keep the view in Robot consistent with the real device state:


Correct view		Incorrect view Search for components using the image comparison will not work and the touch screen events will be misplaced.

4.3 Touch Screen Automation

The touch screen actions are mapped as follows:

Device Action	Computer Action	Test Script Action
Tap (touch)	Left mouse click	Call the Mouse click command (TPR scripts) or one of the mouseClick() methods (Java scripts).
Long tap (touch)	Long left mouse click	Call a sequence of Mouse press, Wait and Mouse release commands or the corresponding Java methods.
Swipe (drag)	Mouse drag (left button down)	Call the Mouse drag command (TPR scripts) or one of the mouseDrag() methods (Java scripts).
Pinch (zooming with 2 fingers)	Pinch/zoom action	Call the Mouse pinch/zoom command (TPR scripts) or one of the mousePinch() or mouseZoom() methods (Java scripts). Supported since Robot 3.5. Requires device running Android OS 4.3 or higher.
Back	Right mouse click	Call the Mouse click btn=right command (TPR scripts) or one of the mouseRightClick() methods (Java scripts). Alternatively use "Press Back" or press("Back").

Robot v3.5.2 and newer supports an option to perform mouse clicks over the UIAutomator interface. This usually resolves occasional touch screen failures for certain app controls. It requires Android 4.1.2 or newer.

To set on the UIAutomator driven mouse clicks globally select the "Enable click/tap over the UIAutomator API" user preference in the Edit->Preferences->Android Over ADB screen. As this gets saved to the user preferences file it will affect all Robot instances running on the local PC. The option may be tweaked for a single Robot instance through the -o/--option CLI option.
To control the UIAutomator clicks temporarily from the script using Robot 4.0.1 or higher set the _UIACLICK variable to true ("enable") or false ("disable"). This mechanism overrides the above user preference. It has no impacts on other test scripts executing concurrently within the same Java process. For example, to enable it just for one particular click use:

4.4 Keyboard Automation

To make the script type a text call the Type or Typeline commands or the type()/typeLine() Java methods. To press a keyboard key call the Press command or the press() method. The keyboard events are supported as follows:

All printable ASCII 7-bit characters (codes 32-127) and part of the keys from the 0-31 range (Enter, Backspace, ...) are supported except for greater than (>), less than (<), double quote (") and circumflex (^) which don't have their counterparts in the Android SDK API.
The only supported modifier keys are Shift and Alt. The Ctrl modifier is not supported because it doesn't have its counterpart in the Android SDK API.
Some characters out of the ASCII 7-bit set can be generated through combinations of Alt+<character>. For example "Press Alt+Z" generates the Euro character on some devices. These composed keys are subject to the device, its language and keyboard settings and they must be explored per device.
The PC keyboard action keys are mapped onto the native Android keys as follows. Support of these keys is subject to the particular Android device and some keys may not have any effect:

Android Device Key	Computer/Script Key (Action)	Description
BACK	Escape, Right mouse click	Back key.
CALL	PageDown	Call key.
CAMERA	F6	Camera key. Used to launch a camera application or take pictures.
CLEAR	F7	Clear key.
DEL	Backspace	Backspace key. Deletes characters before the insertion point.
DPAD_DOWN	Down (Arrow Down), Mouse wheel down	Directional Pad Down key. May also be synthesized from trackball motions.
DPAD_LEFT	Left (Arrow Left)	Directional Pad Left key. May also be synthesized from trackball motions.
DPAD_RIGHT	Right (Arrow Right)	Directional Pad Right key. May also be synthesized from trackball motions.
DPAD_UP	Up (Arrow Up), Mouse wheel up	Directional Pad Up key. May also be synthesized from trackball motions.
ENDCALL	End	End Call key.
FOCUS	F5	Camera Focus key. Used to focus the camera.
HOME	Home	Home key. This key is handled by the framework and is never delivered to applications.
MENU	PageUp	Menu key.
NOTIFICATION	F9	Notification key. Used to display the list of notifications.
POWER	F10	Power key.
SEARCH	F3	Search key.
SOFT_LEFT	F1	Soft Left key. Usually situated below the display on phones and used as a multi-function feature key for selecting a software defined function shown on the bottom left of the display.
SOFT_RIGHT	F2	Soft Right key. Usually situated below the display on phones and used as a multi-function feature key for selecting a software defined function shown on the bottom right of the display.
SYM	F8	Symbol modifier key. Used to enter alternate symbols. On most devices it pops up the "Select input method" window.
VOLUME_DOWN	F11	Volume Down key. Adjusts the speaker volume down.
VOLUME_UP	F12	Volume Up key. Adjusts the speaker volume up.