T-Plan Robot Enterprise 4.4.7 Doc Collection

29/01/19

Automating iOS Over Xcode

Last update: 24 January 2018

Contents:
1. Introduction
2. Setting Up
3. Connecting From Robot
4. Troubleshooting
5. Automation

1. Introduction

NOTE: For the older document describing automation of iOS 9 devices over Xcode 7 go here.

T-Plan Robot Enterprise version 4.3.1 and higher supports automation of iOS 10+ devices over Xcode 8 and higher on Mac OS X. Main features:

Full iOS control enabled without jailbreak.
No modification of the tested application is required.
Device must be connected both to a WiFi network and to Mac OS X over the Lightning USB cable.
Support of automation in the Portrait and Landscape Left (with the device button on the right) screen orientations.
Extra set of features provided through the iOS Over Xcode Extension plugin.

As the connection is slower than the iOS Mirror one it is not suitable for testing of applications that require real time control, such as games etc. Use the iOS Mirror connection instead.

2. Setting Up

Requirements:

Xcode installed, up to date and configured with a valid Apple Developer account (matching provisioning profile, valid certificate chain).
The iOS device is connected over the Lightning USB cable to the Mac machine and is accepted by Xcode (listed in Window -> Devices).

1. Download Instructions:

Download the WebDriverAgent project version 4 from 20 August 2018.

There's no upgrade from the old versions. Abandon the old project and set up the new one from scratch.
Don't use the official WebDriverAgent project. We have added bug fixes and enhancements to enable the landscape mode and full iOS control. There's a switch in the connection preferences allowing Robot to accept a custom project version but we don't guarantee its functionality.

2. Project Setup Instructions:

First time set up: Follow the set up instructions in the project page (or more detailed steps here). In a nutshell:

Install dependencies: npm, Carthage and GIT
Run the ./Scripts/bootstrap.sh script

Repeated set up (after project upgrade):

Run the ./Scripts/bootstrap.sh script. If you are asked to upgrade the dependencies do so and re-execute the script.

3. Xcode Setup Instructions:

Xcode set up must be repeated:

After you update or re-download the WebDriverAgent project.
After the Xcode gets updated.
After the iOS on the target device gets updated.
If you need to set up a new device.

The steps:

Start Xcode. Open the WebDriverAgent.xcodeproj project file from the app folder.
In the Xcode project:

Select the WebDriverAgentRunner scheme and the target device at the top bar.

Select the WebDriverAgentLib target. Make sure that the "Automatically manage signing" option is on and a valid team is set.

Select the WebDriverAgentRunner target. Make sure that the "Automatically manage signing" option is on and a valid team is set.

Select Product->Test in the Xcode menu. Open the debug console through View->Debug Area->Activate Console. If everything works OK the application will start in the test mode and the console output will contain a line at the bottom:

2017-10-19 15:51:28.502141+0100 WebDriverAgentRunner-Runner[785:276949] ServerURLHere->http://192.168.1.3:8100<-ServerURLHere

To verify that the WDA server works open the inspector in Safari using the URL below. It should display a device screen shot and its component hierarchy. It's OK if the page is not populated on slower devices such as iPads where retrieval of the necessary data is painful slow:

http://192.168.1.3:8100/inspector

Robot doesn't need Xcode to run at the time of the connection. The WebDriverAgent application doesn't even have to be installed on the target iOS device. Robot will start it and eventually build it and install it on the device using the Xcode CLI tools. The manual process above is required only to ensure that the WDA project is configured properly to be built and executed using the Xcode CLI tools. It will also compile the code which shortens the connection init time.

3. Connecting From Robot

You must provide:

Name of the device you wish to connect to
Path of the properly configured WDA project

NOTE: Keyboard mapping may be very slow, especially on iPads where it may take up to 10 minutes. Use the stored keymap option to avoid the delay for repeated connections.

4. Troubleshooting

Make sure to use the original Lightning USB cable shipped with your iOS device. Apple is known for making the iOS refuse cables from other producers.
Connect the cable directly to the Mac's USB port. Do not use a USB switch or hub.

5. Automation

To connect to an iOS device from a test script use the Connect command (TPR scripts) or the connect() method (Java test scripts). The argument URL must be in form of "xcode://localhost"

For example, the following command/method call will connect you to the first detected device:

TPR test scripts:

Connect xcode://localhost device="My




























              iPhone" app="/Users/Joe/myapp.ipa"

Java test scripts:

connect("My




























            iPhone",

new
            File(

"/Users/Joe/myapp.ipa"), false, false);

Another alternative is to encode the parameters to the connect URL query. This form is also suitable for the -c/--connect CLI connection. You may copy the URL of any of your recent connections from the Tools->CLI Wizard window. Another alternative is to give your connection a name in the Desktop->Connection Manager window and then use it instead of the URL.

TPR test scripts: Connect "xcode://localhost?device=My%20iPhone&app=/Users/Joe/myapp.ipa"

Touch Screen Support

Only Portrait and Landscape Left screen orientations are supported. Other ones won't break the connection but the touch events (taps, ...) are likely to be misplaced. This is due to a bug in the underlying framework by Apple.

To change the screen orientation press the F4 key or make your test script press it (Press F4). This is supported since v4.4.3.

The mouse gestures are interpreted as follows:

Mouse clicks are mapped onto the touch screen taps. Double clicks (taps) and long clicks (long taps, "touch and hold") are supported since v4.4.2.
Mouse drags are mapped onto swipes.
Mouse wheel events are not supported and they are ignored.
Other touch screen gestures may be supported through the iOS Over Xcode Extension plugin.

Keyboard Support

Hardware key mappings:

iOS Hardware Key	Robot Script Key (Action)	Description
HOME	Home, Right mouse click	Home key. Long press and double Home press is not supported.
<screen rotation>	F4	Rotate the screen from Portrait to Landscape Left or vice versa.

Other keys are simulated through the soft keyboard. The connection acts as an end user, i.e it switches through the keyboard pages and taps individual keys (characters). Rules:

The keyboard must be visible to type anything. Typing while the keyboard is off will be ignored.
Only one keyboard locale (language) is supported at a time. Do not switch among keyboards during automation as it will result in wrong keys being typed.
The connection supports only keys that are on the current keyboard.
Some keys may not be available when the device is in the portrait mode. For example, you can't press the "Hide keyboard" key on iPhone in the portrait mode because the keyboard contains it only in the landscape one.
Extra keys from the list displayed on a long key press are not supported.
The Emoji keyboard is not supported. The workaround is to automate using mouse clicks.

Most keys on the soft keyboard are equivalent to the PC keyboard (a-z, A-Z, 0-9 etc.). The outstanding soft keyboard mappings are summarized at the table below. Availability of the keys marked with a red asterisk may be subject to the device type and screen orientation.

iOS Keyboard Key	Robot Script Key (Action)	Description
ENTER, GO	Enter (new line)	The Enter key.
DELETE	Backspace, Delete	Backspace key. Deletes characters before the insertion point.
SHIFT	Up (Arrow Up)	The shift key switches the keyboard to shifted characters. It's label and/or appearance changes depending on the keyboard page. You don't have to press this key manually to type a shifted character because the connection will switch it on its own.
MORE	Right (Arrow Right)	The shift key switches the keyboard to the next keyboard page. It's label and/or appearance changes depending on the keyboard page. You don't have to press this key manually to type a character from another page because the connection will switch it on its own.
NEXT KEYBOARD	F5	The key switches to the next keyboard locale. As the connection supports (maps) just one locale you may interact with the next keyboard only through taps. To restore the typing support you must switch back to the base keyboard locale.
DICTATION	F6	Starts dictation.

Keyboard Mapping

The connection performs keyboard mapping on the start up. It is a process where it displays all pages (called "layouts") of the current keyboard and records location of its keys. As this is a slow operation it is recommended to enable storing of the map to a file. The mapping will be performed just once and the subsequent connections will reuse it. Be aware that any keyboard change will invalidate the map, namely switching to another locale (language). Update of the iOS may also introduce changes which invalidate it.

The initial keyboard mapping is performed against the keyboard displayed for the URL field of the Safari browser. As other apps may use a different set of keyboard keys or customize the keyboard the Safari map may not work properly. To fix or re-create the keyboard map manually using the Desktop -> iOS Over Xcode Keymap Editor window:

To identify the currently displayed layout select Detect Layout.
The Create Keymap button will repeat the whole mapping process to create a new keyboard map.
To recreate or fix an individual layout displayed on the device screen select Map Layout. For example the 'Q' layout in the above screen shot is not applicable to the Notes application because Safari adds a few extra keys to it. To make the characters type properly it must be remapped.
To add a missing layout use Map Layout as well. For example the map above contains just three portrait layouts. To add the '[' one make it displayed on the screen and map it.
Use Delete to delete keys you want the scripts to ignore. Though you may also delete layouts it's not recommended unless you are about to recreate them.
To apply the changes to the current connection session click Apply To Connection.
To save the changes to the keymap file click Save To File. The map will be loaded by the subsequent connections if the stored keymap flag is enabled.

For script driven keyboard remapping check the iOS Over Xcode Extension plugin (pending).

To delete the current map file and eventually re-create the default Safari one:

Open the Login Dialog, select the "iOS Over Xcode" connection and click the "..." button next to the "Device name" field
The window will show the list of devices connected over USB and indicate whether a device keyboard map has been created. Use the "Delete Keyboard" button to delete the invalid keyboard(s).
Connect again with the stored keymap option enabled to create a new Safari map.

WARNING: The connection forces the screen orientation change during the keyboard mapping process. As this step sets off the built-in gyroscope you won't be able to change the screen orientation by manual device rotation. To re-enable this functionality re-plug the device to the USB and avoid programmatic orientation change through reconnection using the stored keyboard.