T-Plan
                  Home
T-Plan Robot Enterprise 4.2.2
Doc Collection

10/07/17

iOS Mirror Automation

Contents:
1. Introduction
2. Robot Machine Configuration
3. iOS Device Configuration
4. Automation

1. Introduction

T-Plan Robot Enterprise version 3.3 introduced a new connection type called "iOS Mirror". This allows Robot to test Apple iOS devices using our iOS 7&8 Plugin, or iOS 6 Server WiFi connection, where the device screen is displayed on the local desktop using the Apple's 'AirPlay Mirroring' technology or transferred over the Lightning USB cable (see below). This connection allows Robot to test iOS applications on non-jailbroken (non-rooted) devices, where before only a VNC Server connection was supported.

In environments where jailbroken devices are supported, the iOS Mirror connection can be used to supplement the VNC Server (Veency) operation, for increased performance and suitability. E.g. Game testing relying on the OpenGL technology.

Robot version 4.1 delivered two major improvements:

It is recommended to use the T-Plan iOS 7&8 Plugin or iOS 6 Server, instead of VNC due to a number of advantages:


iOS Mirror connection with
T-Plan Plugin / Server
iOS Mirror connection with
VNC Server (Veency)
VNC Server connection with
VNC Server (Veency)
Supported since
Robot v3.5
Robot v3.3
Robot v2.0
Jailbroken device required
NO
YES
YES
Support of apps with OpenGL content
YES
NO
NO
Pinch/Zoom support
YES
NO
NO
UTF-8 character support
YES
NO*
NO*
Display orientation support
YES
LIMITED**
NO
Connections per machine
1*** (Mirror apps)
NO LIMIT**** (Lightning USB)
1*** (Mirror apps)
NO LIMIT**** (Lightning USB)
NO LIMIT***

Device Requirements:

Robot Machine Requirements:

Operating System
Screen Mirroring Technology
Lightning USB Cable
AirServer
X-Mirage
Reflector
Mac OS X 10.7+
YES
NO
YES
YES
MS Windows Vista+
NO
YES
YES
LIMITED

2. Robot Machine Configuration

MS Windows Set Up

Mac OS X Configuration

Accessibility Configuration

Accessibility Configuration

AirServer / Reflector Set Up

While the mirror app is disconnected right click its icon in the system tray (Windows) or go to the Device menu (Mac OS). Select Preferences/Settings. Set the options as follows:

For Reflector 2 also configure:

Robot version 4.0.1 will start the mirror app automatically when the iOS Mirror connection is initiated. For lower versions connect the Apple device to the mirror app: Post connection configuration:

Robot Set Up


3. iOS Device Configuration

As detailed in the Introduction there are two basic high level options:

  1. Non-Jailbroken (non-rooted) devices can be automated using our T-Plan Plugin or T-Plan Server tool with the iOS Mirror connection.
    Note: The iOS Mirror connection can be applied to any device supporting the AirPlay Mirroring technology (as per this table). Mirroring over the Lightning USB cable is possible only for devices equipped with the Lightning connector.
  2. Jailbroken (rooted) devices can be automated using the Veency (Cydia) VNC Server connection.

A. Non-Jailbroken Devices

1. Device Configuration

It is recommended to set off the Auto Lock feature to prevent the device from going to sleep after the specified idle time:

2. Configure Device with T-Plan iOS 7&8 Plugin or iOS 6 Server

The T-Plan iOS 6 Server enables testing of iOS applications on iOS version 5 & 6, using the iOS Mirror connection.

The T-Plan iOS 7&8 Plugin enables testing of iOS applications on iOS version 7 and higher, using the iOS Mirror connection.

3. Start Mirroring


B. Jailbroken Devices Only

1. VNC Setup (Only for Jailbroken devices)

2. Start Mirroring


4. Automation

Connection from Robot

Finding iOS IP address
iOS Mirror parameters
Example 1: Connect to the T-Plan Plugin/Server over TCP/IP using a mirror app:
TPR test scripts:     Connect apple://192.168.100.1:5909
Java test scripts:    connect("apple://192.168.100.1:5909");
Example 2: Connect to the T-Plan Plugin/Server over USB using a mirror app:
TPR test scripts:     Connect apple://localhost:5909
Java test scripts:    connect("apple://localhost:5909");
Example 3: Connect to the VNC Server using a mirror app:
TPR test scripts:     Connect apple://192.168.100.1:5901
Java test scripts:    connect("apple://192.168.100.1:5901");
Example 4: Connect to the T-Plan Plugin/Server over TCP/IP using Lightning USB:
TPR test scripts:     Connect apple://192.168.100.1:5909 device="My iPhone 5"
Java test scripts:    connect("apple://192.168.100.1:5909", null, false, "My iPhone 5");
Example 5: Connect to the T-Plan Plugin/Server over USB using Lightning USB:
TPR test scripts:     Connect apple://localhost:5909 device="My iPhone 5"
Java test scripts:    connect("apple://localhost:5909", null, false, "My iPhone 5");

Automation of multiple devices from a single PC

The preferred way of automation of multiple devices from a single PC is the Lightning USB connection on Mac OS X. For other environments the support is limited to X-Mirage and Robot v4.0.1 or higher. Avoid using the USB tunneling so that the devices can be identified by the IP address. The steps:

*If the mirrors are not running Robot will start them (Robot v4.0.1+ with T-Plan libs v1.0+). This will however make image comparison unreliable because start of each new mirror will make X-Mirage resize all other mirrors to fit the PC screen.

Troubleshooting of mirror apps

Image comparison tips

As the iOS device screen is mirrored using H.264 video with a lossy image compression format, the artifacts (components) on the screen change at the pixel level. To search for them using Robot template images use the search2 comparison method with the pass rate of 50% or lower.

Starting with the 4.0.2 the test scripts may force the screen displayed by a mirror app to a fixed size. This ensures that the images will match even if the mirroring app changes the window size, for example as a result of an update. To set the screen size populate the _DESKTOP_SIZE variable before or after the connection with the target width and height as follows:
Desktop size setting
This functionality is currently supported only for AirServer and Reflector on MS Windows. Future releases are planned to deliver support for the Mac OS. The X-Mirage mirror app is not suitable because it doesn't scale the mirrored screen on window resizing.

To apply the target size to the current iOS Mirror screen highlight (select) the variable code line and click the Selection tool bar button. This will execute the variable command and set the screen size. See the Execute Selection for details on code block execution. Future releases will deliver support of screen size forcing through the GUI tools such as the Login dialog and the Connect command property window.

Automation Tips

  1. Double press the Home button on the iOS device to open the list of running applications.
  2. Scroll to the right (swipe to the left) if needed to locate the T-Plan Server application.
  3. Long press the application icon until it starts shaking. Then press the "red minus" icon to kill it. Finally press Home to go back.
Kill T-Plan Server