vnc

Automating Over VNC Server

Contents:

1. Introduction
2. Supported Configurations
3. Supported VNC Servers
4. Fine Tuning

1. Introduction

The VNC Server connection allows T-Plan Robot Enterprise to automate machines and devices over the Remote Frame Buffer protocol (RFB), which is better known as Virtual Network Computing or VNC. As the VNC technology has been adopted by many products this connection is not limited to computers. It may automate in general any device running an RFB 3.x compatible VNC server. Robot has been successfully deployed to automate a wide range of systems such as:

  • PCs
  • Mobile phones
  • KVM switches
  • POS machines

Advantages:

  • Simplicity
  • Wide adoption, supported by multiple OS
  • Good performance over TCP/IP networks due to image encodings (compression)

Disadvantages:

  • Keyboard support limited to the Latin 1 character set (ISO 8859-1)
  • No support of multi finger gestures on touch devices
  • Limited security (plain password authentication), can be resolved through an SSH tunnel

2. Supported Configurations

VNC Server can be used in a number of configurations:

2.1 Local Desktop Automation

Local desktop automation presumes that Robot automates the same system (desktop) the VNC server runs on. This is typical for single OS solutions relying on MS Windows or Mac OS X. When automation is in progress or when a screen shot is being made the Robot GUI typically hides to allow an undisturbed view of the automated desktop.
To set up this configuration:

  1. Install one of the supported VNC servers and start it.
  2. Install Robot.
  3. Start Robot. Choose the "VNC Server" connection type and enter "localhost" into the Server field. Do NOT use the "127.0.0.1" which is reserved for other configurations. If the server runs on a custom port other than 5900 specify the port after a colon ("localhost:5901")
    An alternative to this configuration is the Local Desktop connection. It automates the the local desktop through the means of the local Java installation. No other component such as the VNC server is required. The disadvantage of this connection is that test scriptscan not be started remotely (from another PC or device) through a terminal such as telnet or rsh. Use the VNC Server instead.


2.2 Single OS With Multiple Desktops


Configurations with a single machine/OS with multiple desktop instances typical for Linux/Unix which allow to run multiple VNC servers (desktops) on a single system. Each server instance runs on its own port and provides a standalone graphical desktop independent from the default system one (the one you see on your screen). The machine in this case serves both as the client system and the SUT. T-Plan Robot typically (but not necessarily) runs on the default system desktop (as is displayed in the picture) and connects to the VNC server(s).
The steps to configure:

  1.   Install any supported VNC server and start it.
  2.   Install Robot.
  3.   Start Robot. Choose the "VNC Server" connection type and enter "127.0.0.1:<port>" into the Server field. Do NOT use the "localhost" name which is reserved for other configurations.

2.3 Single Machine With Multiple OS Instances

This configuration takes advantage of virtualization technologies such as VirtualBoxor VMware. In this scenario your default OS (called "hosting system" in virtualization terminology) runs a virtual machine (VM) with its own OS (called "hosted system"). There's an example of Windows Vista hosting a Windows XP system in VirtualBox on our web site. The hosting and hosted systems may be any combination of OSes supported by the particular virtualization technology.

  • The hosting system typically runs Robot and plays the role of the client system. Though Robot may run on another dedicated VM instance this configuration is not recommended with regard to a number of environment specific issues reported by T-Plan Robot users. Be aware that the client and server roles can not be reversed and the VM can not remote control (and automate) the hosting system.
  • The hosted system serves runs a VNC server.

Configuration instructions:

  1. Install VirtualBox or VMware.
  2. Create a VM and install the hosted OS in there.
  3. Install any supported VNC server on the hosted system and start it.
  4. Configure the VM to make the VNC port visible from the hosting system (VirtualBox steps).
  5. Install Robot on the hosting system.
  6. Start Robot. Choose the "VNC Server" connection type and enter the VM IP address and port into the Server field, for example "192.168.100.2:5901".

NOTES:

  • VNC servers running on VMware have been reported to have problems with key mapping where the client and server have different keyboard layout (see example[1], example[2]). This typically results in some characters being typed incorrectly on the VNC desktop. This is not T-Plan Robot Enterprise failure.
  • Since Robot 4.3+ you can connect to a VirtualBox VM directly using the RDP Server connection.

2.4 Dual System Environment

This scenario presumes that you have a stable dedicated device (another PC, a mobile device, KVM switch, POS machine etc.) runs the VNC server. This configuration is recommended for automation production scenarios. As the System Under Test (SUT) is physically a separate machine it is easy to keep it in a stable state, make necessary back ups or even set up a routine to restore the system after each test cycle. As the device is connected to the network it may also serve as a test environment for multiple users (client systems) over intranet or internet. This configuration is also the only one possible when the SUT doesn't meet requirements for running T-Plan Robot Enterprise (systems not supported by Java SE) and thus a single machine scenario can not be used. Configuration steps are as follows:

  1. Install any supported VNC server on the SUT and start it.
  2. Install Robot on the client system.
  3. Start Robot. Choose the "VNC Server" connection type and enter "<machine_name_or_IP>:<port>" into the Server field. 

3. Supported VNC Servers

Automation through the RFB protocol requires the System Under Test (SUT) to run a VNC server. A good overview of existing VNC products is on Wikipedia, both in the VNC and Comparison Of Remote Desktop Software topics. T-Plan Robot Enterprise should work fine with any VNC server which is RFB 3.x compatible. 

The "vncconfig" utility has to run on your server to make the clipboard transfer working. As some VNC servers do not distribute it (for example TightVNC) the feature may be switched off in T-Plan Robot Enterprise. If you plan on using clipboard changes to transfer text from server to client get a VNC server which has it such as UltraVNC or RealVNC.

Desktop PC Platforms

  • MS Windows - TightVNC, RealVNC, UltraVNC
    • It is recommended that you run VNC server on Windows as a service. This will allow you to restart Windows from T-Plan Robot Enterprise and access the login screen after the system restarts. To send system reserved key combinations like Ctrl+Alt+Delete use the Keys tab situated in the top left corner of the T-Plan Robot Enterprise GUI.
    • If you connect T-Plan Robot Enterprise to a Windows server running TightVNC, you may experience a refresh problem. Application windows sometimes display on the remote desktop without content and pieces of the window image appear as user moves the mouse pointer over the window. To prevent this behavior open the TightVNC settings window and select the 'Poll full screen' check box.
    • Windows specific keys like 'Windows' and 'Properties' may be reproduced in scripts through the Press command as long as the VNC server supports them. The keys work fine on RealVNC and UltraVNC. Should you experience any issues make sure to switch on the scroll lock (this is a workaround described in UltraVNC forums). TightVNC 1.3.10 doesn't support the Windows specific keys but planned to support them in the 2.0 release.
  • Unix/Linux - TightVNC, RealVNC, Remote Desktop (built-in system feature of some distributions such as Ubuntu)
    • Most Linux distributions already contain a VNC server in the package repository and allow to install it through the package manager. To find out whether the software is installed on your machine try to run vncserver in a terminal.
    • The autocutsel utility can be used on Linux/Unix to make the clipboard transfer work instead of vncconfig. It must be executing on the server as "autocutsel -s PRIMARY". If you are running Debian or Ubuntu, you may find the tool in the package repository. Other resources also mention xcutsel but we haven't tested it. Be aware the RFB client can transfer only characters from the Latin-1 (ISO8859-1) character set. This is limitation set by the RFB protocol and we can't do anything about it.
  • Mac OS X - Apple Remote Desktop (built-in Mac OS X system feature), RealVNC, Vine
  • Solaris OS - RealVNC or the default VNC server bundled with Solaris 10 Update 5 or higher.
  • HP UX - RealVNC
  • AIX - RealVNC
  • Other - Canon Remote Operator's Software Kit

Portable Devices

For mappings of the phone keyboard onto standard PC keyboard events see documentation of the particular server you are using.

Be aware that many mobile VNC servers do not fully comply with the RFB 3.x specification and may crash for the standard VNC session settings. When a connection failure is experienced, it is recommended to reconfigure the RFB client on Robot's side (at Edit->Preferences->RFB (VNC) 3.x Client) to the minimal configuration (disable all encodings except the Raw one and disable the custom pixel format). It is also a good idea to set on the Console debug logging preference to get debug messages printed out into the console window (command prompt). If the connection succeeds this time, try adding the encodings and/or setting the pixel format one by one to identify the one which causes the crash. Most problems are due to enabled Cursor or Zlib encodings or when the pixel format is forced to a one which the server can not handle.

Summary of suitable servers for mobile devices:

Some mobile servers such as VMLite allow to avoid the network connection by tunneling of the server communication through the USB cable. Another approach to avoid an unstable WiFi is to use a private router or a USB WiFi Hotspot device. Make sure to configure it not to apply the "AP Isolation" setting which blocks the communication among devices.

The following matrix describes the servers in details. Should you want to contribute to the list with a new unlisted server, contact us through the T-Plan Robot Enterprise Contacts web page.

VNC Server

Platforms

Status/Notes

Tight VNC

all supported by server

Tested by us on Linux & Windows. Be aware that Windows specific keys (Win, Properties) do not work on TightVNC server. The issue has been reported and it is likely to get fixed in TightVNC 1.3.11 or 2.0.

Real VNC

all supported by server

Servers for portable devices (such as mobile phones) distributed by RealVNC in form of OEM software are not compatible and will not work with T-Plan Robot Enterprise. RealVNC Free Edition relies on the standard RFB v3.x protocol and works out of the box. RealVNC Personal Edition and RealVNC Enterprise Edition work on a proprietary enhancement of the RFB protocol coded as v4.x. T-Plan Robot Enterprise can work with these servers only in the 3.3 protocol compatibility mode which must be configured on the server side as follows:
1. On the Connections tab of the VNC server window change the settings:
  Authentication - None
  Encryption - None
  Prompt VNC Server user to approve connections - Untick

2. On the Expert settings tab:
  Protocol 3.3 - True

UltraVNC

all supported by server

Tested by us; reported to work by users.

Remote Desktop(Ubuntu feature compatiblewith VNC)

Ubuntu Linux

Tested by us. To enable the access:

  1. Select System->Preferences->Remote Desktop in the OS menu
  2. Set on the "Allow other users to view your desktop" and "Allow other users to control your desktop" check boxes
  3. Set off the "You must confirm each access to this machine" check box
  4. As the desktop server runs on the default VNC port of 5900, to connect from Robot use either the IP alone (such as "192.168.100.10") or the IP with the port number ("192.168.100.10:5900").

Apple Remote Desktop (ARD; Mac OS featurecompatible with VNC)

10.4 PPC Mac10.5 and higher (Intel Mac)

Tested by us (legacy platforms were reported to work by users). To make ARD work with T-Plan Robot Enterprise perform these steps on the Mac OS X desktop:

  1. Start System Preferences from the system main menu (Apple icon)
  2. Open Sharing in the Internet & Wireless section
  3. Tick the Screen Sharing check box in the list 
  4. Click the Computer Settings button, set off "Anyone may request permission", set on the "VNC viewers may control screen with password" and enter a password
  5. Confirm with OK (authorization may be required) 
  6. The window displays description containing an URL like "vnc://192.168.100.10/". As the desktop server runs on the default VNC port of 5900, to connect from Robot use either the IP alone (such as "192.168.100.10") or the IP with the port number ("192.168.100.10:5900").

Canon Remote Operator's Software Kit


Reported to work by users. Requires T-Plan Robot Enterprise version 3.1.1 or higher to display the desktop in correct colors.

Pocket VNC

Windows CE and Windows Mobile devices

Tested by us. Older versions such as PocketVNC v1.4.3 contain a bug which breaks the desktop image transfer. There's a workaround:

  1. Go to Edit->Preferences in T-Plan Robot Enterprise GUI and locate the "RFB (VNC) Client" panel.
  2. Move the Hextile and Raw encodings to be first and second in the list.
  3. Select "Use custom pixel format" and choose the "16 bit (65k colors)" item in the drop down.
    See the Windows Mobile Network article on the PocketVNC site to find out how to get your mobile connectible from T-Plan Robot Enterprise. If your mobile phone can't have an IP address but is connected to the network, use the RFB listen mode for reverse connection from PocketVNC to T-Plan Robot Enterprise. As some devices disconnect regularly from the network to save battery, Robot may fail to connect with a message like "No route to server" or "Server not found". Pinging the device IP from the PC seems to wake it up in some cases. If it doesn't help, reconnect the device to the network (WiFi) and restart the VNC server to make sure that the connection is active and the device is visible in the local network. NOTE: Pocket VNC in combination with T-Plan Robot Enterprise is one of the very few real black box GUI automation solutions for Windows mobile OS and application testing.

EfonVNC

Windows CE and Windows Mobile devices

Version 4.3 has been reported to work by users. As the server by default uses an unusual 15-bit pixel format, the screen may appear bluish. This can be fixed by making Robot to request one of the standard pixel formats as follows:

  1. Go to Edit->Preferences in the T-Plan Robot Enterprise GUI and locate the "RFB (VNC) Client" panel.
  2. Set on the "Use custom pixel format" check box and choose any standard pixel format in the drop down (32-bit is OK).

mVNC

Symbian OS mobile devices

Reported to work by users. As most keys on the device keyboard are mapped onto the PC numpad, commands automating typing on the device must use the "location=numpad" parameter. For example, to automate typing of a phone number of 123456789 use "Type 123456789 location=numpad".

Veency

iOS (iPod, iPhone)

Tested by us for basic functionality; reported to work by users. Veency requires a jailbroken (rooted) device. There's no alternative because the iOS API does not contain interfaces providing access to the necessary features. You can install Veency from Cydia which gets installed into the device during jailbreaking. Should you experience freezing display image or other connection problems perform the following steps:

  • On Veency side set off the Enable Cursor preference.
  • On Robot side navigate to Edit->Preferences, select the "RFB (VNC) v3.x Client" panel and move the Raw encoding to the top of the list. On newer Robot versions also disable the Cursor encoding (move it to the disable list on the right).

    Veency v0.9.3381 and higher running on iOS 6 may disable the keyboard and mouse input when the connection is not protected by a password. When you experience this behavior set up the password in the Veency UI and restart the device.
    For an iOS testing alternative see the iOS Mirror connection.

VMLite VNC Server

Android

The VMLite VNC Server is a commercial software available for a small fee from the Google Play store. It can run both on on rooted and unrooted devices:

  • Rooted devices allow to start the server directly.
  • Unrooted devices require you to install a small VMLite application on the PC and start the server through it while the device is connected to the USB port. The supports an option to tunnel the server connection to the PC through the USB which makes it suitable for testing of devices which are not connected to a Wi-Fi network.

For an Android testing alternative see the the Android Over ADB connection.

4. Fine Tuning

Client Side Configuration

Robot supports a number of configurable parameters which may improve performance of both the Robot application as well as the VNC connection. The following table lists the most important ones.

Parameter Name (Location)

Description

Encodings
(GUI:Preferences->RFB (VNC) 3.x Client)

Encodings specify how the image data transferred between Robot and VNC server is encoded. They are specified as an ordered list where the first (topmost) item has the highest priority. Each encoding uses a different algorithm of encoding the image data to trade off between the volume of data transferred over the network (better compression = less data) and the local CPU resources needed to decode it.

  • If your VNC server is in a remote location or the network is slow and your local CPU is reasonably fast, move one of the encodings with high compression such as Tight, Zlib or Hextile to the first place.
  • If your local CPU is slow and the network is fast, try Hextile, RRE or CoRRE which have low compression but are cheap to decode.
  • Do not prefer (set as the first) the Raw encoding. It is the least efficient one where the pixels are transferred in raw form with no compression. Keep it however somewhere low in the list because some simple servers (such as mobile device or embedded system ones) support just this one and require it to be present.
  • Keep the Cursor encoding in the list (in any place) unless you experience problems (some servers don't like it and tend to crash when it is present). It makes the client (Robot) render the cursor locally rather then encoding it into the desktop image. This leads to significant desktop performance improvements on slow network environments. It also makes the image comparison easier and more reliable.

To observe efficiency of encodings set the "Console debug logging" parameter in the same panel to "Full" and check the console window (command prompt) for performance data.

Connection pooling (Java API)

Connection pooling allows to reuse server connections which avoids the overhead of reconnection. This mechanism can be applied just from the Java API. See the RemoteDesktopClientFactory class documentation for details.

Server Side Configuration

There are several simple recommendations which may significantly improve performance of the client-server connection:

  • Change the server desktop background to a solid color which can be efficiently compressed through encodings. Pictures or photographs significantly increase volume of image data transferred over the network.
  • Remove all unnecessary dynamically updating objects from the desktop, such as clocks or gadgets showing pictures in a loop. These components are sources of frequent desktop updates and increase network traffic.
  • Try servers from different producers if available. Some products come with custom video drivers or hook ups to boost performance.
  • Use an appropriate remote desktop size (resolution). The volume of image data is linear to the desktop size.