Automating Over VNC Server
The VNC Server connection allows T-Plan Robot Enterprise to automate machines and devices over the Remote Frame Buffer protocol (RFB), which is better known as Virtual Network Computing or VNC. As the VNC technology has been adopted by many products this connection is not limited to computers. It may automate in general any device running an RFB 3.x compatible VNC server. Robot has been successfully deployed to automate a wide range of systems such as:
- Mobile phones
- KVM switches
- POS machines
- Wide adoption, supported by multiple OS
- Good performance over TCP/IP networks due to image encodings (compression)
- Keyboard support limited to the Latin 1 character set (ISO 8859-1)
- No support of multi finger gestures on touch devices
- Limited security (plain password authentication), can be resolved through an SSH tunnel
2. Supported Configurations
VNC Server can be used in a number of configurations:
- Local desktop automation
- Single OS with multiple desktops
- Single machine with multiple OS instances
- Dual system environment
2.1 Local Desktop Automation
Local desktop automation presumes that Robot automates the same system (desktop) the VNC server runs on. This is typical for single OS solutions relying on MS Windows or Mac OS X. When automation is in progress or when a screen shot is being made the Robot GUI typically hides to allow an undisturbed view of the automated desktop.
2.2 Single OS With Multiple Desktops
Configurations with a single machine/OS with multiple desktop instances typical for Linux/Unix which allow to run multiple VNC servers (desktops) on a single system. Each server instance runs on its own port and provides a standalone graphical desktop independent from the default system one (the one you see on your screen). The machine in this case serves both as the client system and the SUT. T-Plan Robot typically (but not necessarily) runs on the default system desktop (as is displayed in the picture) and connects to the VNC server(s).
2.3 Single Machine With Multiple OS Instances
This configuration takes advantage of virtualization technologies such as VirtualBoxor VMware. In this scenario your default OS (called "hosting system" in virtualization terminology) runs a virtual machine (VM) with its own OS (called "hosted system"). There's an example of Windows Vista hosting a Windows XP system in VirtualBox on our web site. The hosting and hosted systems may be any combination of OSes supported by the particular virtualization technology.
2.4 Dual System Environment
This scenario presumes that you have a stable dedicated device (another PC, a mobile device, KVM switch, POS machine etc.) runs the VNC server. This configuration is recommended for automation production scenarios. As the System Under Test (SUT) is physically a separate machine it is easy to keep it in a stable state, make necessary back ups or even set up a routine to restore the system after each test cycle. As the device is connected to the network it may also serve as a test environment for multiple users (client systems) over intranet or internet. This configuration is also the only one possible when the SUT doesn't meet requirements for running T-Plan Robot Enterprise (systems not supported by Java SE) and thus a single machine scenario can not be used. Configuration steps are as follows:
3. Supported VNC Servers
Automation through the RFB protocol requires the System Under Test (SUT) to run a VNC server. A good overview of existing VNC products is on Wikipedia, both in the VNC and Comparison Of Remote Desktop Software topics. T-Plan Robot Enterprise should work fine with any VNC server which is RFB 3.x compatible.
The "vncconfig" utility has to run on your server to make the clipboard transfer working. As some VNC servers do not distribute it (for example TightVNC) the feature may be switched off in T-Plan Robot Enterprise. If you plan on using clipboard changes to transfer text from server to client get a VNC server which has it such as UltraVNC or RealVNC.
Desktop PC Platforms
- MS Windows - TightVNC, RealVNC, UltraVNC
- It is recommended that you run VNC server on Windows as a service. This will allow you to restart Windows from T-Plan Robot Enterprise and access the login screen after the system restarts. To send system reserved key combinations like Ctrl+Alt+Delete use the Keys tab situated in the top left corner of the T-Plan Robot Enterprise GUI.
- If you connect T-Plan Robot Enterprise to a Windows server running TightVNC, you may experience a refresh problem. Application windows sometimes display on the remote desktop without content and pieces of the window image appear as user moves the mouse pointer over the window. To prevent this behavior open the TightVNC settings window and select the 'Poll full screen' check box.
- Windows specific keys like 'Windows' and 'Properties' may be reproduced in scripts through the Press command as long as the VNC server supports them. The keys work fine on RealVNC and UltraVNC. Should you experience any issues make sure to switch on the scroll lock (this is a workaround described in UltraVNC forums). TightVNC 1.3.10 doesn't support the Windows specific keys but planned to support them in the 2.0 release.
- Unix/Linux - TightVNC, RealVNC, Remote Desktop (built-in system feature of some distributions such as Ubuntu)
- Most Linux distributions already contain a VNC server in the package repository and allow to install it through the package manager. To find out whether the software is installed on your machine try to run vncserver in a terminal.
- The autocutsel utility can be used on Linux/Unix to make the clipboard transfer work instead of vncconfig. It must be executing on the server as "autocutsel -s PRIMARY". If you are running Debian or Ubuntu, you may find the tool in the package repository. Other resources also mention xcutsel but we haven't tested it. Be aware the RFB client can transfer only characters from the Latin-1 (ISO8859-1) character set. This is limitation set by the RFB protocol and we can't do anything about it.
- Mac OS X - Apple Remote Desktop (built-in Mac OS X system feature), RealVNC, Vine
- Solaris OS - RealVNC or the default VNC server bundled with Solaris 10 Update 5 or higher.
- HP UX - RealVNC
- AIX - RealVNC
- Other - Canon Remote Operator's Software Kit
For mappings of the phone keyboard onto standard PC keyboard events see documentation of the particular server you are using.
Be aware that many mobile VNC servers do not fully comply with the RFB 3.x specification and may crash for the standard VNC session settings. When a connection failure is experienced, it is recommended to reconfigure the RFB client on Robot's side (at Edit->Preferences->RFB (VNC) 3.x Client) to the minimal configuration (disable all encodings except the Raw one and disable the custom pixel format). It is also a good idea to set on the Console debug logging preference to get debug messages printed out into the console window (command prompt). If the connection succeeds this time, try adding the encodings and/or setting the pixel format one by one to identify the one which causes the crash. Most problems are due to enabled Cursor or Zlib encodings or when the pixel format is forced to a one which the server can not handle.
Summary of suitable servers for mobile devices:
- Windows CE and Windows Mobile - PocketVNC, EfonVNC
- Symbian (S60) - mVNC
- Apple iOS (iPod, iPhone) - Veency
- Android - VMLite VNC Server
Some mobile servers such as VMLite allow to avoid the network connection by tunneling of the server communication through the USB cable. Another approach to avoid an unstable WiFi is to use a private router or a USB WiFi Hotspot device. Make sure to configure it not to apply the "AP Isolation" setting which blocks the communication among devices.
The following matrix describes the servers in details. Should you want to contribute to the list with a new unlisted server, contact us through the T-Plan Robot Enterprise Contacts web page.
all supported by server
Tested by us on Linux & Windows. Be aware that Windows specific keys (Win, Properties) do not work on TightVNC server. The issue has been reported and it is likely to get fixed in TightVNC 1.3.11 or 2.0.
all supported by server
Servers for portable devices (such as mobile phones) distributed by RealVNC in form of OEM software are not compatible and will not work with T-Plan Robot Enterprise. RealVNC Free Edition relies on the standard RFB v3.x protocol and works out of the box. RealVNC Personal Edition and RealVNC Enterprise Edition work on a proprietary enhancement of the RFB protocol coded as v4.x. T-Plan Robot Enterprise can work with these servers only in the 3.3 protocol compatibility mode which must be configured on the server side as follows:
all supported by server
Tested by us; reported to work by users.
Remote Desktop(Ubuntu feature compatiblewith VNC)
Tested by us. To enable the access:
Apple Remote Desktop (ARD; Mac OS featurecompatible with VNC)
10.4 PPC Mac10.5 and higher (Intel Mac)
Tested by us (legacy platforms were reported to work by users). To make ARD work with T-Plan Robot Enterprise perform these steps on the Mac OS X desktop:
Reported to work by users. Requires T-Plan Robot Enterprise version 3.1.1 or higher to display the desktop in correct colors.
Windows CE and Windows Mobile devices
Tested by us. Older versions such as PocketVNC v1.4.3 contain a bug which breaks the desktop image transfer. There's a workaround:
Windows CE and Windows Mobile devices
Version 4.3 has been reported to work by users. As the server by default uses an unusual 15-bit pixel format, the screen may appear bluish. This can be fixed by making Robot to request one of the standard pixel formats as follows:
Symbian OS mobile devices
Reported to work by users. As most keys on the device keyboard are mapped onto the PC numpad, commands automating typing on the device must use the "location=numpad" parameter. For example, to automate typing of a phone number of 123456789 use "Type 123456789 location=numpad".
iOS (iPod, iPhone)
Tested by us for basic functionality; reported to work by users. Veency requires a jailbroken (rooted) device. There's no alternative because the iOS API does not contain interfaces providing access to the necessary features. You can install Veency from Cydia which gets installed into the device during jailbreaking. Should you experience freezing display image or other connection problems perform the following steps:
The VMLite VNC Server is a commercial software available for a small fee from the Google Play store. It can run both on on rooted and unrooted devices:
For an Android testing alternative see the the Android Over ADB connection.