By program level execution we understand running of automated process(es) from a custom Java program or a third party Java application. As T-Plan Robot's role is in this scenario reduced to a Java library, this mode is supported on the level of Java APIs only and there are no GUI or CLI features supporting development of the related Java code. Use a third party tool (an IDE) such as NetBeans or Eclipse for this purpose.
T-Plan Robot Enterprise supports program level execution through the AutomatedRunnable and ApplicationSupport Java interfaces. The API allows to create and start either a single process (effectively simulating a single automatic execution in the GUI or CLI mode) or an unlimited number of processes (executed as Java threads) running in CLI mode (until the system resources are exhausted). Each such a process seamlessly accepts the same set of options as the CLI and executes one test script against one test environment (desktop) at a time.
Program level execution consists of three steps:
- Instantiate the ApplicationSupport class. This initialises the necessary framework and shared services.
- Use the instance to create an AutomatedRunnable. Test script name (file) as well as other options compatible with the CLI specification are accepted through one of the
- Either call the
run()method of the runnable (single process scenario) or encapsulate it in a java.lang.
Threadand start it as a thread (multi-threaded scenario).
The following example starts two automated threads. The first one connects to a VNC server running on port 5900 of machine called testmachine1 and executes the C:\TestScripts\test1.tpr test script . The other one connects to a VNC server running on port 5901 of machine called testmachine2 and executes the C:\TestScripts\test2.tpr test script. Note that both the threads will be executed simultaneously and the program will exit when the last thread finishes.
This multi-threaded approach can be also employed in load testing scenarios. T-Plan Robot can measure the application response time, and it is an excellent engine for the simulation of a number of users accessing the tested application or service in order to generate load. When this model is combined with another tool for performance measuring, it may represent a very cost effective solution.
In addition to supporting the automation of the local desktop, T-Plan Robot utilises the RFB (VNC) technology, and in this mode each thread must automate its own VNC desktop server instance. Such an approach is very efficient with Unix or Linux where a single platform (OS) may run as many VNC servers, as the system resources allow (such as RAM and CPU). This scenario may be exploited for testing of applications running on Unix/Linux and/or for web application testing where the Unix/Linux machine hosts web browser instances. As MS Windows systems may run just one VNC server per OS, load testing on these environments is not so efficient. In these instances it would be better to employ T-Plan Robot ability to automate the local desktop.
T-Plan Robot does provide tools to measure application response time, and produce performance results. As automation is performed on the desktop level, such testing will have an extra added value because it will reflect the true time elapsed between the user request and result. This is in contrast with many other load performance testing tools which typically focus on measuring of one critical phase of the request-response cycle (for example database or web/application server response time). Such results however usually do not correspond with the end user experience, where the time is further on affected by the environment.
Robot can be successfully used to generate load for stress testing. The idea is to start many parallel testing processes to simulate a number of users accessing a service or using an application. As Robot simulates a real user behaviour in a graphical environment, such a scenario will create load on the tested system which is very similar to real production environment.
Robot supports has a Java API which allows to create multiple testing threads within a single Java Virtual Machine. Each thread may independently execute one test script on a single VNC server instance. See the RobotThread interface of theRobot API for more information. Note that Unix/Linux systems are preferred to host the load tests because unlike Windows they are capable of running many VNC server instances on a single machine.
Let's create an example program generating load on a tested application. We need to create three components:
- Test script in the Robot scripting language which will represent a typical user session with the tested application.
- Java wrapper which will take advantage of the Robot Open API to create and start parallel test threads.
- Shell wrapper which will start and eventually shut down the VNC servers (Linux/Unix only).
1. Test Script
There is no test script included in this example. You need to create it before and integrate it with the Java and shell wrappers described below. See the previous examples in this document on how to create a test script in the Robot scripting language. You may easily use the Record & Replay feature to do so. It is highly recommended to use a small remote desktop size (e.g. 800x600 pixels) and avoid image comparison and taking of screenshots. All these factors significantly increase the amount of required memory and CPU time. Another good idea is to set your remote desktop background (wallpaper) to a solid colour. In the next steps we will suppose that the test script already exists and is saved in a file called client.txt.
2. Java Wrapper
Java wrapper is a program which creates and starts the automated testing threads using the Robot API. A universal wrapper code is displayed below:
To run the wrapper compile the Java code to .class. The program accepts up to three parameters - desired number of threads to be started (mandatory), path to the testing script (mandatory) and VNC password (optional). The command should look like:
3. Shell Wrapper
Before you start Robot load testing, you always need to start all the necessary VNC servers. You should also verify whether any application involved in the testing allows only one instance per user. A typical example is Firefox which will not allow a single user to run more than one browser. To resolve this you will need to create a different user for each test thread and start the VNC server under his account.
All these necessary overhead tasks can be efficiently automated on Linux/Unix by simple shell scripts. Let's suppose that our example application doesn't have any instance limitations. All we need to do then is to start the VNC servers, execute the Java load application and kill the VNC servers after the testing gets finished. A Bash script handling these tasks follows:
Before you execute it, set correct values of the THREADCOUNT, SCRIPT and PASSWORD variables. If you define the number of threads too high, some of them may start to fail with java.lang.OutOfMemoryError.
To fix this issue increase the JVM memory heap size specified by the -Xmx Java option in runLoad.sh (default value is set to 512MB).