Testing of Flash or HTML5 Animations or Games (OpenGL)
Testing of applications that employ fast moving graphics has long caused issues for many involved with automation, for different reasons.
Not going into depth with Image v Object based automation techniques (that is one for another time!), it is however important to say many object based tools are unable to penetrate these formats (Flash, Flex, HTML5, OpenGL etc.) in order to access the underlying objects and properties. Some image based tools also are not able to confidently automate such formats due to the nature of the fast moving screen.
Now however, the goalposts have been moved as T-Plan Robot is not just able to connect with such environments, but also to capture and automate them directly through the graphical user interface (GUI), exactly as they are intended to be used by the target audience.
Verify the game sequence is correct…. Verify the app menu’s are correct… Verify the gameplay is as expected… Verify your game/app across multiple platforms, browsers, devices! All with T-Plan Robot Enterprise.
The key point in being able to correctly test OpenGL, Flash etc. is first having the ability to analyse the fast moving display. Traditionally tools that rely on the ability to access the screen data have used methods such as “VNC” to connect to the test environment. However, VNC often falls way short in its ability to encode, transfer and decode the data fast enough, and therefore often results in the screen being partially displayed, if at all.
By leveraging the advances in technologies we are now able to offer specific connection methods targeted directly for such testing. Both iOS and Android mobile platforms allow the “screen mirroring” of their screen to a desktop computer. Performance of this mirroring protocol is unrivalled due to its very nature – intended for live streaming of video and audio. Utilising this feature allows Robot to access the device screen real-time, while also applying actions through the desktop mouse and keyboard.
Once connected to the test environment, the process with which you automate the application is not any different to the testing of standard applications. You must verify the presence, or not, of something on the screen followed by the delivery of a user action; i.e. Tap, Swipe, click, etc.
Although all of Robot’s standard commands can be applied across all test environments, it does also offer some custom commands specifically designed for mobile testing. Commands such as “Swipe” and “Pinch/Zoom” for example make use of the standard Click/Drag commands, but employ tailored timings to ensure they are applied correctly in each situation.
For some examples of T-Plan Robot automating mobile gaming applications please see the following:
Video: Automating 'Need for Speed' by EA Games
The following code is an example of how the script may look within T-Plan Robot, using it’s proprietary scripting language. This is also a cross-platform script that is valid for both iOS and Android. The code is based on the project from the above video - Automating "Need for Speed" by EA Games.
##############################
# Define required procedures:
## Identify the target test platform:
Procedure SetEnv {
If ({_PROTOCOL}==ADB || {_PROTOCOL}==RFB) {
Var Env=Android
} else if ({_PROTOCOL}==APPLE) {
Var Env=iOS
}
}
## Perform tap based on an image search:
Procedure Tap {
Waitfor "match" passrate="85" method="search2" template="NFS\{Env}\{1}.png"timeout="15s"
if ({_EXIT_CODE} > 0) {
Step "Click {1}." result=fail
Exit "1"
} else {
Step "Click {1}."
Mouse "click" to="x:{_COMPARETO_CLICK_X},y:{_COMPARETO_CLICK_Y}"
}
}
## Record displayed text:
Procedure CheckText {
Verify Back
Var Area=
If ({Env}==Android) {
Var Area =x:96,y:222,w:983,h:336
} else if ({Env}==iOS) {
Var Area=x:68,y:161,w:606,h:269
}
Compareto method="tocr" cmparea="{Area}"
if ({_EXIT_CODE} > 0) {
Step "Confirm Text." result="fail" actual="No Text Found!"
Exit "1"
} else {
Step "Confirm Text." actual="{_TOCR_TEXT}"
}
}
## Take a screenshot:
Procedure ScreenCap {
Wait 6s
Screenshot {1}_{Env}.png
}
###### End of Procedures ######
##############################
# Start execution code:
// Identify target test environment
SetEnv
// Open App from Home Screen
Tap NFS_App
If ({Env}==Android) {
Press F4 count=3 wait=1s
}
// Adjust controls
Tap Controls
Tap Minus
Tap Plus
Tap Back
// Verify About statement
Tap About
CheckText
// View available races
Tap Map
#####################
## Lets go racing! ##
Tap PetersonSt
Tap OnTheClock
Tap Tick2
ScreenCap RaceCar
Tap Tick
Timer RaceTime Start
Tap SteerRight
Tap SteerLeft
Tap SteerRight
Tap SteerLeft
Tap SteerRight
Tap SteerRight
Tap SteerRight
Tap SteerLeft
Timer RaceTime Stop
// Invoke Pause menu
If ({Env}==Android) {
Mouse "click" to="x:650,y:31"
} else if ({Env}==iOS) {
Mouse "click" to="x:405,y:23"
}
// Exit Race back to Main screen
Tap Exit
Tap Yes
// Exit app
If ({Env}==Android) {
Tap ExitApp
Tap Yes
Press F4
} else if ({Env}==iOS) {
Mouse click btn=right
}
Report "{Env}_TestResults.xml"
#####################
#### End of Test ####