Localization testing is usually functional testing of multi-language applications across all supported locales or language packs. It typically involves execution of the same test scenario on all supported language configurations. The goal is to make sure that the application behaves the same way with all supported languages, and that translations of the UI strings are correct and accurate.
Localization testing is a perfect candidate for automated testing, because you design the test suite once, and then run it on many configurations. Automation effectively separates the engineering work from the linguistic one, and you don't have to hire QA people who master both the technical skills and foreign languages. Normally it is sufficient to let the engineers do the automation testing, and then outsource review of the test results, like screen shots and reports to linguists.
T-Plan Robot is a great choice for automation of localization testing. Here are some tips to ensure a successful project:
- Design your test scripts to accept variable values from the command line. See -v/--variable switch in the Automatic Executions document for more information.
- When you create the automated test script, try to avoid using mouse clicks, moves and drags to control the tested application. Translations bring differences into GUI component sizes and behaviour and mouse controls are then not reliable. Stick to the keyboard where possible, and control your application through simple keys, shortcut/hot keys, mnemonic keys, and focus controls like Tab/Shift+Tab.
- Make sure your scripts leave the remote desktop in the same state as before execution. A good idea is to start always on a clean desktop, which is not running any visible application. Calling of Ctrl+F4 several times at the beginning of your script for example, will help to ensure a base state.
- If you decide to save all the output into a single folder, make the language code part of the screenshot, and report file names to prevent them from overwriting.
The following example shows a simple localization testing of Firefox on Ubuntu in several different languages: English (default language), Spanish, French, German, Italian, Russian, Japanese and Simplified Chinese.
The script is very simple and follows the rules stated earlier. We will take advantage of Firefox's CLI option -UILocale, which allows you to start the browser in the language specified by a language identifier (which is in fact identical with the user preferred language sent within the HTTP request header).
Note that you must have the specified Firefox language packs installed to achieve this. The language identifier (e.g. en-us, es-es) is in the script defined by a variable called LANG, and its value can be later on rewritten through the -v/--variable Robot CLI option.
The script further on performs a very basic browser testing - it opens the File menu, starts the Preferences and Help windows and displays the Browse bar at the bottom of the browser window. Three screenshots are taken along the way, and a report is generated for each language.
To execute the script on a local VNC server we need to pass the desired language identifier through CLI. The command will look like this:
To test Firefox in the eight languages discussed above we need to execute the command sequentially eight times, once per each language identifier.
The following table lists reports from automated execution for all suggested languages.