Localization

Localization testing is usually functional testing of multi-language applications across all supported locales or language packs. It typically involves execution of the same test scenario on all supported language configurations. The goal is to make sure that the application behaves the same way with all supported languages, and that translations of the UI strings are correct and accurate.

Localization testing is a perfect candidate for automated testing, because you design the test suite once, and then run it on many configurations. Automation effectively separates the engineering work from the linguistic one, and you don't have to hire QA people who master both the technical skills and foreign languages. Normally it is sufficient to let the engineers do the automation testing, and then outsource review of the test results, like screen shots and reports to linguists.

T-Plan Robot is a great choice for automation of localization testing. Here are some tips to ensure a successful project:

Design your test scripts to accept variable values from the command line. See -v/--variable switch in the Automatic Executions document for more information.
When you create the automated test script, try to avoid using mouse clicks, moves and drags to control the tested application. Translations bring differences into GUI component sizes and behaviour and mouse controls are then not reliable. Stick to the keyboard where possible, and control your application through simple keys, shortcut/hot keys, mnemonic keys, and focus controls like Tab/Shift+Tab.
Make sure your scripts leave the remote desktop in the same state as before execution. A good idea is to start always on a clean desktop, which is not running any visible application. Calling of Ctrl+F4 several times at the beginning of your script for example, will help to ensure a base state.
If you decide to save all the output into a single folder, make the language code part of the screenshot, and report file names to prevent them from overwriting.

The following example shows a simple localization testing of Firefox on Ubuntu in several different languages: English (default language), Spanish, French, German, Italian, Russian, Japanese and Simplified Chinese.

The script is very simple and follows the rules stated earlier. We will take advantage of Firefox's CLI option -UILocale, which allows you to start the browser in the language specified by a language identifier (which is in fact identical with the user preferred language sent within the HTTP request header).

Note that you must have the specified Firefox language packs installed to achieve this. The language identifier (e.g. en-us, es-es) is in the script defined by a variable called LANG, and its value can be later on rewritten through the -v/--variable Robot CLI option.

The script further on performs a very basic browser testing - it opens the File menu, starts the Preferences and Help windows and displays the Browse bar at the bottom of the browser window. Three screenshots are taken along the way, and a report is generated for each language.

Var LANG=en-us

# This procedure will open Firefox in the locale specified by argument.
# It will then open the browser menu, Preferences and Help windows and Search bar.
# Screenshots will be taken at major points.
# Parameter: {1} - desired browser locale, e.g. en-us, es-es, de-de
procedure test_firefox {
# Close all windows and open the Run Application window on Gnome Linux
Press Alt+F4 count=2 wait=1s
Press Alt+F2 wait=4s

# Start Firefox in desired language
Typeline "firefox -UILocale {1}"

# Wait for major update of the remote screen
Waitfor update extent="50%" timeout="30s" ontimeout="Exit 1"

# Open the Firefox menu - keys F10 and "arrow down"
Press F10 wait=200
Press Down wait=1s
Screenshot firefox-menu-{1}.jpg desc="Firefox File menu. It should be properly localized into {1}."

# Open the Firefox Preferences window. # As the File menu is open, key "arrow right" will display the Edit menu # and key "arrow up" will jump to the last menu item Preferences
Press Right wait=500
Press Up wait=500
Press Enter wait=5s
Screenshot firefox-preferences-{1}.jpg desc="Firefox Preferences window. It should be properly localized into {1}."

# Escape will dispose the Preferences window
Press Esc wait=500

# Display the Firefox Search bar (F3 key) and open the Help window (F1).
Press F3 wait=1s
Press F1 wait=5s
Screenshot firefox-help-{1}.jpg desc="Firefox Help window and the Search bar (at the bottom). It should be properly localized into {1}."

# Close the browser so that we can rerun the script safely
Press Alt+F4 count=2 wait=1s
}

# Start an HTML report
Report index-{LANG}.html desc="Example of Robot automated localization testing of Firefox."

# Call the test procedure. # Value of the argument variable can be passed from CLI.
test_firefox {LANG}

To execute the script on a local VNC server we need to pass the desired language identifier through CLI. The command will look like this:

robot.sh -c localhost:1 -p <password> -r example3.txt -n -v LANG=<language_identifier>

To test Firefox in the eight languages discussed above we need to execute the command sequentially eight times, once per each language identifier.

The following table lists reports from automated execution for all suggested languages.