Report

3.3.2 Report

DESCRIPTION

Report - Create and start a report provider for the executed script. It is an object which builds a report from the script execution data such as results, screenshots and warnings. Report providers are customizable plugins and their functionality, life cycle and output format are implementation specific. See the Plugin API for more information on how to customize reports. The default provider started by the command is configurable in the Report preferences and defaults to the 'enterprise'.

The command populates relies on the _REPORT_DIR variable setting to define the target folder to store the report files to. It creates and populates the following variables:

Enterprise Report Provider

This provider supports both HTML, XML and ZIP formats. Its code is "enterprise". It copies exactly behaviour of the default provider (except for the scope parameter) and it even reuses all of its configuration parameters. The difference is however in the way how the test output is processed to generate the resulting report file. Unlike the default provider, this one also supports the Script, Step and Timer test results and it is tightly integrated with the script execution log.

When the output format is set to XML (an .xml file is specified in the command argument), the provider generates a simple XML file with all test outputs. Its format is specified in the provider's Java API documentation. As XML reports are linked with an XML style sheet (XSL), they may be displayed by web browsers and they provide exactly the same view of the report as the HTML files generated by the default provider. As the interpretation of the XML data into an HTML view is on the web browser side, the XML output is fast and efficient. The XML file may be in addition reused for export of test results to third party applications.

The tool provides a mechanism in the user preferences to replace the default XSL file with a custom one. This allows customizing the way how web browser displays the XML data. The provider offers additional configurable options affecting mainly performance and allows to customize the frequency of report generating.

If the output is set to HTML, the provider in fact generates the XML output first and then employs the XSL transformation to produce an HTML file. This operation is inefficient and usually very slow. As the provider, in this case, creates the same HTML content as the default one, it is recommended to use the default one instead unless there are custom changes applied to the XSL file.

Since the 4.1.3 the provider accepts multiple report files (semicolon separated). This may be used to create both the HTML and XML reports. It can also create a ZIP file with the report folder contents.

 Default Report Provider

The default provider is a legacy report generator creating simple HTML reports. It is compatible with the previous VNCRobot 1.x versions and also T-Plan Robot Version 2. Its provider code is "default". It has the following lifecycle:

• • Every execution of the Report command creates and starts a new instance of report provider. It immediately generates the very first HTML report using the predefined script variables and internal list of screenshots and warnings. It also inserts some report specific variables into the execution context.
  • A status daemon is created and started. It wakes up periodically after a predefined period of time (typically 10 seconds) and takes a screenshot of the remote desktop. A link to this image is provided in the HTML report and enables users to view the recent state of the desktop.
  • The provider then registers for change notifications with the internal list of output objects. Whenever a new screenshot is taken or a warning is created, the provider wakes up and rebuilds the report to include the new object.
  • When the execution of the script finishes, the status daemon wakes up and refreshes the status image to reflect the exit state. Then the provider wakes up for the last time and rebuilds the report to reflect the execution result based on the exit code (see the Exit command).

The default report provider always processes all images and warnings taken during script execution. It means that even if you call the Report command at the very end of your script, the HTML report will list all screenshots and warnings including those that were taken before the Report command was executed. The list of images can be currently restricted only depending on the image origin (see the scope parameter below).

HTML reports may also contain a table of results of performed image comparisons. This feature can be configured in the T-Plan Robot Preferences window to display all or just failed comparison results executed through the Screenshot commands.
Note that image comparisons performed through the Compareto and Waitfor match/mismatch commands are never listed. The rationale is that you have to create a screenshot anyway to provide a functional link in the table. See the sample report at the link listed above for an example.

The default provider also inserts specific values at the end of HTML reports. They are embedded in HTML comments and they are not displayed in the browser. They provide information about the report state, length of execution, number of failed image comparisons etc. These values can be parsed by third-party applications in order to interpret the report state and result. A list of provided variables with sample values follows:

SYNOPSIS

report <file(s)> [provider=<provider_name>] [desc=<description>] [scope=<scope_id>] [onexit=<true|false>] 
* Red colour indicates obligatory parameters

OPTIONS

file(s)

- File name or names to save the report to. T-Plan Robot Enterprise will then check whether the path and file can be created and report an error if not. The file extension is validated against the list of supported formats declared by the provider and a syntax error is raised on mismatch.

• • The "enterprise" one supports .xml, .htm, .html and .zip files. The command accepts a single file ("results.xml") or a semicolon-separated list of files ("results.xml;results.html;archive.zip") to create multiple formats in parallel. When the ZIP file is listed it will archive the contents of the report directory (including subfolders). For performance reasons, the archive gets created just once after the script finishes. Support of ZIP and multiple input files was introduced in 4.1.3.
  • The "default" provider supports .htm and .html extensions. It accepts a single file only.

Filename can be either relative (e.g. report.xml) or absolute (e.g. /root/report/report.xml). If the path doesn't exist, it is usually created (default provider) but this behaviour may be provider-specific. If you use a relative path, the report file and all associated outputs should be saved to a directory defined by the _REPORT_DIR variable. 

provider=<provider_name>

- Report provider name (optional). There are two providers, the "enterprise" one (actively developed and selected by default) and the "default" one (legacy/obsolete). You may eventually create a new provider with your own name and plug it into Robot through the plugin framework.

desc=<description>

- Report description. It will be displayed in the report header. If the default provider is used, the text may contain any HTML tags.

scope=<scope_id>

- The scope parameter provides a way to define which images should be included in the report based on the screenshot creator. This parameter is supported just by the Default Report Provider and it gets ignored by the Enterprise one.

There are two acceptable values:

• all - This value is the default one. All images taken during the script execution will be included in the report.
• file - Include only those images that were taken within the current script and in procedures called by this script. Other screenshots created by the master script (i.e. a superscript calling this script) and by the scripts executed using the Run command will be excluded. This approach is suitable when you execute a number of smaller scripts using one superscript and you prefer to have a couple of shorter reports instead of a large one.

onexit=<true|false>

- The value of true will not refresh the report on the fly but create it just twice when the method gets called and then after the script gets finished. This mode is recommended for large scripts creating multiple reportable objects where frequent report refreshing may lead to degraded performance. The default value is false. Supported since v4.1.3. 


RETURNS

The Report command returns 0 (zero) if the report provider is started and the first report is created and saved successfully. If any error occurs, i.e. the report file cannot be created, the command returns 1.

EXAMPLES

Report results.xml desc="This is my XML report."

- Create a report generator and start to generate an XML report. As the relative file is provided, the report will be saved to a directory defined by the value of the _REPORT_DIR variable. The provided description will be displayed in the report header.

Report results.xml;results.html;results.zip desc="This is my XML and HTML report together with a ZIP archive."

- Create a report generator and start to generate the XML and HTML reports. Create a ZIP file of the report folder once finished.

Var _REPORT_DIR=/var/apache/htdocs/robot
Report index.html scope=file
Screenshot start_state.jpg desc="Initial state of the {_MACHINE} desktop. Starting execution of the {_FILE} script..." 
...

- This is a typical example of how to use the report command. The first command defines the output path for the report file and screenshots which actually resides inside the Apache document root. This is very convenient because users can watch the report online through a web browser. As both report and screenshot commands use a relative path for the outputs, everything will be saved to the output directory.