Object Search ("object")

4.2.3 Object Search ("object")

DESCRIPTION

The Object Search (code "object") locates objects of a particular colour or colours within a certain RGB range. It also allows filtering the object list by an optional template image with detection of eventual object rotation. The method was designed to handle simple image analysis tasks such as "test if the screen contains a particular colour", "find all red objects on the screen" or "find all black triangles or other shapes in any level of rotation on the screen".

As the method always locates the largest possible object and it doesn't search recursively for any other ones it may contain, this approach is not suitable for classic GUI components which are often framed, overlaid and merged together. The method, however, performs very well in the testing of schemes, maps and images with low colour scenes generated for example by Geographic Information System (GIS) applications or systems producing simple image data. The method can be also employed to check whether an object of a certain colour or colour range is present on the screen, for example, to answer a question like "is there any red message in this particular area?".

The algorithm first scans the image for the first pixel which meets the input colour criteria. Then it follows the outer shape of each such an object and stores its bounding rectangle into the context in the form of a set of variables:

The original shapes (java.awt.Shape instances) that describe the objects more accurately are available just to Java scripts through the getContext().getObjectSearchShapes() method.

OPTIONS

The method optionally accepts one template image specified through the hosting command or Java method call.

The passrate parameter plays a role only when the "object" method is called with a template image. The image must contain exactly one shape of the specified colour (colour range) in any level of rotation. The list of objects identified by the input "colour" parameter will be then filtered to leave just shapes which are similar to the template image shape up to the ratio specified by the passrate parameter. If the rotations parameter is also specified and is greater than 1, the object list will be matched against the list of template shapes rotated the specified number of times. If the template image is not specified the method locates all objects matching the specified colour and size parameters regardless of the passrate value.

The standard parameter of "cmparea" is optional and defaults to the full screen when omitted.

Supported specific parameters:

color=<HTMLColor>

The color parameter defines the object color in the HTML-style 6-character RGB hexadecimal format (white is '000000', black is 'ffffff'). If the parameter is not specified it defaults to black.

tolerance=<0-255>

The tolerance parameter is an optional integer number between 0 and 255. It is similar to the RGB Tolerance one in the Image Search method. It indicates how much the Red, Green and Blue components of a pixel may differ at a maximum to consider the colour be equivalent to the object colour specified by the colour parameter. This parameter allows specifying that the object colour may be in a colour range rather than a single colour.  It also helps deal with blurred images where the shapes are not formed by a solid colour but rather a range of similar colours. The higher the tolerance value, the higher the probability of false shape detections is. In most scenarios, the value should be in the [0, 100] range depending on the blurring level. If the parameter is not specified, it defaults to zero and the algorithm looks for solid colour objects only.

bridgecolors=<HTMLColor1;HTMLColor2;...HTMLColorN>

The bridgecolors parameter is optional and it may contain a semicolon-separated list of additional admissible object colours. This allows searching for continuous objects which are expected to have other objects of known colours painted over them. These colours are always compared using the exact match and they do not work with the tolerance parameter.

rotations=<1-360>

The optional rotations parameter makes only sense when a template image is provided. It defines rotation granularity. It basically says how many times to rotate the template object for object filtering purposes. For example, the value of 36 will rotate the shape in 10-degree steps. Obviously, a high number of rotations increases accuracy of recognition of rotated objects but decreases performance. Low rotation values may be compensated through lowering the passrate parameter. This approach is somewhat better in terms of performance but raises the risk of wrong detection of objects similar enough to the template one.

min=w:<width>;h:<height>

Optional minimum size (width and/or height) to filter the resulting objects by. The parameter can specify both the width and height separated by a semicolon (for example "w:100;h:100") or just one of the dimensions ("h:100").

max=w:<width>;h:<height>

Optional maximum size (width and/or height) to filter the resulting objects by. The parameter can specify both the width and height separated by a semicolon (for example "w:100;h:100") or just one of the dimensions ("h:100").

overlapmode=<true|false>

The optional overlap mode flag defines whether the objects on the screen may overlap (conflict). If the parameter value is "true" the method makes an extra effort to look for partially overlapped objects.

draw=<true|false>

The draw parameter is an optional flag which enables highlighting of the located objects on the Robot's desktop view. When the parameter is "true" the algorithm draws the object rectangles over the desktop image in the desktop viewer GUI component. This is useful for debugging and demonstration purposes. These drawings are reset (removed) when the script finishes or another Object Search with the parameter of "clear=true" is executed.

clear=<true|false>

The clear parameter allows to clear up all drawings created by the previously called Object Search methods. See the draw parameter for details.

RETURNS

The method makes the calling command (method call) return 0 (success) if at least one object meeting the specified criteria is found. Otherwise it returns the value of 1.

 TROUBLESHOOTING

To deal with a failing "object" comparison:

• If the script is designed to terminate after a failed search it is a good practice to create a screenshot in a lossless format (PNG or BMP). An example of such a code is shown in the example below. This will allow you to reproduce and debug the failed comparison after you load the screenshot either through the "Static Image Client" in the Login Dialog or using the "Load RD Image" button in the comparison command/method call property window.
• Decrease the pass rate or increase the RGB tolerance parameter ("tolerance") and test it through the "Compare" button until you get a match. If the parameters don't appear to be able to fix the broken search create an alternative template image and add it to the template list or image collection.
• For the most common factors causing comparison failures, see the Image Comparison Recommendations chapter.
• Should the search fail for no obvious reason please mail the screenshot together with the image template(s) to the T-Plan Support.