public class GoogleVisionOCR extends Object implements ImageComparisonModule, Plugin, ImageObserver, ExtendedParamsObject, Configurable, ExtendedSearchCapabilities, com.tplan.robot.gui.components.ComparisonModuleGuiSupport, AdvancedImageComparisonModule, TextRecognitionGUISupport, Stoppable, Measurable, LocalizedPlugin
T-Plan Robot Enterprise, (C) 2009-2022 T-Plan Limited. All rights reserved.
OCRConstants.OCR_ENGINES
CONTEXT_OBJECT_IMAGE_COMPARISON_FAILURE_DESC, SEARCH_COORD_LIST
ABORT, ALLBITS, ERROR, FRAMEBITS, HEIGHT, PROPERTIES, SOMEBITS, WIDTH
PROPERTY_NAME_OBJECT_CHANGED
CFG_DEBUG_MODE, CFG_DOTALL_MODE, CFG_LEAVE_FILES, CFG_OCR_ENGINE, CONTEXT_TOCR_ERROR, CONTEXT_TOCR_TEXT, DEFAULT_FONT_SIZE, PARAM_DISTANCE, PARAM_FILTER, PARAM_FONTSIZE, PARAM_LANGUAGE, PARAM_MODE, PARAM_PATTERN, PARAM_SCALE, PARAM_TEXT, VAR_TOCR_ERROR, VAR_TOCR_LINE, VAR_TOCR_LINE_COUNT, VAR_TOCR_LINE_H, VAR_TOCR_LINE_W, VAR_TOCR_LINE_X, VAR_TOCR_LINE_Y, VAR_TOCR_MATCH, VAR_TOCR_MATCH_CLICK_X, VAR_TOCR_MATCH_CLICK_Y, VAR_TOCR_MATCH_COUNT, VAR_TOCR_MATCH_H, VAR_TOCR_MATCH_INDEX, VAR_TOCR_MATCH_W, VAR_TOCR_MATCH_X, VAR_TOCR_MATCH_Y, VAR_TOCR_TEXT, VAR_TOCR_TEXT_H, VAR_TOCR_TEXT_W, VAR_TOCR_TEXT_X, VAR_TOCR_TEXT_Y
Constructor and Description |
---|
GoogleVisionOCR() |
Modifier and Type | Method and Description |
---|---|
void |
checkDependencies(PluginManager manager)
Check whether the current product installation contains all dependencies
(other plugins) required to install this plugin.
|
float |
compare(Image desktopImage,
Rectangle area,
Image image,
String methodParams,
ScriptingContext repository,
float passRate)
Compare the desktop image to another image and return a number indicating how much they match.
|
float |
compareToBaseImage(Image desktopImage,
Rectangle area,
String methodParams,
ScriptingContext context,
float passRate)
Compare the desktop image to the base image and return a number indicating how much they match.
|
Map<Rectangle,int[]> |
getAllFailedPixels() |
String |
getCode()
Get plugin code.
|
Map<String,Object> |
getComparisonParams() |
Date |
getDate()
Get plugin release date.
|
float |
getDefaultPassRate()
Get the default pass rate value.
|
String |
getDescription()
Get plugin description to be displayed in the GUI.
|
String |
getDisplayName()
Get short plugin name to be displayed in the GUI.
|
String |
getDisplayNameKey()
Get the key to the plugin name in the resource bundle.
|
List<Point> |
getFailedPixels() |
String |
getHitDescription(Rectangle r,
StringBuilder label)
Get description of a particular match area.
|
List<Rectangle> |
getHits() |
Class |
getImplementedInterface()
Get Class of the exposed functional interface that this plugin implements.
|
LinkedHashMap<Rectangle,String> |
getLineRects() |
String |
getLongDescription()
Get long description of the object - purpose, usage, parameter syntax etc.
|
int[] |
getLowestSupportedVersion()
Get the lowest required version of T-Plan Robot Enterprise.
|
List<TextSearchResult> |
getMatches() |
String |
getMessageAfterInstall()
Get text of a message to be displayed after installation of this plugin.
|
String |
getMessageBeforeInstall()
Get text of a message to be displayed before installation of this plugin.
|
String |
getMethodDescription()
Get the method description.
|
String |
getMethodName()
Get the method name.
|
List<String> |
getParameters()
Get the list of parameter names supported by the object implementing this
interface.
|
List |
getParameterValues(String parameterName)
Get the list of acceptable values of a particular parameter (if known).
|
String |
getPassRateName()
Get alternative name for the default "pass rate".
|
List<Preference> |
getPreferences()
Get metadata of displayable/editable configurable parameters.
|
Component |
getPreviewComponent(Window owner,
File selectedFile,
StringBuilder title)
Get the preview component.
|
float |
getProgress()
Get the task progress where 0 means "0% done" and 1 means "100% done".
|
Rectangle |
getRectangle() |
String |
getResultMessage(boolean useHtml,
StringBuilder title,
StringBuilder buttonLabel)
Get the message describing results of the last performed image
comparison.
|
String |
getShortDescription()
Get short description which describes in a few words purpose and role of
the object.
|
String |
getSupportContact()
Get support contact.
|
Rectangle |
getText(StringBuilder out) |
String |
getUniqueId()
Get unique ID associated with the plugin.
|
String |
getVendorHomePage()
Get the vendor home page.
|
String |
getVendorName()
Get vendor (provider) name to be displayed in the GUI.
|
int[] |
getVersion()
Get plugin version in form of an integer array.
|
List<Preference> |
getVisualParameters()
Get the list of supported parameters encapsulated in
Preference
instances. |
boolean |
imageUpdate(Image img,
int infoflags,
int x,
int y,
int width,
int height)
Dummy method implementing the ImageObserver interface.
|
boolean |
isMethodParamsSupported()
Define whether this module supports some custom parameters which can be passed via the
methodparams
parameter of the scripting language. |
boolean |
isSecondImageRequired()
Determine whether we are comparing two images or just analyze the desktop image.
|
boolean |
isStopped()
Find out whether the task has been already stopped.
|
boolean |
isTrackingOfFailedPixelsSupported() |
boolean |
requiresRestart()
Indicate whether installation of this plugin requires application restart.
|
void |
setBaseImage(Image img)
Set the base image.
|
void |
setConfiguration(UserConfiguration cfg)
If an object implementing this interface is a plugin (i.e.
|
void |
setParameters(Map<String,String> paramsAndValues)
Set parameters and their values in the object.
|
void |
setTrackingOfFailedPixelsEnabled(boolean enabled) |
void |
stop()
Call this method to request the implementing object to stop the current
task or activity.
|
String |
toString() |
boolean |
usesPassRate()
Indicate whether the module uses the pass rate as a parameter or not.
|
equals, getClass, hashCode, notify, notifyAll, wait, wait, wait
compare, getMethodName, isMethodParamsSupported, isSecondImageRequired, setBaseImage
imageUpdate
getAllFailedPixels, getHits, getRectangle, isTrackingOfFailedPixelsSupported, setTrackingOfFailedPixelsEnabled
getHitDescription, getPreviewComponent, getResultMessage
getDefaultPassRate, getPassRateName, usesPassRate
getComparisonParams, getLineRects, getMatches, getText
getProgress
checkDependencies, getCode, getDate, getImplementedInterface, getLowestSupportedVersion, getMessageAfterInstall, getMessageBeforeInstall, getSupportContact, getVendorHomePage, getVendorName, getVersion, requiresRestart
public float compareToBaseImage(Image desktopImage, Rectangle area, String methodParams, ScriptingContext context, float passRate)
ImageComparisonModule
This is just a convenience method used by the 'Waitfor match'
command to perform repeated image
comparisons against one image.
The default image comparison algorithm is based on color histograms. A call of this method performs image comparison against precalculated base image histogram rather that against the base image pixels. As calculation of a color histograms is quite a time expensive operation, this approach allows to achieve higher performance.
Any custom implementation of this interface may or may not take advantage of this approach. If you don't want
to diferentiate between repeated and one time comparison calls, define the setBaseImage()
and
compareToBaseImage()
methods as follows:
// Base image private Image baseImage; // Set the base image public void setBaseImage(Image img) { baseImage = img; } // Compare an image with the base image public float compareToBaseImage(Image desktopImage) { return compare(desktopImage, baseImage); }
compareToBaseImage
in interface ImageComparisonModule
desktopImage
- an image to be compared to the base image.area
- a rectangle area of the remote desktop to be used for image comparison. If the argument is null,
the whole remote desktop image will be used.methodParams
- method parameters passed by the 'methodparams'
parameter of
the 'CompareTo'
or 'Waitfor update'
command.context
- a Map with execution context. Note that most of the objects contain there are not public
and the parameter is specified here to allow compatibility among the custom and internal comparsion methods.passRate
- pass rate in form of a float value between 0 and 1. This user input value indicates
the lowest result which will be considered as match ("passed"). It is in fact the number specified by the
'passrate' parameter of the commands Screenshot,
Compareto and
Waitfor match/mismatch.public String getMethodDescription()
ImageComparisonModule
getMethodDescription
in interface ImageComparisonModule
public String getDisplayName()
Plugin
getDisplayName
in interface Plugin
public String getDisplayNameKey()
LocalizedPlugin
getDisplayNameKey
in interface LocalizedPlugin
public String getDescription()
Plugin
getDescription
in interface Plugin
public String getUniqueId()
Plugin
Get unique ID associated with the plugin. The plugin manager uses the ID together with the version string to identify whether a plugin is already installed and whether a newer version of the same plugin is available.
The unique ID in fact identifies a particular plugin delivered by a particular vendor. Plugin developers are recommended to choose an ID and keep it constant for all versions of one particular plugin. The ID is never displayed in the GUI so it doesn't have to be a readable text. To avoid conflicts with other vendors it is recommended to elaborate vendor or author name and feature description into the ID, for example "custom RFB client implemented by John Doe".
getUniqueId
in interface Plugin
public List<String> getParameters()
ExtendedParamsObject
getParameters
in interface ExtendedParamsObject
public List getParameterValues(String parameterName)
ExtendedParamsObject
getParameterValues
in interface ExtendedParamsObject
parameterName
- parameter namepublic void setParameters(Map<String,String> paramsAndValues)
ExtendedParamsObject
CaseTolerantHashMap
class);setParameters
in interface ExtendedParamsObject
paramsAndValues
- map of parameters and their values.public List<Preference> getVisualParameters()
ExtendedParamsObject
Preference
instances. It allows to specify a larger set of parameter properties such as
parameter name ("configuration key" in the Preferences object), label (short description),
description, expected value type and optional value limits and relationships with other
parameters. This allows the GUI to build and display the parameters in form of GUI components
in command property dialogs and validate the values entered by users..getVisualParameters
in interface ExtendedParamsObject
Preference
instances.public void setConfiguration(UserConfiguration cfg)
Configurable
If an object implementing this interface is a plugin (i.e. implements also the Plugin interface) and is instantiated through a supported plugin factory, the Plugin Manager calls this method right after an instance of this object is created.
Custom objects which do not already have their configuration parameters in
the default configuration file should take advantage of this method
to store their configuration into the shared User Configuration instance.
It is recommended to call the UserConfiguration.saveConfiguration()
method in the end to save the configuration to the hard drive.
Objects wishing to be notified of changes of configuration parameters
should implement the ConfigurationChangeListener interface and register
with the UserConfiguration instance through the
addConfigurationChangeListener()
method.
setConfiguration
in interface Configurable
cfg
- global shared instance of user configuration preloaded with
parameters from the default and user configuration files.public List<Preference> getPreferences()
Configurable
getPreferences
in interface Configurable
public String getShortDescription()
ExtendedParamsObject
getShortDescription
in interface ExtendedParamsObject
public String getLongDescription()
ExtendedParamsObject
CustomHyperlinkListener
. This is optional
and the method may return null. The long description is typically
displayed by the GUI when user selects to view details of the object.getLongDescription
in interface ExtendedParamsObject
public void setBaseImage(Image img)
ImageComparisonModule
This is just a convenience method used by the 'Waitfor match'
command to perform repeated image
comparisons against one image.
The default image comparison algorithm is based on color histograms. When you call this method,
a histogram is calculated for the base image and cached for future repeated comparisons performed through the
compareToBaseImage()
method calls. As calculation of a color histograms is quite a time expensive
operation, this approach allows to achieve higher performance.
See the compareToBaseImage()
method doc for more info.
setBaseImage
in interface ImageComparisonModule
img
- the base (component) image.public float compare(Image desktopImage, Rectangle area, Image image, String methodParams, ScriptingContext repository, float passRate)
ImageComparisonModule
compare
in interface ImageComparisonModule
desktopImage
- desktop image.area
- a rectangle area of the remote desktop to be used for image comparison. If the argument is null,
the whole remote desktop image will be used.image
- another image to be compared to the desktop image.methodParams
- method parameters passed by the 'methodparams'
parameter of
the 'CompareTo'
and 'Waitfor update'
commands. You can use this to pass custom
parameters from the script to this module.repository
- a Map with execution context. Note that most of the objects contain there are not public
and the parameter is specified here to allow compatibility among the custom and internal comparsion methods.passRate
- pass rate in form of a float value between 0 and 1. This user input value indicates
the lowest result which will be considered as match ("passed"). It is in fact the number specified by the
'passrate' parameter of the
commands Screenshot,
Compareto and
Waitfor match/mismatch.public Rectangle getText(StringBuilder out)
getText
in interface TextRecognitionGUISupport
public String getHitDescription(Rectangle r, StringBuilder label)
com.tplan.robot.gui.components.ComparisonModuleGuiSupport
Get description of a particular match area. This is used by the
Search Hits
dialog which displays the text in a
text field whenever the match is selected in the drop down. This is
useful for example for OCR modules which wish to display the recognized
text along with its rectangle.
Modules which want to skip this functionality and made the description components invisible should make the method return null when the rectangle argument is null.
getHitDescription
in interface com.tplan.robot.gui.components.ComparisonModuleGuiSupport
r
- selected rectangle. The argument is null if no rectangle is
selected. As this typically corresponds to the view of all matches the
method should return a general text and/or label.label
- optional label to display the text with.public Map<Rectangle,int[]> getAllFailedPixels()
getAllFailedPixels
in interface ExtendedSearchCapabilities
public boolean usesPassRate()
AdvancedImageComparisonModule
usesPassRate
in interface AdvancedImageComparisonModule
public String getPassRateName()
AdvancedImageComparisonModule
getPassRateName
in interface AdvancedImageComparisonModule
public float getDefaultPassRate()
AdvancedImageComparisonModule
AdvancedImageComparisonModule.usesPassRate()
returns true.getDefaultPassRate
in interface AdvancedImageComparisonModule
public LinkedHashMap<Rectangle,String> getLineRects()
getLineRects
in interface TextRecognitionGUISupport
public Map<String,Object> getComparisonParams()
getComparisonParams
in interface TextRecognitionGUISupport
public List<TextSearchResult> getMatches()
getMatches
in interface TextRecognitionGUISupport
public void stop()
Stoppable
public boolean isStopped()
Stoppable
public float getProgress()
Measurable
getProgress
in interface Measurable
public boolean isSecondImageRequired()
ImageComparisonModule
If this method returns true, commands CompareTo
and WaitFor update
of the scripting language will require presence of the 'template'
parameter which
specifies an image located in the filesystem to be compared to the current desktop.
If you wish just to analyze the desktop image instead of comparing it to another one, implement this method
to return false. The tool will not validate whether an image from the file system is
passed correctly. Note that commands will supply null instead of an image in all arguments
corresponding to the template image, i.e. expect method calls like compare(desktopImage, null)
and
setBaseImage(null)
.
isSecondImageRequired
in interface ImageComparisonModule
public boolean isMethodParamsSupported()
ImageComparisonModule
methodparams
parameter of the scripting language.
When this method returns false, the tool will report a syntax error when this module gets used in a command
together with the methodparams
parameter.
When this method returns true, commands using this module may take advantage of the methodparams
parameter to pass custom parameters which are not defined by the scripting language.
The default image comparison algorithm doesn't provide any custom parameters.
isMethodParamsSupported
in interface ImageComparisonModule
public String getCode()
Plugin
Get plugin code. This string serves as an identifier of the functionality delivered by the plugin. For example, desktop client plugins return protocol name like "RFB" or "RDP" as their code.
Plugin code is used by pluggable instance factories to identify
a particular plugin. It may be used to replace internal plugins with
third party code. For
example if you develop a class which implements this and the
com.tplan.robot.remoteclient.RemoteDesktop
interfaces and the
getCode()
method returns "RFB", it will replace the internal
implementation of the RFB (VNC) client.
public String getMethodName()
ImageComparisonModule
module
of the 'CompareTo'
and 'Waitfor update'
commands) and in default comparison method configuration (user config parameter
CompareToCommand.defaultComparisonModule
).
Method names are not case sensitive. Be sure not to specify a name which is reserved by
the tool or already used by another method. Version 1.3 reserves two names, 'default'
and search
. You can't override these names.
getMethodName
in interface ImageComparisonModule
public String getVendorName()
Plugin
getVendorName
in interface Plugin
public String getSupportContact()
Plugin
getSupportContact
in interface Plugin
public int[] getVersion()
Plugin
new int[] { 1, 2, 3 }
.getVersion
in interface Plugin
public Class getImplementedInterface()
Plugin
com.tplan.robot.remoteclient.RemoteDesktopClient.class
.getImplementedInterface
in interface Plugin
public boolean requiresRestart()
Plugin
requiresRestart
in interface Plugin
public String getVendorHomePage()
Plugin
getVendorHomePage
in interface Plugin
public int[] getLowestSupportedVersion()
Plugin
getLowestSupportedVersion
in interface Plugin
public String getMessageBeforeInstall()
Plugin
getMessageBeforeInstall
in interface Plugin
public String getMessageAfterInstall()
Plugin
getMessageAfterInstall
in interface Plugin
public void checkDependencies(PluginManager manager) throws DependencyMissingException
Plugin
DependencyMissingException
if one or more dependencies are missing.checkDependencies
in interface Plugin
manager
- shared instance of the plugin manager.DependencyMissingException
- when one or more dependencies requested by this plugin is missing.public boolean imageUpdate(Image img, int infoflags, int x, int y, int width, int height)
imageUpdate
in interface ImageObserver
public void setTrackingOfFailedPixelsEnabled(boolean enabled)
setTrackingOfFailedPixelsEnabled
in interface ExtendedSearchCapabilities
public boolean isTrackingOfFailedPixelsSupported()
isTrackingOfFailedPixelsSupported
in interface ExtendedSearchCapabilities
public Rectangle getRectangle()
getRectangle
in interface ExtendedSearchCapabilities
public List<Rectangle> getHits()
getHits
in interface ExtendedSearchCapabilities
public Component getPreviewComponent(Window owner, File selectedFile, StringBuilder title)
com.tplan.robot.gui.components.ComparisonModuleGuiSupport
getPreviewComponent
in interface com.tplan.robot.gui.components.ComparisonModuleGuiSupport
owner
- the owning window.selectedFile
- the selected template image file.title
- optional output buffer for the preview title.public String getResultMessage(boolean useHtml, StringBuilder title, StringBuilder buttonLabel)
com.tplan.robot.gui.components.ComparisonModuleGuiSupport
getResultMessage
in interface com.tplan.robot.gui.components.ComparisonModuleGuiSupport
useHtml
- true should return the message in HTML, false in plain
text.title
- a buffer for the optional custom message title.buttonLabel
- label (name) for the button in the intended pop up.
The message mentions it.