Image-based automation tools analyze images of the application or the whole computer desktop. They get the images usually from the underlying operating system, for example from the desktop image buffer or through attaching to the graphic driver. Advantage of this approach is that the underlying technology is irrelevant and the tool can automate any application that displays on the desktop of the operating system. Such automation is also simple, close to the end-user experience and easy to learn even without programming experience. It can also reveal layout problems in testing projects such as overlapping or ill-fitting GUI components.

T-Plan Robot Enterprise is an image-based automation tool. Due to the nature of the way it performs the automation of the system, the tool itself in the industry is termed a Black Box Automation Tool. In essence, Robot provides image comparison methods, allowing to search for GUI or text components on the desktop, and verify the status of the tested application, by comparing against collected image templates or applying actions to these found components to drive workflow in RPA projects. 

Image-based automation is also termed within the industry as "Black Box" given its nature of applying the user input and verifying or applying the graphical (application) output. Black Box automation does not get involved with what occurs in between these two events in terms of how they are processed by the applications, what code branches were executed, object properties, etc. – this would be otherwise known as "White Box" automation.

As T-Plan Robot interacts in this way, it connects to the SUA (system under automation) via the input (mouse/keyboard) event queue and the display image buffer. This is achieved primarily through the use of VNC (Virtual Network Computing) for connections to secondary machines or using the direct Java and OS API's for the local operating system in a local desktop setup.

By comparing the displayed screen against a set of predefined image templates we can determine that the system is fit for purpose in the realms of UAT (User Acceptance Testing), or drive user actions for Robotic Process Automation (RPA). The key point here is that T-Plan Robot does not "see" the code of your application, which is never seen by the user, but instead what is delivered to the user via the screen.