Image Based Text Recognition
2. Character Image Collections
3. Character Capture Wizard
Image Based Text Recognition (IBTR)
T-Plan Robot Enterprise 3.0 allowing to recognize
and its coordinates on the
screen based on a collection of presaved characters.
The IBTR functionality consists of three components:
This feature is not parameter compatible with the Image Based Text Recognition plugin provided as an add-on for
2.x releases. To convert the plugin based script code replace the tolerance
parameter with the passrate one where
- The "text" comparison method which is described in the Scripting
- Character image collections,
- Character Capture wizard.
passrate = 100 * (1 -
An approximate table of the sample tolerance values converted to
2. Character Image Collections
A Character Image Collection
is a directory with character images which meets these conventions:
An example of a character collection containing two variants of the
character and one 'o' character may look like:
- The collection directory contains
folders named 'uNNNN' (a "character
folder") where 'NNNN' is the four digit upper case
UTF-8 character code. For example, the 'M' character must be
represented by a folder called 'u004D' because its UTF-8
- Each character folder contains one or
more character images in a Java supported lossless format (PNG
preferred, BMP is also supported but tends to be large). The
names are not relevant but the file extensions must be correct
regard to the image format. For example, a PNG image must be
a file with the ".png" extension.
Though Robot supports creation and maintenance of character image
collections through the Character Capture Wizard,
be created and edited by hand or in third
party image editors provided that the conventions are met.
may be useful to exploit some features of the image
algorithm used internally by the IBTR. For example, to
create a character collection which would work on any background one
may edit the character images in an image editor and make the
background transparent or translucent.
- As the internal algorithm derives the
height of the text line and the space size from the collection
do not mix images of characters of different
size into a single collection. Mixing of
character images of different font and background colors
is OK as
long as the characters are of the same font type and size.
3.2 Character Capture Wizard
The Character Capture Wizard
is the front end GUI window allowing to create, view and maintain
character image collections. The wizard can be started through the Tools->Character
Capture menu item.
The very first wizard screen titled Select Image Collection
deals with selection ot the character image directory to work with:
Collection button is enabled only if the Collection
contains an existing directory. It opens a new window with the
collection details consisting of a tree of character images and a
char set table showing which characters are covered by the
The viewer also allows basic maintenance tasks such as editing or
deletion of the character images.
To add new character images to the selected collection click Next to
to the Select
screen. Note that this button is enabled only when Robot is
to a desktop. If you need to extract characters from a static image
stored to a file, use the Login
Window to load the image though the Static Image Client.
To extract new characters from the screen perform the following
Once you have selected the image and provided the text hit Next to
to the last screen called Confirm Character Images:
- Click the Select Text Area
button. It will open a new window with a copy of the remote
Then drag your mouse in the image to mark the area containing
the recommendations chapter
text selection tips. Once you are satisfied click either the
button next to the selection or the Save & Close
the tool bar.
- Type the text the image contains into
the editor situated above the button. Make sure to provide it
as it is displayed with all spaces and lines.
This screen shows the list of characters contained in the provided
text. If Robot recognizes the individual characters on the pixel
in the image the list gets populated with the suggested character
images. Characters may be removed or modified the the Delete and Edit
aware that you don't have to deal with duplicate images. If a
image already exists and it is the same as the newly extracted one,
will be skipped automatically.
In some scenarios Robot may fail to parse the text area image for
individual characters. This may happen for example when the selected
text area does not meet the recommendations
criteria. Unrecognized characters then appear in the list with
red N/A icon
This behavior does not mean that the character can not be searched
using the character collections. It merely indicates that the wizard
was not able to suggest the character image. To fix this simply
the character in the list, click Edit and define
the its image in the text area manually.
These tips suggest how to select the text area on the screen in
to minimize the number of unrecognized characters (N/A)
have to be fixed manually:
- The very first pixel of the selected
area (the top left corner) must be of the background color.
- The area should contain text of a
font type, size and color displayed on a solid color background
for example a block of black text on a white background. If the
does not meet these requirements process it in smaller parts
this criteria (by words and/or lines).
- The area should contain one
block of text displayed on a single graphical component, such as
message, a button label, a menu item etc.. Do not select for
area which contains two or more buttons because their texts may
aligned differently along the horizontal axis.
- As Robot separates the characters by
looking for a space of background color between them, fonts
characters overlap along the vertical or horizontal axis fail to
and must be extracted manually. If such a character image
parts of another character, these must be removed in a third
image editor such as Gimp and the affected must be made
make the character search algorithm skip them.