Prequisites

Java Installation

JAVA should be installed on the machine where SIkuli will be executed. The installer can be downloaded from the following link: https://java.com/en/download/

Sikuli Installation

The latest version of SIKULI should be downloaded and installed on the machine where Sikuli will be executed.The installer can be downloaded from the following link: https://launchpad.net/sikuli/sikulix/

The latest version released is Sikuli 1.1.2 and by default it will install the files to the C:\users\AppData\Roaming\Sikulix folder. After installation, please copy the C:\users\AppData\Roaming\Sikulix\SikulixDownloads folder in C:\ProgramData\Sikulix\SikulixDownloads for all users’ accessibility. Final step is to create a system environment variable named SIKULI_HOME with value C:\ProgramData\Sikulix\SikulixDownloads.

NOTE: Flowster Studio installer will create the variable if it doesn’t exist, but make sure that the path is the same as the one mentioned above.

Server Settings

User Account Control should be disabled (to allow Sikuli to start a process as a different user).

Local Security Policy

User whose active session ID will be used for ‘Connect To UIRecorder’ activity must be added to the following security privileges. In order to do this, log in to the remote server with a user with elevated privileges and go to Control Panel-> Administrative Tools -> Local Security Policy. When Local Security Policy opens, go to Local policies -> User Rights Assignment and add the desired user to the following:

Act as part of the operating system
Create a token object
Log on as a batch job
Replace a process level token

Local Users and Groups

User whose active session ID will be used for ‘Connect To UIRecorder’ activity must be added to the local administrators group.

Activities

Click
Connect To UIRecorder
Disconnect From UIRecorder
Double-Click
Exists
Right-Click
Type Text
Wait Vanish

Connect To UIRecorder

This activity creates a UI Recorder session.

Activity Parameters

The Connect to UIRecorder activity parameters can be provided by manual inserted values or via IN arguments or variables:

Parameters:

ClientHash (text box editor type) (type: string): enter the RestAPI client hash code (the RestAPI client can be defined in Flowster Administrator – Integrator Page, for more info click here). This parameter is only needed when the workflow is executed on an Execution Agent. Leave empty if the execution is local (Designer execution).
Password (password editor type) (type: string): insert the user password defined for the RestAPI client.
Password Binding (text box editor type) (type: string): bind to a variable containing the encrypted password for the connection. (This parameter is used only when the user wants to send an encripted password from outside the workflow, for example Flowster Portal).
Session ID (text box editor type) (type: UInt32): specify an active session iD returned by 'Connect To Remote Desktop' activity. If no user session ID is specified, the first active user session ID will be used.
Timeout (text box editor type) (type: UInt32): specify the number of minutes to wait for the operation. The Timeout parameter is optional to be changed, by default providing a 1 minute wait.
Username (text box editor type) (type: string): specify a user which has rights over the RestAPI client (also defined on the RestAPI client from Administrator – Integrator Page – select the client, assign the rights and a password in the Security tab).

The Read Only Output variable are the possible output values that the activity will provide:

Connection (type: object): outputs a Sikuli Connection. This parameter should be used as input for all Connection parameters used for activities placed under UIRecorder category.

Disconnect From UIRecorder

This activity disconnects from the specified UIRecorder session.

Activity Parameters

The Disconnect From UIRecorder activity parameters can be provided by manual inserted values or via IN arguments or variables:

Parameters:

Connection (text box editor type) (type: string): the Connection object - bind to the output of a Connect To UIRecorder activity.

Click

This activity executes a single click on a recorded item.

Requirements:

The pattern should exist on the screen when the action will be made

Parameters:

Connection (text box editor type) (type: Object): bind this parameter to the output from 'Connect To UIrecorder' activity.
Offset X (password editor type) (type: string): specify the pointer's offset value on the X axis.
Offset Y (password editor type) (type: string): specify the pointer's offset value on the Y axis.

NOTE: These parameters, Offset X and Offset Y (the coordinates are pixels and they will be specified by integer values), can be provided if the exact screen coordinates of the item are known, otherwise the fields can be left empty

Pattern File (text box editor type) (type: string): enter the full path to the pattern file of type PNG that will be searched on the screen and if found it will be clicked, This parameter takes precedence over 'Selected Pattern' parameter.
Selected Pattern (custom selection pattern editor type) (type: custom): this field provides the user the opportunity to take a screen capture while developing the workflow from the Designer. By clicking the button, the recorder will be opened and screen capturing will become available.

NOTE: the Designer will be put in background during this operation and it will not be visible on the screen or in the taskbar. After taking the capture, the user will be asked to select the point where the click should be made (in the example below the click will be made on the Log in):

If no point is desired to be specified, click the OK button (by default, for this case, the Recorder will click on the middle of the pattern during the playback).

The Read Only Output variable are the possible output values that the activity will provide:

Output (type: Boolean): outputs True if the operation succeeded or False otherwise.

Double-Click

This activity executes a double click on a recorded item.

Requirements:

The pattern should exist on the screen when the action will be made

Parameters:

Connection (text box editor type) (type: Object): bind this parameter to the output from 'Connect To UIrecorder' activity.
Offset X (password editor type) (type: string): specify the pointer's offset value on the X axis.
Offset Y (password editor type) (type: string): specify the pointer's offset value on the Y axis.

Pattern File (text box editor type) (type: string): enter the full path to the pattern file of type PNG that will be searched on the screen and if found it will be double-clicked, This parameter takes precedence over 'Selected Pattern' parameter.
Selected Pattern (custom selection pattern editor type) (type: custom): this field provides the user the opportunity to take a screen capture while developing the workflow from the Designer. By clicking the button, the recorder will be opened and screen capturing will become available.

If no point is desired to be specified, click the OK button (by default, for this case, the Recorder will double-click on the middle of the pattern during the playback).

The Read Only Output variable are the possible output values that the activity will provide:

Output (type: Boolean): outputs True if the operation succeeded or False otherwise.

Right-Click

This activity executes a right click on a recorded item.

Requirements:

The pattern should exist on the screen when the action will be made

Parameters:

Connection (text box editor type) (type: Object): bind this parameter to the output from 'Connect To UIrecorder' activity.
Offset X (password editor type) (type: string): specify the pointer's offset value on the X axis.
Offset Y (password editor type) (type: string): specify the pointer's offset value on the Y axis.

Pattern File (text box editor type) (type: string): enter the full path to the pattern file of type PNG that will be searched on the screen and if found it will be right-clicked, This parameter takes precedence over 'Selected Pattern' parameter.
Selected Pattern (custom selection pattern editor type) (type: custom): this field provides the user the opportunity to take a screen capture while developing the workflow from the Designer. By clicking the button, the recorder will be opened and screen capturing will become available.

If no point is desired to be specified, click the OK button (by default, for this case, the Recorder will double-click on the middle of the pattern during the playback).

The Read Only Output variable are the possible output values that the activity will provide:

Output (type: Boolean): outputs True if the operation succeeded or False otherwise.

Exists

This activity checks if the pattern exists on the screen.

Requirements:

The pattern should exist on the screen when the action will be made

Parameters:

Connection (text box editor type) (type: Object): bind this parameter to the output from 'Connect To UIrecorder' activity.
Pattern File (text box editor type) (type: string): enter the full path to the pattern file of type PNG that will be searched on the screen.This parameter takes precedence over 'Selected Pattern' parameter.
Timeout(text box editor type) (type: UInt32): specify the number of seconds to wait for the operation.
Selected Pattern (custom selection pattern editor type) (type: custom): this field provides the user the opportunity to take a screen capture while developing the workflow from the Designer. By clicking the button, the recorder will be opened and screen capturing will become available.

If no point is desired to be specified, click the OK button.

The Read Only Output variable are the possible output values that the activity will provide:

Output (type: Boolean): outputs True if the operation succeeded or False otherwise.

Type Text

This activity sends characters to the application.

Parameters:

Connection (text box editor type) (type: Object): bind this parameter to the output from 'Connect To UIrecorder' activity.
Text (text box editor type) (type: string): specify the text to be typed in the application.

NOTE: double-escape special characters (e.g. for '\\n' type '\\\n' or @'\\n' ).

IMPORTANT: Sikuli java api does not support non english keyboard layout.

The Read Only Output variable are the possible output values that the activity will provide:

Output (type: Boolean): outputs True if the operation succeeded or False otherwise.

Wait Vanish

This activity waits for the pattern to disappear from the screen.

Requirements:

The pattern should exist on the screen when the action will be made

Parameters:

Connection (text box editor type) (type: Object): bind this parameter to the output from 'Connect To UIrecorder' activity.
Pattern File (text box editor type) (type: string): enter the full path to the pattern file of type PNG that will be searched on the screen.This parameter takes precedence over 'Selected Pattern' parameter.
Timeout (text box editor type) (type: UInt32): specify the number of minutes to wait for the operation.
Selected Pattern (custom selection pattern editor type) (type: custom): this field provides the user the opportunity to take a screen capture while developing the workflow from the Designer. By clicking the button, the recorder will be opened and screen capturing will become available.

The Read Only Output variable are the possible output values that the activity will provide:

Output (type: Boolean): outputs True if the operation succeeded or False otherwise.

Browser not supported