RDP + Calculator
RDP + Calculator
Introduction
The purpose of sample is to perform simple actions and OCR on windows calculator opened inside remote desktop session.
The technologies employed are screen based automation, OCR and desktop automation.
Prerequisites
Running this sample on RPA cluster
This AP only works when Remote Desktop is visible on screen.
That means If AP is started on the remote machine, then it's desktop must be connected to and visible. Otherwise, the bot will be unable to perform screen recognition on invisible RDP.
There is one possible workaround - remote EasyRPA node must be running in console session which allows screen recognition inside it.
It can be started by launching following batch script under Administrator on target node. This will cause remote desktop window to close immediately and the current session to proceed in console mode.
for /f "skip=1 tokens=3" %%s in ('query user %USERNAME%') do (
%windir%\System32\tscon.exe %%s /dest:console
)Target node screen must be configured with 1920x1080 resolution
Make sure the size of text, apps and other items is 100% (96 dpi)
RDP host (winrdp.ip param) must be configured as trusted on target node
to run the Automation process (AP), the node should have Capabilities specified in the AP Details tab. Capabilities for RDP + Calculator: AP_RUN, SCREEN, DESKTOP
repo id: eu.ibagroup.samples.ap:easy-rpa-rdp-ap:jar:full:<easyRPA version>
module class: eu.ibagroup.easyrpa.ap.rdp.RdpAp
The package of the automation process can be found in the following directory: https://<CS host>/nexus/repository/rpaplatform/eu/ibagroup/samples/ap/easy-rpa-rdp-ap/<version>/easy-rpa-rdp-ap-<version>-bin.zip
The souce code: https://code.easyrpa.eu/easyrpa/easy-rpa-samples/-/tree/dev/easy-rpa-rdp/easy-rpa-rdp-ap
configuration params:
| key | default value | required | description |
|---|---|---|---|
| winrdp.ip | 127.0.0.11 | yes | target RDP host |
| app.rdp.path | C:WindowsSystem32mstsc.exe | no | |
| rdpLogin | false | no | if false -runs ap without rdp session |
secret vault:
| alias | value |
|---|---|
| rdp.secrets | { "user": "rdp_demo", "password": "rdp_demo" } |
AP is implemented in eu.ibagroup.easyrpa.ap.rdp.RdpAp class and consists of a single task PerformCalcMultiUser
In standalone mode the AP can be started from main method of the same eu.ibagroup.easyrpa.ap.rdp.RdpAp class
Included Steps
First the bot starts RDP session by launching mstsc windows command. When doing so, it passes username and password as arguments so RDP screen opens immediately without prompting for login.
Next opens the remote desktop where bot clicks on Start menu and types in "calculator" to open the target application.
In order to bring the standardized view of calculator, the bot maximizes it window and then switches to scientific mode from upper left menu.
After series of clicks we get calculation result displayed.
Next the bot highlights the region approximately corresponding to displayed result and tries to OCR it.
The OCR result can be found in AP logs prefixed with "output="
Finally the bot presses Alt+F4 to close RDP window and AP finishes