Desktop Driver
Desktop Driver
This page describes only Desktop Driver specific features and methods which may be different in other drivers.
Implementation which is common for all drivers is described in Drivers page. Please, read it before.
Despite the ever-growing popularity of web-based applications in the workplace, many business processes still involve desktop applications for reasons of legacy, security, or hardware needs. Being able to control these types of applications programmatically opens up a world of automation possibilities.
DesktopDriver is used to automate desktop applications. Desktop automation allows your robot to accomplish tasks acting like a human operator, directly controlling a desktop interface. This includes operations like opening and closing applications, simulating mouse movements and clicks, triggering keyboard keys and shortcuts.
Compared to browser automation, desktop automation is a more varied and complex field. However, the main idea stays the same across operating systems and access methods: you need a way to point the robot to specific parts of the desktop screen, and once a target is identified, the robot can be instructed to interact with it by clicking on it, typing on it, dragging it, etcetera.
This driver is based on the Mmarquee UIAutomation project which a Java-based wrapper of Microsoft UIAutomation Library. This framework is for automating rich client applications based on Win32 (including Delphi), WPF and other Windows applications (including Java SWT). It uses the JNA library to make calls to the COM-based WIndows automation library.
Driver Initialization
You can initialize the driver in your automation process by the following way:
@Driver private DesktopDriver desktopDriver;
Driver Params
| Param name | Measure | Default Value | Description |
|---|---|---|---|
| DriverParams.All.EXPLICITLY_WAIT_TIMEOUT_SECONDS | seconds | 5 | Wait functions default timeout in seconds. |
| DriverParams.All.EXPLICITLY_WAIT_POLLING_INTERVAL_MILLISECONDS | milliseconds | 500 | Wait functions default pooling interval in milliseconds. |
| DriverParams.All.LAUNCH_APPLICATION_TIMEOUT | ms | 5000 | Parameter is responsible for timeout which driver should wait after it called the command to start application and until application is started. |
@Driver(param = {@DriverParameter(key = DriverParams.Desktop.LAUNCH_APPLICATION_TIMEOUT, direct = "50000")})
private DesktopDriver desktopDriver;Opening new applications
Desktop Driver supports 2 methods to start new application: using path or path with arguments.
public void get(String path) public void get(String... command)
Bellow is an example of how to start application using path:
driver.get("C:\\Windows\\system32\\calc.exe");
//or
driver.get("notepad.exe");Window management
Desktop Driver supports switching to window by various combination using window class name, title or title regexp, window handle. For that it has the following methods allowing you to switch to window:
public void switchToWindow(String titleOrHandle) public void switchToWindow(Pattern titleRegexp) public void switchToWindow(String className, String titleOrHandle) public void switchToWindow(String className, Pattern titlePattern)
Also it supports methods which help you to wait until window appears and switch to it only after that, with the various combination of input parameters. It returns "true" if it switched successfully:
/** * Switches to the desktop 'window' associated with the class name and title regexp (or window handle). * * @param className Class name to search for. * @param titleRegexp Regexp for title to search for or window handle. * @param titleOrHandle title to search for or window handle. * @param secondsToPoll Seconds to wait for windows to be appeared. * @param suppressTimeoutException Set true to don't let method throw an exception, so it returns "false" value instead. * * @return True if driver switched successfully * */ public boolean waitAndSwitchToWindow(String titleOrHandle, int... timeouts) public boolean waitAndSwitchToWindow(Pattern titleRegexp, int... timeouts) public boolean waitAndSwitchToWindow(String titleOrHandle, boolean optional, int... timeouts) public boolean waitAndSwitchToWindow(Pattern titleRegexp, boolean optional, int... timeouts) public boolean waitAndSwitchToWindow(String className, String titleOrHandle, int... timeouts) public boolean waitAndSwitchToWindow(String className, Pattern titleRegexp, int... timeouts) public boolean waitAndSwitchToWindow(String className, String titleOrHandle, boolean optional, int... timeouts) public boolean waitAndSwitchToWindow(String className, Pattern titlePatternOrHandle, boolean optional, int... timeouts)
Example:
driver.switchToWindow("Untitled - Notepad");Working with UI Elements
Inspector
One way to automate Windows applications is to target UI components with their identifiers. Microsoft recommends Accessibility Insights for viewing the UI automation properties. Legacy tools such as Inspect.exe can also be used. This tool allows to get information about interface elements such as Name, ID, Type, etc.
After installing and launching Accessibility Insights for Windows, inspecting Windows applications is straight-forward. Using the Windows Calculator as an example, hovering over the application displays the properties of the UI components.
By default, Accessibility Insights displays only a few properties, including the accessible Name of the UI component in the DETAILS pane. In this case, the name of the button is Five. Using localized names for automation is not the most robust option since the labels change based on Windows language settings.
To see more properties, click on the settings icon and select Include all properties that have values:
This will include the AutomationId property. In this case, the value of that property is num5Button:
Find elements
Before you switch to any windows using driver, the desktop (root) window is considered as the active one. As the desktop window is the root window for all elements and for all other windows on the screen - searching will be perform slower if you don't switch to any specific window.
So when you call "findElement(s)" method it's strongly recommended to switch to target window before, to don't have the root window as the active one.
Please, check the table which locators are supported by DesktopDriver:
| Selector type |
|---|
| DesktopSearch.query |
| DesktopSearch.name |
| DesktopSearch.className |
| DesktopSearch.namePattern |
| DesktopSearch.automationId |
Desktop driver supports only "DesktopSearch.desktopSearch" element locator. This locator accept special UIQuery object which you can build providing necessary element attributes.
Bellow an example of how to find element by name, class name and control type (all information can be extracted by Inspector):
DesktopElement desktopElement = driver.findElement(DesktopSearch.query(DesktopSearch.UIQuery.builder()
.name("Text Editor")
.className("Edit")
.controlType(ControlType.Edit.getValue())
.build()
));