Different Tools to Inspect Desktop App Elements
Introduction:
Desktop applications can be accessed from the machines on which they are deployed. Generally, single users only use it, or whoever is signed in to the account at the time. Whereas Web applications don’t have any location constraint to be accessed as they are deployed on servers that can be accessed by any user with an internet connection and can be accessed via a URL or browser.
In terms of determining locators to use in test scripts, a different approach is necessary for different applications. With web applications, for instance, the browser’s developer tools can inspect the HTML elements on the page. But for desktop applications, you need to download an application to be able to inspect UI elements on the screen. So in software testing, when it comes to testing desktop applications, one of the key challenges is to identify the locators for different UI elements in an application.
In this blog, we will explore different tools specifically designed to help testers to find the right locators for UI elements in desktop applications and also discuss how to use these tools effectively.
1. Inspect tool
2. WinAppDriver UI Recorder
3. Appium
1. Inspect Tool:
What is an Inspect Tool?
It is a Windows-based tool that can select any UI element and view its accessibility data and also test the navigational structure of the automation elements in the UI Automation tree and the accessible objects in the Microsoft Active Accessibility hierarchy.
Where do you get Inspect tool?
This is shipped together with Visual studio.Inspect.exe. and it’s inside the SDK directory like this: “C:\Program Files (x86)\Windows Kits\10\bin\10.0.16299.0\x64\inspect.exe”
Note: If Windows kits are not available on your machine then use the following location to download Windows SDK https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/
Now let’s see how to inspect desktop application elements using Inspect tool:
1. Double-click on the Inspect.exe file to open the Inspect tool: When you start Inspect, you will see Tree view on the left side it will show all the applications open in your machine, and on the right side you will see Data view which shows all the accessibility information about selected element also you can navigate the UI to view accessibility information about every element in the UI. By default, Inspect tracks the keyboard or mouse focus. As focus changes, the data view updates with the property information of the element with focus as shown in the below screenshot:
2. Second step is to open the desktop application you want to inspect.
3. Move your mouse cursor over the element that you want to inspect: then on the left side of the tool you will see the hierarchical structure of UI elements as a tree-view control and on the right side, you will see all the properties of selected elements like Name, AutomationId, Class Name, Control Type, Native Window Handle and so on…
You can also try the following options from Inspector
- MSAA Mode- Displays Microsoft Active Accessibility property information.
- UI Automation Mode-Displays UI Automation property information.
- Active Hover Toolbar-Activates toolbar buttons on mouse hover, instead of requiring a mouse click.
- Show Highlight Rectangle-Highlights a rectangle around the element with focus.
- Show Information Tooltip-Shows property information in a tooltip.
then you can use these properties in your code as examples shown below:
winAppDriver.findElementByName("Add").click();
winAppDriver.findElementByAccessibilityId("93").click();
winAppDriver.findElementByXPath("//*[@LocalizedControlType='button' and contains(@Name,'Add')]");2. WinAppDriver UI Recorder:
What is a UI recorder?
This is a tool for Windows Application Driver (WinAppDriver) that allows you to record user actions on a Windows application for selected UI elements and view their attribute data to generate XPath queries.
WAD UI Recorder will record the absolute xPath for an application object on mouse hover or click. Then you can make some slight modifications to the recorded absolute xPath for use within your step. You can also use the absolute xPath recorded in WAD UI Recorder to help you construct a relative xPath that may be more readable and manageable within your Cycle steps.
Where do we get a UI recorder?
Download the WinAppDriverUIRecorder.zip from the following link:
Then in this unzip file you will see WinAppDriverUIRecorder.exe to launch the UI recorder
Now let’s see how to inspect desktop application elements using UI Recorder:
1. Run WAD UI Recorder as administrator. The application window will open on your desktop which is divided into two panels, as shown below. The top panel will display the absolute XPath recorded by the application and the bottom panel displays xPath node information.
There are two buttons located at the bottom of the window which are used to Record and Clear the panels. The Record button will change to a Pause button while the recording is in progress.
2. Open the desktop application you want to inspect
3. Click on the Record button to start recording xPath. The WAD UI Recorder will now start recording xPaths of any object or element you interact with on your desktop.
4. Move your mouse cursor or click on the element in your target application to generate the absolute xPath. The xPath appears in the top panel of WAD UI Recorder. Clicking the Pause button will stop the WAD UI Recorder from recording xPath.
Then Xpath is recorded in WAD UI Recorder, as shown below
5. Copy the xPath from the top panel of WAD UIRecorder and paste it into your text editor of choice then will need to be modified in order to use it in your step. You will need to replace the escaped double quote characters ( \” ) with single quotes ( ‘ ) and change the starting point of the xPath and also remove the “Desktop” portion from the absolute xPath. This portion of the xPath is not necessary then we can use that xPath in your step as shown below:
winAppDriver.findElementByXPath("//*[@ClassName='CalcFrame']/Button[@ClassName='Button'][@Name='Add']").click();
winAppDriver.findElementByXPath("//*[@ClassName=\"CalcFrame\"]/Button[@ClassName=\"Button\"][@Name=\"Add\"]").click();3. Appium:
What is Appium?
Appium is an open-source tool used for automating mobile and desktop applications. It supports a variety of programming languages, including Java, PHP, Objective C, Python, and JavaScript.
Appium is a combination of two components:
- Appium Server: To ensure test automation of the applications, we use the server.
- Appium Inspector: We use the feature to inspect and obtain all the specifications of the application’s UI element. For inspecting the element, you have to set the desired capabilities that can inform the Appium server about the type of application and platform you aim to automate.
Where do we get an Appium?
Download the exe file of the Appium desktop setup from the link:
On this Github page, you will see the latest release details as shown below:
Under the Assets section, click and download the Appium-Inspector-windows exe file and save it on your machine.
- Under the Assets section, click and download the Appium-Inspector-windows exe file and save it on your machine.
- Open the downloaded exe folder and double-click on the exe to start the installation process.
- After completing the installation, the Appium GUI window will open as shown below.
Now let’s see how to inspect desktop application elements using Appium:
1. First step before opening the Appium you should launch the winAppDriver by running the nan automation script.
2. Second step is to open the app from which you can inspect the elements
3. Third step is to start the Appium server by clicking on the “Start Server” button it will launch the Appium server on your local machine.
4. Fourth step is to click on Attached Session button, then attach the root path of the application with Appium by clicking Attach Session button present at the bottom
5. Then in App Source you will see all the elements and all opened apps and in Selected Elements, you will see the details of selected elements such as ID, class name, or XPath, that you can use in your test script as shown in the following example.
winAppDriver.findElementByName("Add").click();
winAppDriver.findElementByXPath("/Window[2]/Pane/Button[23]").click();Comparison amongst the above three:
In the Inspect tool, you will immediately get the element attributes, there is no need to start the session every time and you can directly use these attributes in your script without any modifications. Whereas with a UI recorder, you can get an Xpath and with Appium you will get a unique xpath but you need to start the recorder and Appium every time. Along with these tools, you can also explore TestArchitect, Ranorex, Tricentis Tosca, SikuliX, and Askui’s James tools.
Conclusion:
I have used Inspect, UI Recorder, and Appium Inspector tools, the ease of using these tools depends upon the application’s complexity. For me, getting attributes of the UI element Inspect tool will prove better and for Xpath UI recorder will help.
An SDET with hands-on experience in Manual and Automation testing for Desktop and Web applications. Practical experience in Python, Java, Selenium, Playwright, Behave, Pytest, Appium, and Mysql.