In the realm of automation testing, the conventional practice of identifying locators such as XPath, CSS, and ID is widely employed. However, there are scenarios where substantial time is expended in locating elements within diverse components, such as popup windows and Microsoft Foundation Class (MFC) windows. Additionally, there are cases where element location proves to be impossible. These challenges often impede progress and create bottlenecks. Hence, here in this blog, my aim is to propose a solution for addressing these issues and optimizing time allocation.
What if there was a way to bypass the traditional locator-finding technique and still identify and interact with elements?
Well, it is indeed possible using Sikuli. Sikuli offers an alternative approach to automation by leveraging visual patterns, allowing users to interact with elements on the screen without relying on traditional locator-based techniques.
Let’s understand What is Sikuli:
Sikuli is an open-source and powerful test automation tool that excels when there is limited access to a GUI’s internal or source code. Instead of relying on XPath, CSS, or ID, Sikuli employs image recognition and GUI component control to identify objects displayed on the screen. It is operate as a separate tool to employees’ image recognition mechanism with some action perform on the element.
Sikuli is a versatile tool that integrates seamlessly with popular programming languages like Python and Java. It is compatible with various operating systems, including Windows, Mac, and Linux as well as integrating with Selenium and Pycharm. By adopting this approach, we significantly reduce the time required for element location, simplifying the automation process.
Pre-requisite For Sikuli:
To get started using Sikuli, we need to install the following things.
Download and Installed any IDE as per your preference. Here we are using Intellij Idea
Install Other required dependencies like selenium, web driver, etc
Create a folder to store screenshots in a project.
To take a screenshot, you can use a built-in snippet tool available on your system. Alternatively, you can install tools like Inspector, PowerShell, or AutoIT, which provide x and y coordinates. For more information on these tools, you can refer to this blog: https://spurqlabs.com/different-tools-to-inspect-desktop-app-elements/
Using x and y coordinates, we take a screenshot during execution and store it in a specific path. We have written the code below:
Create one Java class.
Build your project.
Architecture of Sikuli:
Sikuli is a framework that assists in automating various elements on web pages.
The framework utilizes an image recognition mechanism to identify elements on a webpage.
Image recognition is achieved by comparing the elements on the webpage with provided images.
If a provided image is not found on the webpage, Sikuli raises an exception.
In specific scenarios, it is advisable to select an appropriate image that precisely highlights a single element on the webpage.
Selecting a precise image helps to ensure greater accuracy in element identification.
The Sikuli framework offers different methods to execute actions on web pages.
These methods provide versatility and flexibility in achieving automation objectives.
Screen Class:
The Sikuli framework has an inbuilt Screen class, a predefined method for performing actions on web elements using images. To access methods of the Screen class, we need to declare a reference to this class and initialize it.
Screen screen = new Screen();
In the above code, the variable “screen” is declared as an instance of the Screen class, and the new keyword is used to create a new object of the Screen class.
Here are some of the methods available in the Screen class that can be used efficiently:
Click on Element- Image:
To perform a left click on an element, provide an image to locate/identify the element to be clicked.
Ex: screen.click(“image path”);
Right-click on the element:
This method is used to perform a right-click on an element by providing an image to locate/identify the element to be clicked.
Ex: s.rightClick(“Image Path”);
Double-click on the element:
We use this method to perform a double-click action on an element. It first locates the element on the screen and then performs a double left click on the element.
Ex. s.doubbleClick(“Image Path”);
Type on Element :
In the Sikuli framework, you use the Type method to send Keys by providing an image path and sending text as a method argument.
Ex: screen.type(“Image path”, ”Send Key”);
Find() :
We can use this method to check the element’s visibility on a webpage.
EX. screen.find(“Image Path”);
DragDrop :
Users use this method to perform the action as drag and drop. We provide a source image and target image to the drag-drop method argument.
In the world of automation testing, we use an Integrated Development Environment (IDE) to write code. Nowadays, it has become common to create Maven projects to facilitate collaboration with various add-ons. As a widely used automation tool, Selenium supports integration with many add-ons. To integrate Sikuli into the Selenium framework, we need to add the required dependencies in the pom.xml file of our project.
To find the Sikuli dependencies, we can search the Maven repository at https://mvnrepository.com/artifact/org.sikuli. From this repository, we can copy the necessary dependencies and paste them into the pom.xml file of our project.
By adding the Sikuli dependencies to the pom.xml file, we ensure that the required libraries and resources are properly imported and utilized within our Selenium-Sikuli integration. This allows us to leverage the capabilities of Sikuli for image recognition and interaction within our Selenium automation framework.
We are creating sikuli funcion
1. Create a maven project, create a class with the main method where a set a browser and launch a browser:
public static void main (String [] argos){
WebDriverManager.chromedriver().setup();
ChromeDriver driver = new ChromeDriver();
driver.get(“https://demoqa.com/”);
driver.manage().window().maximize();}
2. Take a screenshot and store it in a specific location:
We are well aware that the Snipping Tool is a reliable tool for capturing screenshots. By utilizing this tool, we can capture customized screenshots and save them within the project. folder.
From the above image, we are cropping a single element image and saving it in the project screenshot folder.
How to take screenshots by using x,y coordinates:
There is an alternative method to capture screenshots without relying on external tools.
We can utilize the Robot class and its methods to capture screenshots based on x and y coordinates.
To capture a rectangular screenshot, we need two sets of x and y coordinates.
The first set represents the top-left corner of the rectangle, and the second set represents the bottom-right corner.
By specifying these coordinates, we can define the area of the screen to capture.
An example code snippet captures a screenshot based on the specified coordinates.
Our framework saves the captured screenshot to a specific location.
String fileName1 = "";
try {
Robot robot = new Robot();
String imgeFormat = ".png";
StringBuilder str = new StringBuilder("imageFolderPath" +
System.currentTimeMillis() + image format);
fileName1 = str.toString();
Rectangle captureRect = new Rectangle(xStart, yStart, xEnd - xStart, yEnd - yStart);
BufferedImage screenFullImage = robot.createScreenCapture(captureRect);
format = "png";
System.out.println(" Path is " + fileName1);
ImageIO.write(screenFullImage, format, new File(fileName1));
System.out.println("A partial screenshot saved!");
} catch (AWTException | IOException ex) {
System.err.println(ex);
}
Explanation of the above code:
Declares a variable fileName1 of type String and initializes it as an empty string.
The try-catch block handles potential exceptions that may occur during execution.
Creates a new instance of the Robot class, which allows for programmatic control of the mouse and keyboard
Declare an image format variable that assigns value as ‘.png’
Constructs a StringBuilder object to create the file path for the screenshot. It concatenates the image folder path, the current system time in milliseconds, and the image format.
Converts the StringBuilder object to a String and assigns it to the ‘fileName1’ variable.
Defines a Rectangle object that represents the area of the screen to be captured. It takes the starting coordinates (xStart, yStart) and the width and height calculated from (xEnd – xStart) and ( yEnd – yStart)
Uses the ‘createScreenCapture(captureRect)’ method of the Robot class to capture the screen within the specified Rectangle area. It returns a BufferedImage object representing the captured image.
Writes the captured image to the file specified by fileName1 using the write() method from the ImageIO class.
3. Click on Element by using the previous taking screenshot:
As we mentioned above sikuli methods, by using this method we will do multiple actions performed on elements.
Screen s = new Screen();
s.find(fileName1);
s.click(fileName1);
Limitations:
Manage a number of screenshots:
Managing a large number of screenshots can be a complex and time-consuming process. Locating a specific screenshot among many can become challenging. To simplify this process, a recommended solution is to establish a specific naming convention for the screenshots.
Two similar images are available on the webpage:
If there is more than one image available on the webpage, Sikuli cannot accurately categorize and distinguish a specific image. If it’s not a recognized image then it throws an exception.
Conclusion:
To overcome the challenges of locating elements in automation testing, especially within popup windows and MFC windows, we have successfully implemented Sikuli as a solution. So by adopting Sikuli, we can eliminate the need for traditional locators, leading to enhanced execution time and improved efficiency in our automation efforts. Sikuli’s visual recognition capabilities can help users swiftly identify and interact with GUI elements, enabling faster automation execution. Overall, Sikuli proves to be a valuable alternative in scenarios where traditional locators are insufficient or inaccessible.
An SDET with hands-on experience in the life science domain, including manual testing, functional testing, Jira, defect reporting, web application, and desktop
application testing. I also have extensive experience in web and desktop automation using Selenium, WebDriver, WinAppDriver, Playwright, Cypress, Java, JavaScript, Cucumber, maven, POM, Xray, and building frameworks.
Let’s explore the topic of image comparison testing, which is often overlooked by testers who typically focus more on validating texts, buttons, forms, fields, and other similar elements. However, in some instances, businesses may require testing of images such as logos, infographics, and other graphics.
I encountered such a scenario in my project, where I had to compare two images from different windows. This project was related to life sciences, specifically comparing images of two different animals. During this process, I discovered two methods for image comparison testing through automation using Selenium WebDriver and Java. Today, we will look into these two aspects of image comparison testing.
I don’t think any industries are left that are not involved in the minor or major level of image testing. At least, organizations will ask testers to perform testing of the logo placed on their website and it is very common to ask. Still, I have listed examples of some of the industries below where Image comparison testing can be extensively used.
Prerequisite:
Need to download API dependency, BufferedImage library (Optional – Pointlib and Sikuli )
Applications of Image Comparison in Various Domains:
E-commerce: Ensuring accurate representation of products and their images is crucial for online retailers.
Advertising and Marketing: Verification of visual advertisements, banners, and promotional materials is essential for maintaining brand consistency.
Gaming: Testing game graphics, character designs, and visual effects are vital for delivering an immersive gaming experience.
Healthcare and Medical Imaging: Evaluating medical images for accuracy and precision is critical for diagnostic purposes.
Automotive: Comparing images of vehicle designs, safety features, and user interfaces helps ensure optimal user experiences.
Image comparison cannot be directly performed using Selenium WebDriver alone. However, when there is a specific need for image comparison, we can rely on third-party APIs to achieve this functionality. One such API is AShot, which allows us to compare two images. In this blog, I will explain how to compare two images using the AShot API.
Sometimes during automation, we need to compare two images for verification. We can compare two images using Java selenium with the help of Ashot API as the web driver does not support image comparison.
Step-by-Step Implementation Guide:
We need to capture the screenshot of a particular element and store it in the framework of the project during execution.
Then compare the captured image with the expected image which was already stored in the project.
To capture the image during automation Selenium has to take a screenshot interface which allows us to take the screenshot of the whole screen. The code snippet to take screenshots is explained below
But when there is a need to take a screenshot of a particular element or image then there are various methods available in Selenium like BufferedImage, PointLib library, and external libraries like Sikuli.
While doing image comparison some things need to be kept in mind such as both images should have the same dimensions and if images are grid images then both images must be grid images otherwise they won’t be recognized as the same image.
Java ImageIO class:
This is a final class that belongs to the javax.imageio package. This class provides a convenient method for reading and writing images and performing simple encoding and decoding. The class provides a lot of utility methods related to image processing. Using this class, we can deal with popular image extensions like .jpg, .gif, .png, etc.
BufferedImage class:
This is a subclass of the Image class. It handles and manipulates the image data. A ColorModel of image data makes up a Buffered Image. The bufferedImage class consists of so many things like getHeight, getWidth,getRGB, etc.
In the first Image comparison method, we take both images’ RGB values with the Color class’s help. The Color class is a part of the Java Abstract Window Toolkit (AWT) package. The Color class creates color by using the given RGBA values and has different methods which return the component in the range of 0-255.
Comparison Scenarios: Considered Image Scenarios:
Positive scenario:
Where both the images are the same. For this scenario both the methods mentioned below will give the result as – images are the same.
Negative scenario:
Where one small change is made in the images. one small black dot is added in the image as seen below. For this scenario, the first method will give the result as images are not the same and the second method will give the result as images are the same. So we can confirm that the first method is more accurate to check every small change in the image.
Method 1:
This method of comparing the RGB value is more reliable than the second one. The second method only reflects the difference when there is a significant change in the images.
Here is the link provided for the code of both methods.
So here, we use the BufferedImage class to read the images and save them as BufferedImage objects named img1 and img2, respectively.
By using the getWidth()and getHeight() methods we are reading the height and width of both the images as the height and width of both the images should be the same otherwise it will not proceed further to find out the difference. And if it is not the same, the program will print “Both images should have the same dimensions” as we compare them in an if loop.
If the images are the same then it will execute 2 for loop for height and width. We are reading the RGB value of both the images by method getRGB. And we are storing that value in integers defined as pixels and pixel integer values are passing values into the Color class.
The color class has a method to read the red, green, and blue values of the images and then find the sum of the differences in RGB values of the two images.
From the differences we find out the average and the percentage and if the percentage is equal to 0 then the images are the same and if the rate is more than 0 then the photos are different.
Method 2:
Let’s Understand Method 2 in detail
Again, we are using the BufferedImage class to read the images, and we are saving them as BufferedImage objects named img1 and img2, respectively.
Now we are creating the object of the ImageDiffer class and the ImageDiffer class has a built-in method makeDiff which compares two images. If there is a difference, it returns the ImageDiff class object.
Then we are using the hasDiff built-in method to check the value of the diff objects and confirm whether the images are different.
This way we can compare two images whether they are the same or not using Ashot. Here the intention of showing a negative scenario is to make sure that the scenarios are working fine even if there is a small change in the expected image. Method 1 is more accurate as it is comparing the RGB values of the images and when there is a small change in the image also it will give an accurate result.
Conclusion:
Image comparison using the AShot API provides a reliable and efficient method to verify and validate images during automation testing. By using the capabilities of Ashot and implementing the image comparison process, we can enhance the quality and reliability of our automation tests, ensuring that the visual aspects of our applications meet the desired expectations.
An SDET with hands-on experience in Functional testing on Web and Desktop applications, Requirement analysis, and Defect reporting. Practical experience in Cucumber, Selenium, Java, TestNG, Playwright, Appium, X-Ray, Jira and SQL.
In this blog, we will explore how to build a robust mobile test automation framework using Appium in Python (behave framework). As a result, it will be very useful for executing the program.
Mobile test automation can be more challenging than web automation, as inspecting and interacting with mobile elements requires additional effort. However, with the help of Appium, an open-source tool, it is possible to overcome these challenges and build a powerful mobile test automation framework. In this blog, we will explore how to create a robust framework using Appium in conjunction with the Behave framework in Python.
Let’s talk about robust test frameworks
Robust test automation framework ranks highly on the list of Software Testing “must-haves”.
It helps improve the overall quality and reliability of software when executed in a structured manner.
If we don’t build the right framework then the results will be: Inconsistent test results, Non-modularized tests, and Maintenance difficulties. The automation framework needs to be well organized to make it easier to understand. An organized framework provides an easier way to maintain and expand.
There are many features that we should consider to make the automation framework more robust.
Scalability – The automation framework that you have in your organization should be scalable. It should not just apply to one project. Your automation framework should be applied throughout projects across the organization. It should be an organization-wide test automation framework.
Re-portability – Every automation framework should have a good reporting capability. The test framework engineer can choose a third-party reporting library.
Configurable – A framework should be configurable. It should execute scripts in different test environments. The automation framework should not be restricted to a single test environment. The user credentials should not be “hard-coded” in the automation script itself.
Re-usability – The framework should follow re-usability. We should use the same methods, and page objects in all the test scenarios in the test automation framework.
Extendability – You should be able to integrate easily with other third-party tools via APIs. Automation frameworks should be easily integrated with security testing tools, web proxy debugging tools, test case management tools, or with other global frameworks thereby making it more hybrid in nature.
Reduce the product release cycle time – Improve the time to market, Reduce QA cycle time
Let’s start with basic
Appium is an open source Test Automation Framework which is used for automating mobile applications.
Appium supports Android, IOS mobile apps, and Windows PC Desktop apps. We can automate Native, Hybrid, and Mobile web apps using Appium.
Uses of Appium:
Appium is open source and it is free of cost.
Appium supports Android, IOS, and Windows to run test scripts.
Appium supports languages such as Python, Java, Perl, PHP, C#, and Ruby
Appium supports different operating systems such as Mac, Windows, Linux, UNIX, etc.
Functional test cases of mobile applications can be easily automated.
Appium Inspector
Appium Inspector is a tool for QA engineers who want to automate mobile applications. Basically, this tool also serves as the standard procedure for identifying mobile application elements.
The following are the used for inspecting the mobile element for both Android and iOS.
After downloading the exe file launch the Appium inspector.exe file. On top of the web page, select to Cloud-based platform – BrowserStack
BrowserStack is a cloud-based real devices platform that provides support for both manual and automated testing of mobile apps for both Android and iOS devices. One of its standout features is the App Live feature, which allows users to manually test their mobile apps on over 3000 real Android and iOS devices.
BrowserStack supports testing across different environments, including Dev, QA, Staging, and Production apps from the play store or app store. This makes it easy for developers to test their apps in various environments and ensure that they are working correctly in each environment.
To proceed further we need BrowserStack Username and BrowserStack Access Key
Log in to your BrowserStack account ->Navigate to the “Account” section ->Then Go to Summary
After going to Summery Section you will get Username and Password. Copy both Username and Password and paste them into Appium inspector fields
Go to Desired Capabilities -> Here we need to add basic capabilities which are required for starting the session. Below image will guide you
To add capabilities need to click on the “+” symbol as shown in the below image
Add the capabilities with desired values as shown in the below image
In the Value field, we need to add data that we want to add.
The most important thing here is for the last field “Appium: app”, we have to upload the .ipa or .ipk or .aab file on BrowserStack.
For that, there are 2 ways mentioned below
Through Command Line
Directly through BrowserStack
The most important thing here is for the last field “Appium: app”, we have to upload the .ipa or .ipk or .aab file on BrowserStack.
Let’s start
Through Command Line
To upload the .ipa or .ipk or .aab file on BrowserStack, the following Curl command is very useful.
Curl is a command line tool that enables data exchange between a device and a server through a terminal. Using this command line interface (CLI), a user specifies a server URL (the location where they want to send a request) and the data they want to send to that server URL.
Go to cmd and write this curl command
curl -u “username:accesskey” -X POST “https://api-cloud.browserstack.com/app-automate/upload” -F “file=@path of the file where you save your apk or IPA file” -F “custom_id=any name “
Directly through BrowserStack:
Go to “App Live” -> Click on Uploaded Apps -> And Upload your file.
But here 1st one is preferable: Through this command, we get “app_url” which is required in Desired Capabilities:
Copy that app_url and paste it into Desired Capabilities.
Here you need to Click on Start Session -> You will get below the window.
You can select the element in the App or from the App Source section and the attributes including ID, Name, Text, etc will be displayed on the right side under the Selected Element section and you can create Xpaths using those attributes.
We can also install all the required packages using the requirement.txt file using the below command.
pip install -r requirement.txt
Framework Structure Overview
Here is the overview of our mobile test automation framework using Appium in Python.
You have to follow the below 7 steps to build a robust mobile test automation framework using Appium.
Step 1:
Create a project in Pycharm (here I am using Pycharm professional) and as mentioned in the prerequisites install the packages
Step 2:
In this step, we will be creating a Features folder in which we will be creating our feature files for different scenarios. Every step in a Feature File describes the action we are going to perform on UI. A feature file is something that holds your test cases in the form of a scenario and scenario outline. In this framework, we are using a scenario. Both scenario and scenario outlines contain steps that are easy to understand for non-technical persons. We are giving tags for the feature files. We can also give it for the scenarios present in that file. Depending on our test cases. Note that the feature file should end with a .feature extension.
@iOS
#@android
Feature: Simple Calculator
Addition of two numbers
Scenario: Verify addition of two numbers
Given I am on calculator home page
When I enter '4'
And I enter operator of addition
And I enter operator of addition this should be like And I enter
‘+’ operator so if possible you can update code as per this
And Enter number '2'
And I enter operator '='
Then I see result as '6'
Step 3:
After creating the feature file now create a step file. Both feature files and step files are essential parts of the BDD framework. The steps with ‘When’ are related to the user actions like navigation, clicking on the button, and filling in the information in input boxes. The steps with ‘Then’ are related to the verifications or Assertions. In this, we are using both iOS and Android, so the step file should look like this. We are creating only one-step files for both iOS and Android.
Purpose of Step file: The step file is to attach steps from the feature file to the page file, where actual implementation is available.
from behave import *
use_step_matcher("parse")
@given("I am on calculator home page")
def step_impl(context):
print("User is on Homepage")
@when("I enter '{number}'")
def step_impl(context, number):
str = context.config.userdata["deviceType"]
print("str " + str)
if str == "['iOS']":
context.iOS_cal.iOS_tap_number1(number)
else:
context.android_cal.tap_number1()
@step("I enter operator of addition")
def step_impl(context):
str = context.config.userdata["deviceType"]
print("str " + str)
if str == "['iOS']":
context.iOS_cal.iOS_tap_operator()
else:
context.android_cal.tap_operator()
@step("Enter number '{number}'")
def step_impl(context, number):
str = context.config.userdata["deviceType"]
print("str " + str)
if str == "['iOS']":
context.iOS_cal.iOS_tap_number1(number)
else:
context.android_cal.tap_number2()
@step("I enter operator '{operator}'")
def step_impl(context, operator):
str = context.config.userdata["deviceType"]
print("str " + str)
if str == "['iOS']":
context.iOS_cal.iOS_equals(operator)
else:
context.android_cal.equals()
@then("I see result as '{result}'")
def step_impl(context, result):
str = context.config.userdata["deviceType"]
print("str " + str)
if str == "['iOS']":
flag = context.iOS_cal.iOS_verify_result()
assert flag == True
else:
flag = context.android_cal.verify_result()
assert flag == True
Step 4:
In this step, we are creating two-pagefiles one for iOS and one for Android that contains all the locators and the action methods to perform the particular action on the web element. We are going to add all the locators at the class level only and will be using them in the respective methods.
iOS page file:
import time
from appium.webdriver.common.mobileby import MobileBy
from selenium.common import NoSuchElementException
from selenium.webdriver.support import expected_conditions as EC
from time import sleep
from Features.Pages.BasePage import Basepage
class iOS_Calculator_Page(Basepage):
def __init__(self, context):
Basepage.__init__(self, context.driver)
self.context = context
self.add_operator = "//XCUIElementTypeStaticText[@name='+']"
self.result = "(//XCUIElementTypeStaticText)[1]"
def iOS_tap_number1(self,number):
time.sleep(2)
tap_on = self.wait.until(
EC.presence_of_element_located((MobileBy.XPATH, "//XCUIElementTypeButton[@name='"+number+"']")))
tap_on.click()
def iOS_tap_operator(self):
time.sleep(2)
tap_on = self.wait.until(
EC.presence_of_element_located((MobileBy.XPATH, self.add_operator)))
tap_on.click()
def iOS_equals(self, operator):
time.sleep(2)
tap_on = self.wait.until(
EC.presence_of_element_located((MobileBy.XPATH, "//XCUIElementTypeStaticText[@name='"+operator+"']")))
tap_on.click()
def iOS_verify_result(self):
sleep(5)
try:
verify_element = self.wait.until(EC.presence_of_element_located(
(MobileBy.XPATH, self.result))).is_displayed()
except NoSuchElementException:
verify_element = False
return verify_element
The next one is the base page file. We are creating a base page file to make an object of the driver so that we can easily use that for our page and environment file. On this page, we can create a method that gets used frequently in our code like the click() method or send_keys() method, etc.
from selenium.webdriver.support.ui import WebDriverWait
# In the base page we are creating an object of the driver.
# We are using this driver in the other pages and environment page.
class Basepage(object):
def __init__(self, driver):
self.driver = driver
self.wait = WebDriverWait(self.driver, 60)
self.implicit_wait = 25
Step 5:
Environment file (i.e. Hooks file).
This file contains hooks for before and after scenarios to start and close the browser. Also if you want you can add after-step hooks for capturing screenshots for reporting. We have added a method to capture screenshots after every step and will attach them to the allure report. We have added before feature hooks.
In the feature file, we have given tags(@iOS and @android) before the feature.
def before_feature hook: This will check for which device type (iOS or Android) we are executing the code.
def before_scenario hook: We are checking the execution mode and within that adding device type conditions for iOS and Android.
Here we are using “context. config.userdata[]” This will read data from the behave.ini file
INI files are configuration files used by Windows to initialize program settings. The main role is to set values for parameters and configuration data required at startup or used by setup installers.
The configuration files should begin with the keyword [behave] and follow Windows INI style format.
Copy user userName and accessKey of the user BrowserStack account. And iOS_broserstack_appUrl – Uploaded .ipa file through curl command. android_broserstack_appUrl – Uploaded .apk file through curl command.
Congratulations, finally we have created our own Python Selenium Behave BDD framework.
Step 7:
As I mentioned earlier we will be using Allure for reporting the test result. For this use the below command in the terminal and it will generate the result folder for you.
Creating a robust mobile testing framework using Appium is very important as well as feels like a tedious task but with the right guidelines, everyone can create a testing framework. This framework helps improve the quality and efficiency of the testing process. I hope this blog will help everyone to create a robust mobile testing framework using Appium. Here, we choose a behave framework over other existing frameworks because of its better understanding, ease of adaptation, and ease to understand for end users.
Swapnil is an SDET with 1+ years of hands-on experience in Manual, Automation, and API testing. The technologies I have worked on include Selenium, Playwright, Cucumber, Appium, Postman, SQL, GitHub, Java, and Python. Also, I love Blog writing and learning new technologies.
Desktop applications can be accessed from the machines on which they are deployed. Generally, single users only use it, or whoever is signed in to the account at the time. Whereas Web applications don’t have any location constraint to be accessed as they are deployed on servers that can be accessed by any user with an internet connection and can be accessed via a URL or browser.
In terms of determining locators to use in test scripts, a different approach is necessary for different applications. With web applications, for instance, the browser’s developer tools can inspect the HTML elements on the page. But for desktop applications, you need to download an application to be able to inspect UI elements on the screen. So in software testing, when it comes to testing desktop applications, one of the key challenges is to identify the locators for different UI elements in an application.
In this blog, we will explore different tools specifically designed to help testers to find the right locators for UI elements in desktop applications and also discuss how to use these tools effectively.
1. Inspect tool
2. WinAppDriver UI Recorder
3. Appium
1. Inspect Tool:
What is an Inspect Tool?
It is a Windows-based tool that can select any UI element and view its accessibility data and also test the navigational structure of the automation elements in the UI Automation tree and the accessible objects in the Microsoft Active Accessibility hierarchy.
Where do you get Inspect tool?
This is shipped together with Visual studio.Inspect.exe. and it’s inside the SDK directory like this: “C:\Program Files (x86)\Windows Kits\10\bin\10.0.16299.0\x64\inspect.exe”
Note: If Windows kits are not available on your machine then use the following location to download Windows SDK https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/
Now let’s see how to inspect desktop application elements using Inspect tool:
1. Double-click on the Inspect.exe file to open the Inspect tool: When you start Inspect, you will see Tree view on the left side it will show all the applications open in your machine, and on the right side you will see Data view which shows all the accessibility information about selected element also you can navigate the UI to view accessibility information about every element in the UI. By default, Inspect tracks the keyboard or mouse focus. As focus changes, the data view updates with the property information of the element with focus as shown in the below screenshot:
2. Second step is to open the desktop application you want to inspect.
3. Move your mouse cursor over the element that you want to inspect: then on the left side of the tool you will see the hierarchical structure of UI elements as a tree-view control and on the right side, you will see all the properties of selected elements like Name, AutomationId, Class Name, Control Type, Native Window Handle and so on… You can also try the following options from Inspector
MSAA Mode- Displays Microsoft Active Accessibility property information.
Active Hover Toolbar-Activates toolbar buttons on mouse hover, instead of requiring a mouse click.
Show Highlight Rectangle-Highlights a rectangle around the element with focus.
Show Information Tooltip-Shows property information in a tooltip.
then you can use these properties in your code as examples shown below:
winAppDriver.findElementByName("Add").click();
winAppDriver.findElementByAccessibilityId("93").click();
winAppDriver.findElementByXPath("//*[@LocalizedControlType='button' and contains(@Name,'Add')]");
2. WinAppDriver UI Recorder:
What is a UI recorder?
This is a tool for Windows Application Driver (WinAppDriver) that allows you to record user actions on a Windows application for selected UI elements and view their attribute data to generate XPath queries.
WAD UI Recorder will record the absolute xPath for an application object on mouse hover or click. Then you can make some slight modifications to the recorded absolute xPath for use within your step. You can also use the absolute xPath recorded in WAD UI Recorder to help you construct a relative xPath that may be more readable and manageable within your Cycle steps.
Where do we get a UI recorder?
Download the WinAppDriverUIRecorder.zip from the following link:
Then in this unzip file you will see WinAppDriverUIRecorder.exe to launch the UI recorder
Now let’s see how to inspect desktop application elements using UI Recorder:
1. Run WAD UI Recorder as administrator. The application window will open on your desktop which is divided into two panels, as shown below. The top panel will display the absolute XPath recorded by the application and the bottom panel displays xPath node information.
There are two buttons located at the bottom of the window which are used to Record and Clear the panels. The Record button will change to a Pause button while the recording is in progress.
2. Open the desktop application you want to inspect
3. Click on the Record button to start recording xPath. The WAD UI Recorder will now start recording xPaths of any object or element you interact with on your desktop.
4. Move your mouse cursor or click on the element in your target application to generate the absolute xPath. The xPath appears in the top panel of WAD UI Recorder. Clicking the Pause button will stop the WAD UI Recorder from recording xPath.
Then Xpath is recorded in WAD UI Recorder, as shown below
5. Copy the xPath from the top panel of WAD UIRecorder and paste it into your text editor of choice then will need to be modified in order to use it in your step. You will need to replace the escaped double quote characters ( \” ) with single quotes ( ‘ ) and change the starting point of the xPath and also remove the “Desktop” portion from the absolute xPath. This portion of the xPath is not necessary then we can use that xPath in your step as shown below:
Appium is an open-source tool used for automating mobile and desktop applications. It supports a variety of programming languages, including Java, PHP, Objective C, Python, and JavaScript.
Appium is a combination of two components:
Appium Server: To ensure test automation of the applications, we use the server.
Appium Inspector: We use the feature to inspect and obtain all the specifications of the application’s UI element. For inspecting the element, you have to set the desired capabilities that can inform the Appium server about the type of application and platform you aim to automate.
Where do we get an Appium?
Download the exe file of the Appium desktop setup from the link:
https://github.com/appium/appium-desktop/releases
On this Github page, you will see the latest release details as shown below:
Under the Assets section, click and download the Appium-Inspector-windows exe file and save it on your machine.
Under the Assets section, click and download the Appium-Inspector-windows exe file and save it on your machine.
Open the downloaded exe folder and double-click on the exe to start the installation process.
After completing the installation, the Appium GUI window will open as shown below.
Now let’s see how to inspect desktop application elements using Appium:
1. First step before opening the Appium you should launch the winAppDriver by running the nan automation script.
2. Second step is to open the app from which you can inspect the elements
3. Third step is to start the Appium server by clicking on the “Start Server” button it will launch the Appium server on your local machine.
4. Fourth step is to click on Attached Session button, then attach the root path of the application with Appium by clicking Attach Session button present at the bottom
5. Then in App Source you will see all the elements and all opened apps and in Selected Elements, you will see the details of selected elements such as ID, class name, or XPath, that you can use in your test script as shown in the following example.
In the Inspect tool, you will immediately get the element attributes, there is no need to start the session every time and you can directly use these attributes in your script without any modifications. Whereas with a UI recorder, you can get an Xpath and with Appium you will get a unique xpath but you need to start the recorder and Appium every time. Along with these tools, you can also explore TestArchitect, Ranorex, Tricentis Tosca, SikuliX, and Askui’s James tools.
Conclusion:
I have used Inspect, UI Recorder, and Appium Inspector tools, the ease of using these tools depends upon the application’s complexity. For me, getting attributes of the UI element Inspect tool will prove better and for Xpath UI recorder will help.
An SDET with hands-on experience in Manual and Automation testing for Desktop and Web applications. Practical experience in Python, Java, Selenium, Playwright, Behave, Pytest, Appium, and Mysql.
Selenium is an open-source Web UI automation testing suite/tool. It supports automation across different browsers, platforms, and programming languages which includes Java, Python, C#, .net, Ruby, PHP, and Perl, etc. for developing automated tests. Selenium can be easily deployed on Windows, Linux, Solaris, and Macintosh Operating Systems. It also provides support for mobile applications like iOS, windows mobile, and Android for different Operating Systems.
Selenium consists of drivers specific to each language. Additionally, the Selenium Web driver is mostly used with Java and C#.
Test scripts can be coded in selenium in any of the supported programming languages and can be run directly in most modern web browsers which include Internet Explorer, Microsoft Edge, Mozilla Firefox, Google Chrome, Safari, etc.
Furthermore, C# is an object-oriented programming language derived from C++ and Java. It supports the development of console, windows, and web-based applications using Visual Studio IDE on the .Net platform.
With Selenium C#, there is a wide variety of automation frameworks that can be used for automated browser testing. Each framework has its own advantages and disadvantages, they are chosen on the basis of their requirement, compatibility, and the kind of solution they’d prefer. These are the most popular Selenium C# frameworks used for test automation.
NUnit:
It is a unit testing tool ported initially from JUnit for .Net Framework and is an Open Source Project. NUnit was released in the early 2000s, though the initial Nunit was ported from Junit, the recent .Net version 3 is completely rewritten from scratch.
To run the Nunit test we need to add attributes to our methods. An example, attribute [Test], Indicates the Test method. Below are the NuGet Packages required by NUnit
NUnit NUnit3TestAdapter Microsoft.NET.Test.Sdk
XUnit:
XUnit is a unit testing tool for .Net Framework which was released in 2007 as an alternative to Nunit. xUnit has attributes for the execution of tests but is not similar to NUnit. [Fact] and [Theory] attributes are similar to [Test]
MSTest is a unit testing framework developed by Microsoft and ships with Visual Studio. However, Microsoft made version 2 open-source which can easily be downloaded. Additionally, MSTest has an attributes range similar to NUnit and provides a wide range of attributes along with parallel run support at the Class and Method level.
Prerequisite:
To get started with Selenium C# and the Page Object Model framework, first, we need to have the following things installed.
1) IDE: Download and install any IDE of your choice.
After downloading the Visual Studio Installer, select the .NET desktop development option and then click on Install.
Now let the Visual Studio Installer download the packages and perform the installation.
Install the latest version of the .NET Framework on your machine.
2) Create New Project: After the installation is over, begin using Visual Studio.
select the Create a new project option, then select the xUnit Test Project option for C#.
3) Selenium Webdriver for Chrome Browser: You must also install Selenium’s web driver for Chrome browser.
In Visual Studio navigate to Tools -> NuGet Package Manager -> Manage NuGet Packages for Solution.
In the Search Bar, enter the name of the packages you want to install (e.g. Selenium .WebDriver).
Check the Project checkbox, and click on Install.
In the dialogue box asking to accept the licences click on Accept button.
This will start the installation process and install the Selenium WebDriver.
Selenium.WebDriver
This package contains the .NET bindings for concise and object-based
Selenium WebDriver API, which uses native OS-level events to manipulate the browser, Selenium.Chrome.WebDriver (chrome driver exe) This NuGet package installs Chrome Driver (Win32) for Selenium WebDriver in your xUnit Test Project.
Once Visual Studio is finished with the successful installation of the Selenium WebDriver, it will show a message in the output window. Once the Visual Studio is set up with all dependencies, it’s ready for work.
Note:We will be using the demo testing website (https://www.calculator.net/) and trying to achieve the addition and subtraction operations for our automation test.
Writing the First Selenium C# Test:
Download the WebDriverManager from Tools -> NuGet Package Manager -> Manage NuGet Packages for Solution.
WebDriverManager is an open-source Java Library that automates the management of driver executables required by Selenium WebDriverby performing the four steps (find, download, setup, and maintenance) for the drivers required for Selenium tests. Here are some benefits of WebDriverManager in Selenium:
WebDriverManager automates the management of WebDriver binaries, thereby avoiding installing any device binaries manually.
WebDriverManager checks the version of the browser installed on your machine and downloads the proper driver binaries into the local cache (~/.cache/selenium by default) if not already present.
WebDriverManager matches the version of the drivers. If unknown, it uses the latest version of the driver.
WebDriverManager offers cross-browser testing without the hassle of installing and maintaining different browser driver binaries.
In the UnitTest1 file, the final code looks like this:
public class UnitTest1
{
IWebDriver driver;
CalculatorPage calc_page;
public void Initialize_driver()
{
new WebDriverManager.DriverManager().SetUpDriver(new ChromeConfig());
driver = new ChromeDriver();
calc_page = new CalculatorPage();
}
public void Close_driver()
{
driver.Close();
}
[Fact]
public void Add()
{
initialize_driver();
calc_page.Initialize(driver);
string actualresult = calc_page.calculate("14", "+", "5");
Assert.Equal("19", actualresult);
Close_driver();
}
[Fact]
public void Subtract()
{
initialize_driver();
calc_page.Initialize(driver);
string actualresult = calc_page.calculate("24", "-", "5");
Assert.Equal("19", actualresult);
Close_driver()
}
}
Now just build your code by right-clicking the project xUnitTestProject1 or by pressing Ctrl + Shift + B and you will be able to see your test in “Test Explorer”.
After following the above procedure, run the test case. But this code will not execute unless the Chrome driver for the Selenium is not downloaded and unzipped on the system.
When developing a scalable and robust automation framework, it is important to consider the following challenges:
Keeping up with UI changes: The primary goal of automated UI web tests is to validate the functionality of web page elements. However, the UI is subject to constant evolution, leading to changes in web locators. These frequent changes in web locators pose a challenge to code maintenance.
Code maintenance: With the ever-changing UI, it is crucial to maintain the automation codebase effectively. Failing to update Selenium test automation scripts to reflect changes in web locators can result in test failures. Proper maintenance is essential to ensure the longevity and reliability of the test scripts.
Test failure due to lack of maintenance: Inadequate maintenance of automation scripts can lead to scenarios where tests fail. One common cause is a change in web locators. If the Selenium test automation scripts are not updated accordingly, it can cause a significant number of tests to fail, impacting the overall test suite’s reliability.
So to address this, restructure the Selenium test automation scripts for increased modularity and reduced code duplication.
Utilizing the Page Object Model (POM) design pattern achieves code restructuring and minimizes the effort required for test code maintenance.
Now, let’s delve into a comprehensive overview of the Page Object Model, including the implementation and effective maintenance of your Selenium test automation scripts.
Why do we need Page Object Model in Selenium C#?
Selenium test automation scripts become more complex as the web applications add more features and web pages. With every new page added, new test scenarios are included in the Selenium test automation scripts. With this increase in lines of code, its maintenance can become very tedious and time-consuming. Also, the Repetitive use of web locators and their respective test methods can make the test code difficult to read.
Instead of spending time updating the same set of locators in multiple Selenium test automation scripts, a design pattern such as the Page Object Model can be used to develop and maintain code.
What is Page Object Model In Selenium C#?
Page Object Model is the most widely used design pattern by the Selenium communityfor automation testsin which each web page (or significant ones) is considered as a separate class and a central object repository is created for controls on a web page.
Each Page Object (or page classes) contain the elements of the corresponding web page along with the necessary methods to access the elements on the page.
Thus it is a layer between the test scripts and UI and encapsulates the features of the page.
The Selenium test automation scripts do not interact directly with web elements on the page, instead, a new layer (i.e. page class/page object) resides between the test code and UI on the web page.
Hence, Selenium test automation implementation that uses the Page Object Model in Selenium C# will constitute different classes for each web page thereby making code maintenance easier.
In complex test automation scenarios, automation scripts based on Page Object Model can have several page classes (or page objects). It is recommended that you follow a common nomenclature while coming up with file names (representing page objects) as well as the methods used in the corresponding classes. For example, if automation for a login page & dashboard page is to be performed, our implementation will have a class each for login & dashboard. The controls for the login page are in the ‘login page’ class and the controls for the dashboard page are in the ‘dashboard page’ class.
How to Use Page Object Model:
We will now implement the Page Object Model for the use case we considered above i.e. trying to achieve the addition and subtraction operations for our automation test on the Calculator page. Create a class file – CalculatorPage.csfor Calculator page operation. This page class contains the locator information of the elements on that page. Also, we need to define the methods for that page in the CalculatorPage.cs class and call the methods from UnitTest1.cs.
We are initializing the Chromedriver object and launching the web page from the initialiseDriver() method from UnitTest1.cs. Also, we are creating the instance of CalculatePage from the same method. The CalculatePage.cs contain an instance of IWebDriver and the following methods –
Initialize(): this method takes one IWebDriver object as an input parameter and it is assigned to a locally defined IWebDriver object. Also, the required web page is launched using this driver.
Calculate(): this method is actually used to do the calculation operation of two numbers..either addition or subtraction using 3 input parameters as user input number value1, number value 2, and operator like -’+’ or ‘-’. The required elements from the page are located and as per the operator, the required operation is performed on those. The final code of CalculatePage.cs would look like the below:
public class CalculatePage
{
IWebDriver driver;
public void Initialize(IWebDriver driver)
{
this.driver = driver;
driver.Navigate().GoToUrl("https://www.calculator.net/");
}
public string Calculate(string no1, string op, string no2)
{
IWebElement number1;
char[] ch = no1.ToCharArray();
for (int i = 0; i < no1.Length; i++)
{
number1 = driver.FindElement(By.XPath("//span[@onclick='r(" + ch[i] + ")']"));
number1.Click();
}
IWebElement op_element = driver.FindElement(By.XPath("//span[@onclick=\"r('" +op + "')\"]"));
op_element.Click();
ch = no2.ToCharArray();
for (int i = 0; i < no2.Length; i++)
{
number1 = driver.FindElement(By.XPath("//span[@onclick='r(" + ch[i] + ")']"));
number1.Click();
}
IWebElement result = driver.FindElement(By.Id("sciOutPut"));
string actual_result = result.Text.Trim();
return actual_result;
}
}
Advantages of Page Object Model in Selenium C#:
Page Object Model is a widely used design pattern nowadays. It reduces code duplication, enhances code readability, and improves maintainability by emphasizing reusability and extensibility. Furthermore, below are some of the major advantages of using the Page Object Model in Selenium C#.
Better Maintenance – With separate page objects (or page classes) for different web pages, functionality or web locator changes will have less impact on the change in test scripts. This makes the code cleaner and more maintainable as Selenium test automation implementation is spread across separate page classes.
Minimal Changes Due To UI Updates – The effect of changes in the web locators will only be limited to the page classes, created for automated browser testing of those web pages. This reduces the overall effort spent in changing test scripts due to frequent UI updates.
Reusability – The page object methods defined in different page classes can be reused across Selenium test automation scripts. This, in turn, results in a reduction of code size as there is increased usage of reusability with the Page Object Model in Selenium C#.
Simplification –One more important point of using this design pattern is that it simplifies the visualization of the functionality and model of the web page as both these entities are located in separate page classes.
Execution:
Navigate to Test -> Run All Tests. This will launch the test explorer in Visual Studio and will run our test.
You can run the test from the command prompt or visual studio’s terminal (Developer Command Prompt) with the following command-
dotnet test
This dotnet test command is used to run the tests in the project in the current directory. The dotnet test command builds the solution and runs a test host application for each test project in the solution. While running the tests from the project, you can put different filters while running the test, like running the tests with particular tags, from specific projects, with particular names, etc.
You can find this framework in the following Git Repository.
Implementing the Page Object Model in Selenium with C# provides a structured approach to automation testing, making the code more maintainable and reusable. It simplifies the handling of UI changes and enhances the overall efficiency of the testing process for large-scale applications.
Trupti is a Sr. SDET at SpurQLabs with overall experience of 9 years, mainly in .NET- Web Application Development and UI Test Automation, Manual testing. Having hands-on experience in testing Web applications in Selenium, Specflow and Playwright BDD with C#.
In this Blog will learn How to create a BDD automation framework using Cucumber in JavaScript and Playwright. The playwright is an open-source automation tool. Playwright offers several features that make it stand out are Multi-browser support, Automatic waiting Headless and headful mode, etc. Cucumber is a popular open-source BDD(Behavior Driven Development) testing framework, Which helps build a framework that can be easily understood by both technical and non-technical stakeholders.
Prerequisite:
To get started with Cucumber BDD with playwright-js we need to have the following things installed.
2)Node.js: It is a cross-platform JavaScript runtime environment that allows developers to run JavaScript code outside of a web browser. To get Node.js you can visit: https://nodejs.org/en/download To check Node.js is installed on your system you can use the command ‘node –version’ on CMD or VS Code Terminal.
3)Cucumber (Gherkin) Full Support Extention: This extension provides support for the Cucumber (Gherkin) language to VS Code. To get the extension click on the extension icon on the left side panel of VS Code, search for the extension and install it.
Project setup:
Before starting with framework development we need to create a folder structure.
Create a folder structure with the following steps: 1) Create a folder as ‘Playwright-JS-Demo’, 2) In VS Code open the Playwright-JS-Demo folder 3) Install Playwright and Cucumber by executing the following commands in terminal a. npm init playwright@latest
b.npm i @cucumber/cucumber After execution of these commands, you can see package.json, node_modules, and playwright.config.js is created in the folder structure.
4) once you open the project, create a folder structure as below.
As our project structure is ready, we can start with the framework.
1. TestHooks.js:
Test Hooks mainly contain methods to execute Before and After every execution.
The Before hook gets executed before each Scenario in a cucumber test suite.
In this hook, a new Chrome browser instance is launched for every Test Case.
A new page is created in context with ‘await context.newPage()‘ and assigned to the global variable ‘page‘, which is accessible for any Test Case. This ensures that the browser page is available for each scenario in the test suite.
Once execution is done After the method gets executed It closes the current browser instance.
With Cucumber-BDD we can write scenarios in understandable language. Here I have created a Scenario for OrangeHRM Login.
We can create a separate scenario for each functionality.
Every step in a Feature File describes the action we are going to perform on UI.
In the feature file, we can add specific tags for scenarios or complete feature file ex. @login, @smoke.
If you add a tag for a specific scenario then it will only execute the particular scenario. But if you add tags for the Feature It will execute all the Scenarios from the Feature File.
It will make test execution easy if you want to execute test cases for specific functionality.
Feature: Login Feature
@login
Scenario: Login to OrangeHRM
When I Visit the OrangeHRM login page
And I enter username
And I enter Password
And I click on Login button
Then I verify dashboard URL
3. LoginSteps.js:
Essentially, the purpose of the step file is to attach steps from the feature file to the page file, where actual implementation is available.
We use the “Given-When-Then” (BDD) format to write step definitions. The required statement imports the necessary modules like:
Cucumber library that contains definitions for When Then etc.
A custom LoginPage module that likely contains functions for interaction with the login page.
The steps with ‘When’ are related to the user actions like navigation, clicking on the button, and filling in the information in input boxes, etc. The steps with ‘Then’ are related to the verifications or Assertions, just like in this case I have verified if the Login is successful.
The page file contains the actual implementation of the scenario, We also defined all the functions needed for test execution. Additionally, the LoginPage class contains functions that interact with the login page elements, like navigation to the Login page, entering the username and password, clicking the login button, and verifying the dashboard URL. Moreover, the playwright provides different functions to handle the UI elements and pages. For this test case I have used goto(), click(), fill() waitFor().
process.env.WEB_URL, process.env.WEB_USERNAME and process.env.WEB_PASSWORD are the variables we are accessing from the .env file. (The use of the .env file is explained below in point no 6).
To access the .env file I have imported const path = require(‘path’);
In this file, we specify the paths of all the required files. It helps to identify files in the framework. Whenever you create any new file in the framework, you have to add a path for that new file in this Cucumber.json file so that it can be accessible during the test case execution.
.env file is a Configuration file that contains environment variables. Using a .env file is a best practice for keeping sensitive information separate from the code. Moreover, for this framework, we have stored information like URL to navigate and username and password to Login into the OrangeHRM Web application.
We are executing the ‘npx’ command to run the cucumber-js package which is a test framework for BDD. 1) –require ./steps/*.js specifies The step files from the specified path that should be loaded. 2)–tags @login specifies scenarios with @login tags are going to be executed. You can add more than one tag if needed.
3)–publish flag specifies that test results should be published to the Cucumber Cloud which is a service for storing and analyzing test results.
To start execution you can execute the command ‘npm run test’ in the terminal. Once execution is completed you can see the link for cucumber reports is available in the terminal. By clicking on this link you can see the execution report.
This is how the report looks when you click on the above URL
In conclusion, the BDD automation framework using Cucumber in JavaScript and Playwright helps improve the quality and efficiency of their testing process. This framework will help you to write test cases for any web application very efficiently, It also provides a great reusability of code.
Priyanka is an SDET with 2.5+ years of hands-on experience in Manual, Automation, and API testing. The technologies she has worked on include Selenium, Playwright, Cucumber, Appium, Postman, SQL, GitHub, and Java. Also, she is interested in Blog writing and learning new technologies.
