Let’s explore the topic of image comparison testing, which is often overlooked by testers who typically focus more on validating texts, buttons, forms, fields, and other similar elements. However, in some instances, businesses may require testing of images such as logos, infographics, and other graphics.
I encountered such a scenario in my project, where I had to compare two images from different windows. This project was related to life sciences, specifically comparing images of two different animals. During this process, I discovered two methods for image comparison testing through automation using Selenium WebDriver and Java. Today, we will look into these two aspects of image comparison testing.
I don’t think any industries are left that are not involved in the minor or major level of image testing. At least, organizations will ask testers to perform testing of the logo placed on their website and it is very common to ask. Still, I have listed examples of some of the industries below where Image comparison testing can be extensively used.
Prerequisite:
Need to download API dependency, BufferedImage library (Optional – Pointlib and Sikuli )
Applications of Image Comparison in Various Domains:
E-commerce: Ensuring accurate representation of products and their images is crucial for online retailers.
Advertising and Marketing: Verification of visual advertisements, banners, and promotional materials is essential for maintaining brand consistency.
Gaming: Testing game graphics, character designs, and visual effects are vital for delivering an immersive gaming experience.
Healthcare and Medical Imaging: Evaluating medical images for accuracy and precision is critical for diagnostic purposes.
Automotive: Comparing images of vehicle designs, safety features, and user interfaces helps ensure optimal user experiences.
Image comparison cannot be directly performed using Selenium WebDriver alone. However, when there is a specific need for image comparison, we can rely on third-party APIs to achieve this functionality. One such API is AShot, which allows us to compare two images. In this blog, I will explain how to compare two images using the AShot API.
Sometimes during automation, we need to compare two images for verification. We can compare two images using Java selenium with the help of Ashot API as the web driver does not support image comparison.
Step-by-Step Implementation Guide:
We need to capture the screenshot of a particular element and store it in the framework of the project during execution.
Then compare the captured image with the expected image which was already stored in the project.
To capture the image during automation Selenium has to take a screenshot interface which allows us to take the screenshot of the whole screen. The code snippet to take screenshots is explained below
But when there is a need to take a screenshot of a particular element or image then there are various methods available in Selenium like BufferedImage, PointLib library, and external libraries like Sikuli.
While doing image comparison some things need to be kept in mind such as both images should have the same dimensions and if images are grid images then both images must be grid images otherwise they won’t be recognized as the same image.
Java ImageIO class:
This is a final class that belongs to the javax.imageio package. This class provides a convenient method for reading and writing images and performing simple encoding and decoding. The class provides a lot of utility methods related to image processing. Using this class, we can deal with popular image extensions like .jpg, .gif, .png, etc.
BufferedImage class:
This is a subclass of the Image class. It handles and manipulates the image data. A ColorModel of image data makes up a Buffered Image. The bufferedImage class consists of so many things like getHeight, getWidth,getRGB, etc.
In the first Image comparison method, we take both images’ RGB values with the Color class’s help. The Color class is a part of the Java Abstract Window Toolkit (AWT) package. The Color class creates color by using the given RGBA values and has different methods which return the component in the range of 0-255.
Comparison Scenarios: Considered Image Scenarios:
Positive scenario:
Where both the images are the same. For this scenario both the methods mentioned below will give the result as – images are the same.
Negative scenario:
Where one small change is made in the images. one small black dot is added in the image as seen below. For this scenario, the first method will give the result as images are not the same and the second method will give the result as images are the same. So we can confirm that the first method is more accurate to check every small change in the image.
Method 1:
This method of comparing the RGB value is more reliable than the second one. The second method only reflects the difference when there is a significant change in the images.
Here is the link provided for the code of both methods.
So here, we use the BufferedImage class to read the images and save them as BufferedImage objects named img1 and img2, respectively.
By using the getWidth()and getHeight() methods we are reading the height and width of both the images as the height and width of both the images should be the same otherwise it will not proceed further to find out the difference. And if it is not the same, the program will print “Both images should have the same dimensions” as we compare them in an if loop.
If the images are the same then it will execute 2 for loop for height and width. We are reading the RGB value of both the images by method getRGB. And we are storing that value in integers defined as pixels and pixel integer values are passing values into the Color class.
The color class has a method to read the red, green, and blue values of the images and then find the sum of the differences in RGB values of the two images.
From the differences we find out the average and the percentage and if the percentage is equal to 0 then the images are the same and if the rate is more than 0 then the photos are different.
Method 2:
Let’s Understand Method 2 in detail
Again, we are using the BufferedImage class to read the images, and we are saving them as BufferedImage objects named img1 and img2, respectively.
Now we are creating the object of the ImageDiffer class and the ImageDiffer class has a built-in method makeDiff which compares two images. If there is a difference, it returns the ImageDiff class object.
Then we are using the hasDiff built-in method to check the value of the diff objects and confirm whether the images are different.
This way we can compare two images whether they are the same or not using Ashot. Here the intention of showing a negative scenario is to make sure that the scenarios are working fine even if there is a small change in the expected image. Method 1 is more accurate as it is comparing the RGB values of the images and when there is a small change in the image also it will give an accurate result.
Conclusion:
Image comparison using the AShot API provides a reliable and efficient method to verify and validate images during automation testing. By using the capabilities of Ashot and implementing the image comparison process, we can enhance the quality and reliability of our automation tests, ensuring that the visual aspects of our applications meet the desired expectations.
An SDET with hands-on experience in Functional testing on Web and Desktop applications, Requirement analysis, and Defect reporting. Practical experience in Cucumber, Selenium, Java, TestNG, Playwright, Appium, X-Ray, Jira and SQL.
In this blog, we will explore how to build a robust mobile test automation framework using Appium in Python (behave framework). As a result, it will be very useful for executing the program.
Mobile test automation can be more challenging than web automation, as inspecting and interacting with mobile elements requires additional effort. However, with the help of Appium, an open-source tool, it is possible to overcome these challenges and build a powerful mobile test automation framework. In this blog, we will explore how to create a robust framework using Appium in conjunction with the Behave framework in Python.
Let’s talk about robust test frameworks
Robust test automation framework ranks highly on the list of Software Testing “must-haves”.
It helps improve the overall quality and reliability of software when executed in a structured manner.
If we don’t build the right framework then the results will be: Inconsistent test results, Non-modularized tests, and Maintenance difficulties. The automation framework needs to be well organized to make it easier to understand. An organized framework provides an easier way to maintain and expand.
There are many features that we should consider to make the automation framework more robust.
Scalability – The automation framework that you have in your organization should be scalable. It should not just apply to one project. Your automation framework should be applied throughout projects across the organization. It should be an organization-wide test automation framework.
Re-portability – Every automation framework should have a good reporting capability. The test framework engineer can choose a third-party reporting library.
Configurable – A framework should be configurable. It should execute scripts in different test environments. The automation framework should not be restricted to a single test environment. The user credentials should not be “hard-coded” in the automation script itself.
Re-usability – The framework should follow re-usability. We should use the same methods, and page objects in all the test scenarios in the test automation framework.
Extendability – You should be able to integrate easily with other third-party tools via APIs. Automation frameworks should be easily integrated with security testing tools, web proxy debugging tools, test case management tools, or with other global frameworks thereby making it more hybrid in nature.
Reduce the product release cycle time – Improve the time to market, Reduce QA cycle time
Let’s start with basic
Appium is an open source Test Automation Framework which is used for automating mobile applications.
Appium supports Android, IOS mobile apps, and Windows PC Desktop apps. We can automate Native, Hybrid, and Mobile web apps using Appium.
Uses of Appium:
Appium is open source and it is free of cost.
Appium supports Android, IOS, and Windows to run test scripts.
Appium supports languages such as Python, Java, Perl, PHP, C#, and Ruby
Appium supports different operating systems such as Mac, Windows, Linux, UNIX, etc.
Functional test cases of mobile applications can be easily automated.
Appium Inspector
Appium Inspector is a tool for QA engineers who want to automate mobile applications. Basically, this tool also serves as the standard procedure for identifying mobile application elements.
The following are the used for inspecting the mobile element for both Android and iOS.
After downloading the exe file launch the Appium inspector.exe file. On top of the web page, select to Cloud-based platform – BrowserStack
BrowserStack is a cloud-based real devices platform that provides support for both manual and automated testing of mobile apps for both Android and iOS devices. One of its standout features is the App Live feature, which allows users to manually test their mobile apps on over 3000 real Android and iOS devices.
BrowserStack supports testing across different environments, including Dev, QA, Staging, and Production apps from the play store or app store. This makes it easy for developers to test their apps in various environments and ensure that they are working correctly in each environment.
To proceed further we need BrowserStack Username and BrowserStack Access Key
Log in to your BrowserStack account ->Navigate to the “Account” section ->Then Go to Summary
After going to Summery Section you will get Username and Password. Copy both Username and Password and paste them into Appium inspector fields
Go to Desired Capabilities -> Here we need to add basic capabilities which are required for starting the session. Below image will guide you
To add capabilities need to click on the “+” symbol as shown in the below image
Add the capabilities with desired values as shown in the below image
In the Value field, we need to add data that we want to add.
The most important thing here is for the last field “Appium: app”, we have to upload the .ipa or .ipk or .aab file on BrowserStack.
For that, there are 2 ways mentioned below
Through Command Line
Directly through BrowserStack
The most important thing here is for the last field “Appium: app”, we have to upload the .ipa or .ipk or .aab file on BrowserStack.
Let’s start
Through Command Line
To upload the .ipa or .ipk or .aab file on BrowserStack, the following Curl command is very useful.
Curl is a command line tool that enables data exchange between a device and a server through a terminal. Using this command line interface (CLI), a user specifies a server URL (the location where they want to send a request) and the data they want to send to that server URL.
Go to cmd and write this curl command
curl -u “username:accesskey” -X POST “https://api-cloud.browserstack.com/app-automate/upload” -F “file=@path of the file where you save your apk or IPA file” -F “custom_id=any name “
Directly through BrowserStack:
Go to “App Live” -> Click on Uploaded Apps -> And Upload your file.
But here 1st one is preferable: Through this command, we get “app_url” which is required in Desired Capabilities:
Copy that app_url and paste it into Desired Capabilities.
Here you need to Click on Start Session -> You will get below the window.
You can select the element in the App or from the App Source section and the attributes including ID, Name, Text, etc will be displayed on the right side under the Selected Element section and you can create Xpaths using those attributes.
We can also install all the required packages using the requirement.txt file using the below command.
pip install -r requirement.txt
Framework Structure Overview
Here is the overview of our mobile test automation framework using Appium in Python.
You have to follow the below 7 steps to build a robust mobile test automation framework using Appium.
Step 1:
Create a project in Pycharm (here I am using Pycharm professional) and as mentioned in the prerequisites install the packages
Step 2:
In this step, we will be creating a Features folder in which we will be creating our feature files for different scenarios. Every step in a Feature File describes the action we are going to perform on UI. A feature file is something that holds your test cases in the form of a scenario and scenario outline. In this framework, we are using a scenario. Both scenario and scenario outlines contain steps that are easy to understand for non-technical persons. We are giving tags for the feature files. We can also give it for the scenarios present in that file. Depending on our test cases. Note that the feature file should end with a .feature extension.
@iOS
#@android
Feature: Simple Calculator
Addition of two numbers
Scenario: Verify addition of two numbers
Given I am on calculator home page
When I enter '4'
And I enter operator of addition
And I enter operator of addition this should be like And I enter
‘+’ operator so if possible you can update code as per this
And Enter number '2'
And I enter operator '='
Then I see result as '6'
Step 3:
After creating the feature file now create a step file. Both feature files and step files are essential parts of the BDD framework. The steps with ‘When’ are related to the user actions like navigation, clicking on the button, and filling in the information in input boxes. The steps with ‘Then’ are related to the verifications or Assertions. In this, we are using both iOS and Android, so the step file should look like this. We are creating only one-step files for both iOS and Android.
Purpose of Step file: The step file is to attach steps from the feature file to the page file, where actual implementation is available.
from behave import *
use_step_matcher("parse")
@given("I am on calculator home page")
def step_impl(context):
print("User is on Homepage")
@when("I enter '{number}'")
def step_impl(context, number):
str = context.config.userdata["deviceType"]
print("str " + str)
if str == "['iOS']":
context.iOS_cal.iOS_tap_number1(number)
else:
context.android_cal.tap_number1()
@step("I enter operator of addition")
def step_impl(context):
str = context.config.userdata["deviceType"]
print("str " + str)
if str == "['iOS']":
context.iOS_cal.iOS_tap_operator()
else:
context.android_cal.tap_operator()
@step("Enter number '{number}'")
def step_impl(context, number):
str = context.config.userdata["deviceType"]
print("str " + str)
if str == "['iOS']":
context.iOS_cal.iOS_tap_number1(number)
else:
context.android_cal.tap_number2()
@step("I enter operator '{operator}'")
def step_impl(context, operator):
str = context.config.userdata["deviceType"]
print("str " + str)
if str == "['iOS']":
context.iOS_cal.iOS_equals(operator)
else:
context.android_cal.equals()
@then("I see result as '{result}'")
def step_impl(context, result):
str = context.config.userdata["deviceType"]
print("str " + str)
if str == "['iOS']":
flag = context.iOS_cal.iOS_verify_result()
assert flag == True
else:
flag = context.android_cal.verify_result()
assert flag == True
Step 4:
In this step, we are creating two-pagefiles one for iOS and one for Android that contains all the locators and the action methods to perform the particular action on the web element. We are going to add all the locators at the class level only and will be using them in the respective methods.
iOS page file:
import time
from appium.webdriver.common.mobileby import MobileBy
from selenium.common import NoSuchElementException
from selenium.webdriver.support import expected_conditions as EC
from time import sleep
from Features.Pages.BasePage import Basepage
class iOS_Calculator_Page(Basepage):
def __init__(self, context):
Basepage.__init__(self, context.driver)
self.context = context
self.add_operator = "//XCUIElementTypeStaticText[@name='+']"
self.result = "(//XCUIElementTypeStaticText)[1]"
def iOS_tap_number1(self,number):
time.sleep(2)
tap_on = self.wait.until(
EC.presence_of_element_located((MobileBy.XPATH, "//XCUIElementTypeButton[@name='"+number+"']")))
tap_on.click()
def iOS_tap_operator(self):
time.sleep(2)
tap_on = self.wait.until(
EC.presence_of_element_located((MobileBy.XPATH, self.add_operator)))
tap_on.click()
def iOS_equals(self, operator):
time.sleep(2)
tap_on = self.wait.until(
EC.presence_of_element_located((MobileBy.XPATH, "//XCUIElementTypeStaticText[@name='"+operator+"']")))
tap_on.click()
def iOS_verify_result(self):
sleep(5)
try:
verify_element = self.wait.until(EC.presence_of_element_located(
(MobileBy.XPATH, self.result))).is_displayed()
except NoSuchElementException:
verify_element = False
return verify_element
The next one is the base page file. We are creating a base page file to make an object of the driver so that we can easily use that for our page and environment file. On this page, we can create a method that gets used frequently in our code like the click() method or send_keys() method, etc.
from selenium.webdriver.support.ui import WebDriverWait
# In the base page we are creating an object of the driver.
# We are using this driver in the other pages and environment page.
class Basepage(object):
def __init__(self, driver):
self.driver = driver
self.wait = WebDriverWait(self.driver, 60)
self.implicit_wait = 25
Step 5:
Environment file (i.e. Hooks file).
This file contains hooks for before and after scenarios to start and close the browser. Also if you want you can add after-step hooks for capturing screenshots for reporting. We have added a method to capture screenshots after every step and will attach them to the allure report. We have added before feature hooks.
In the feature file, we have given tags(@iOS and @android) before the feature.
def before_feature hook: This will check for which device type (iOS or Android) we are executing the code.
def before_scenario hook: We are checking the execution mode and within that adding device type conditions for iOS and Android.
Here we are using “context. config.userdata[]” This will read data from the behave.ini file
INI files are configuration files used by Windows to initialize program settings. The main role is to set values for parameters and configuration data required at startup or used by setup installers.
The configuration files should begin with the keyword [behave] and follow Windows INI style format.
Copy user userName and accessKey of the user BrowserStack account. And iOS_broserstack_appUrl – Uploaded .ipa file through curl command. android_broserstack_appUrl – Uploaded .apk file through curl command.
Congratulations, finally we have created our own Python Selenium Behave BDD framework.
Step 7:
As I mentioned earlier we will be using Allure for reporting the test result. For this use the below command in the terminal and it will generate the result folder for you.
Creating a robust mobile testing framework using Appium is very important as well as feels like a tedious task but with the right guidelines, everyone can create a testing framework. This framework helps improve the quality and efficiency of the testing process. I hope this blog will help everyone to create a robust mobile testing framework using Appium. Here, we choose a behave framework over other existing frameworks because of its better understanding, ease of adaptation, and ease to understand for end users.
Mrunmayee is a Jr SDET who has expertise in manual and automation testing for web and mobile both. She has worked on Python, Selenium, Cucumber, Java, Appium, Postman, Maven, Behave, BrowserStack, SQL, GitHub, HTML, and CSS. She loves to explore new technologies and products which put an impact on future technologies.
Selenium is an open-source Web UI automation testing suite/tool. It supports automation across different browsers, platforms, and programming languages which includes Java, Python, C#, .net, Ruby, PHP, and Perl, etc. for developing automated tests. Selenium can be easily deployed on Windows, Linux, Solaris, and Macintosh Operating Systems. It also provides support for mobile applications like iOS, windows mobile, and Android for different Operating Systems.
Selenium consists of drivers specific to each language. Additionally, the Selenium Web driver is mostly used with Java and C#.
Test scripts can be coded in selenium in any of the supported programming languages and can be run directly in most modern web browsers which include Internet Explorer, Microsoft Edge, Mozilla Firefox, Google Chrome, Safari, etc.
Furthermore, C# is an object-oriented programming language derived from C++ and Java. It supports the development of console, windows, and web-based applications using Visual Studio IDE on the .Net platform.
With Selenium C#, there is a wide variety of automation frameworks that can be used for automated browser testing. Each framework has its own advantages and disadvantages, they are chosen on the basis of their requirement, compatibility, and the kind of solution they’d prefer. These are the most popular Selenium C# frameworks used for test automation.
NUnit:
It is a unit testing tool ported initially from JUnit for .Net Framework and is an Open Source Project. NUnit was released in the early 2000s, though the initial Nunit was ported from Junit, the recent .Net version 3 is completely rewritten from scratch.
To run the Nunit test we need to add attributes to our methods. An example, attribute [Test], Indicates the Test method. Below are the NuGet Packages required by NUnit
NUnit NUnit3TestAdapter Microsoft.NET.Test.Sdk
XUnit:
XUnit is a unit testing tool for .Net Framework which was released in 2007 as an alternative to Nunit. xUnit has attributes for the execution of tests but is not similar to NUnit. [Fact] and [Theory] attributes are similar to [Test]
MSTest is a unit testing framework developed by Microsoft and ships with Visual Studio. However, Microsoft made version 2 open-source which can easily be downloaded. Additionally, MSTest has an attributes range similar to NUnit and provides a wide range of attributes along with parallel run support at the Class and Method level.
Prerequisite:
To get started with Selenium C# and the Page Object Model framework, first, we need to have the following things installed.
1) IDE: Download and install any IDE of your choice.
After downloading the Visual Studio Installer, select the .NET desktop development option and then click on Install.
Now let the Visual Studio Installer download the packages and perform the installation.
Install the latest version of the .NET Framework on your machine.
2) Create New Project: After the installation is over, begin using Visual Studio.
select the Create a new project option, then select the xUnit Test Project option for C#.
3) Selenium Webdriver for Chrome Browser: You must also install Selenium’s web driver for Chrome browser.
In Visual Studio navigate to Tools -> NuGet Package Manager -> Manage NuGet Packages for Solution.
In the Search Bar, enter the name of the packages you want to install (e.g. Selenium .WebDriver).
Check the Project checkbox, and click on Install.
In the dialogue box asking to accept the licences click on Accept button.
This will start the installation process and install the Selenium WebDriver.
Selenium.WebDriver
This package contains the .NET bindings for concise and object-based
Selenium WebDriver API, which uses native OS-level events to manipulate the browser, Selenium.Chrome.WebDriver (chrome driver exe) This NuGet package installs Chrome Driver (Win32) for Selenium WebDriver in your xUnit Test Project.
Once Visual Studio is finished with the successful installation of the Selenium WebDriver, it will show a message in the output window. Once the Visual Studio is set up with all dependencies, it’s ready for work.
Note:We will be using the demo testing website (https://www.calculator.net/) and trying to achieve the addition and subtraction operations for our automation test.
Writing the First Selenium C# Test:
Download the WebDriverManager from Tools -> NuGet Package Manager -> Manage NuGet Packages for Solution.
WebDriverManager is an open-source Java Library that automates the management of driver executables required by Selenium WebDriverby performing the four steps (find, download, setup, and maintenance) for the drivers required for Selenium tests. Here are some benefits of WebDriverManager in Selenium:
WebDriverManager automates the management of WebDriver binaries, thereby avoiding installing any device binaries manually.
WebDriverManager checks the version of the browser installed on your machine and downloads the proper driver binaries into the local cache (~/.cache/selenium by default) if not already present.
WebDriverManager matches the version of the drivers. If unknown, it uses the latest version of the driver.
WebDriverManager offers cross-browser testing without the hassle of installing and maintaining different browser driver binaries.
In the UnitTest1 file, the final code looks like this:
public class UnitTest1
{
IWebDriver driver;
CalculatorPage calc_page;
public void Initialize_driver()
{
new WebDriverManager.DriverManager().SetUpDriver(new ChromeConfig());
driver = new ChromeDriver();
calc_page = new CalculatorPage();
}
public void Close_driver()
{
driver.Close();
}
[Fact]
public void Add()
{
initialize_driver();
calc_page.Initialize(driver);
string actualresult = calc_page.calculate("14", "+", "5");
Assert.Equal("19", actualresult);
Close_driver();
}
[Fact]
public void Subtract()
{
initialize_driver();
calc_page.Initialize(driver);
string actualresult = calc_page.calculate("24", "-", "5");
Assert.Equal("19", actualresult);
Close_driver()
}
}
Now just build your code by right-clicking the project xUnitTestProject1 or by pressing Ctrl + Shift + B and you will be able to see your test in “Test Explorer”.
After following the above procedure, run the test case. But this code will not execute unless the Chrome driver for the Selenium is not downloaded and unzipped on the system.
When developing a scalable and robust automation framework, it is important to consider the following challenges:
Keeping up with UI changes: The primary goal of automated UI web tests is to validate the functionality of web page elements. However, the UI is subject to constant evolution, leading to changes in web locators. These frequent changes in web locators pose a challenge to code maintenance.
Code maintenance: With the ever-changing UI, it is crucial to maintain the automation codebase effectively. Failing to update Selenium test automation scripts to reflect changes in web locators can result in test failures. Proper maintenance is essential to ensure the longevity and reliability of the test scripts.
Test failure due to lack of maintenance: Inadequate maintenance of automation scripts can lead to scenarios where tests fail. One common cause is a change in web locators. If the Selenium test automation scripts are not updated accordingly, it can cause a significant number of tests to fail, impacting the overall test suite’s reliability.
So to address this, restructure the Selenium test automation scripts for increased modularity and reduced code duplication.
Utilizing the Page Object Model (POM) design pattern achieves code restructuring and minimizes the effort required for test code maintenance.
Now, let’s delve into a comprehensive overview of the Page Object Model, including the implementation and effective maintenance of your Selenium test automation scripts.
Why do we need Page Object Model in Selenium C#?
Selenium test automation scripts become more complex as the web applications add more features and web pages. With every new page added, new test scenarios are included in the Selenium test automation scripts. With this increase in lines of code, its maintenance can become very tedious and time-consuming. Also, the Repetitive use of web locators and their respective test methods can make the test code difficult to read.
Instead of spending time updating the same set of locators in multiple Selenium test automation scripts, a design pattern such as the Page Object Model can be used to develop and maintain code.
What is Page Object Model In Selenium C#?
Page Object Model is the most widely used design pattern by the Selenium communityfor automation testsin which each web page (or significant ones) is considered as a separate class and a central object repository is created for controls on a web page.
Each Page Object (or page classes) contain the elements of the corresponding web page along with the necessary methods to access the elements on the page.
Thus it is a layer between the test scripts and UI and encapsulates the features of the page.
The Selenium test automation scripts do not interact directly with web elements on the page, instead, a new layer (i.e. page class/page object) resides between the test code and UI on the web page.
Hence, Selenium test automation implementation that uses the Page Object Model in Selenium C# will constitute different classes for each web page thereby making code maintenance easier.
In complex test automation scenarios, automation scripts based on Page Object Model can have several page classes (or page objects). It is recommended that you follow a common nomenclature while coming up with file names (representing page objects) as well as the methods used in the corresponding classes. For example, if automation for a login page & dashboard page is to be performed, our implementation will have a class each for login & dashboard. The controls for the login page are in the ‘login page’ class and the controls for the dashboard page are in the ‘dashboard page’ class.
How to Use Page Object Model:
We will now implement the Page Object Model for the use case we considered above i.e. trying to achieve the addition and subtraction operations for our automation test on the Calculator page. Create a class file – CalculatorPage.csfor Calculator page operation. This page class contains the locator information of the elements on that page. Also, we need to define the methods for that page in the CalculatorPage.cs class and call the methods from UnitTest1.cs.
We are initializing the Chromedriver object and launching the web page from the initialiseDriver() method from UnitTest1.cs. Also, we are creating the instance of CalculatePage from the same method. The CalculatePage.cs contain an instance of IWebDriver and the following methods –
Initialize(): this method takes one IWebDriver object as an input parameter and it is assigned to a locally defined IWebDriver object. Also, the required web page is launched using this driver.
Calculate(): this method is actually used to do the calculation operation of two numbers..either addition or subtraction using 3 input parameters as user input number value1, number value 2, and operator like -’+’ or ‘-’. The required elements from the page are located and as per the operator, the required operation is performed on those. The final code of CalculatePage.cs would look like the below:
public class CalculatePage
{
IWebDriver driver;
public void Initialize(IWebDriver driver)
{
this.driver = driver;
driver.Navigate().GoToUrl("https://www.calculator.net/");
}
public string Calculate(string no1, string op, string no2)
{
IWebElement number1;
char[] ch = no1.ToCharArray();
for (int i = 0; i < no1.Length; i++)
{
number1 = driver.FindElement(By.XPath("//span[@onclick='r(" + ch[i] + ")']"));
number1.Click();
}
IWebElement op_element = driver.FindElement(By.XPath("//span[@onclick=\"r('" +op + "')\"]"));
op_element.Click();
ch = no2.ToCharArray();
for (int i = 0; i < no2.Length; i++)
{
number1 = driver.FindElement(By.XPath("//span[@onclick='r(" + ch[i] + ")']"));
number1.Click();
}
IWebElement result = driver.FindElement(By.Id("sciOutPut"));
string actual_result = result.Text.Trim();
return actual_result;
}
}
Advantages of Page Object Model in Selenium C#:
Page Object Model is a widely used design pattern nowadays. It reduces code duplication, enhances code readability, and improves maintainability by emphasizing reusability and extensibility. Furthermore, below are some of the major advantages of using the Page Object Model in Selenium C#.
Better Maintenance – With separate page objects (or page classes) for different web pages, functionality or web locator changes will have less impact on the change in test scripts. This makes the code cleaner and more maintainable as Selenium test automation implementation is spread across separate page classes.
Minimal Changes Due To UI Updates – The effect of changes in the web locators will only be limited to the page classes, created for automated browser testing of those web pages. This reduces the overall effort spent in changing test scripts due to frequent UI updates.
Reusability – The page object methods defined in different page classes can be reused across Selenium test automation scripts. This, in turn, results in a reduction of code size as there is increased usage of reusability with the Page Object Model in Selenium C#.
Simplification –One more important point of using this design pattern is that it simplifies the visualization of the functionality and model of the web page as both these entities are located in separate page classes.
Execution:
Navigate to Test -> Run All Tests. This will launch the test explorer in Visual Studio and will run our test.
You can run the test from the command prompt or visual studio’s terminal (Developer Command Prompt) with the following command-
dotnet test
This dotnet test command is used to run the tests in the project in the current directory. The dotnet test command builds the solution and runs a test host application for each test project in the solution. While running the tests from the project, you can put different filters while running the test, like running the tests with particular tags, from specific projects, with particular names, etc.
You can find this framework in the following Git Repository.
Implementing the Page Object Model in Selenium with C# provides a structured approach to automation testing, making the code more maintainable and reusable. It simplifies the handling of UI changes and enhances the overall efficiency of the testing process for large-scale applications.
Trupti is a Sr. SDET at SpurQLabs with overall experience of 9 years, mainly in .NET- Web Application Development and UI Test Automation, Manual testing. Having hands-on experience in testing Web applications in Selenium, Specflow and Playwright BDD with C#.
In this Blog will learn How to create a BDD automation framework using Cucumber in JavaScript and Playwright. The playwright is an open-source automation tool. Playwright offers several features that make it stand out are Multi-browser support, Automatic waiting Headless and headful mode, etc. Cucumber is a popular open-source BDD(Behavior Driven Development) testing framework, Which helps build a framework that can be easily understood by both technical and non-technical stakeholders.
Prerequisite:
To get started with Cucumber BDD with playwright-js we need to have the following things installed.
2)Node.js: It is a cross-platform JavaScript runtime environment that allows developers to run JavaScript code outside of a web browser. To get Node.js you can visit: https://nodejs.org/en/download To check Node.js is installed on your system you can use the command ‘node –version’ on CMD or VS Code Terminal.
3)Cucumber (Gherkin) Full Support Extention: This extension provides support for the Cucumber (Gherkin) language to VS Code. To get the extension click on the extension icon on the left side panel of VS Code, search for the extension and install it.
Project setup:
Before starting with framework development we need to create a folder structure.
Create a folder structure with the following steps: 1) Create a folder as ‘Playwright-JS-Demo’, 2) In VS Code open the Playwright-JS-Demo folder 3) Install Playwright and Cucumber by executing the following commands in terminal a. npm init playwright@latest
b.npm i @cucumber/cucumber After execution of these commands, you can see package.json, node_modules, and playwright.config.js is created in the folder structure.
4) once you open the project, create a folder structure as below.
As our project structure is ready, we can start with the framework.
1. TestHooks.js:
Test Hooks mainly contain methods to execute Before and After every execution.
The Before hook gets executed before each Scenario in a cucumber test suite.
In this hook, a new Chrome browser instance is launched for every Test Case.
A new page is created in context with ‘await context.newPage()‘ and assigned to the global variable ‘page‘, which is accessible for any Test Case. This ensures that the browser page is available for each scenario in the test suite.
Once execution is done After the method gets executed It closes the current browser instance.
With Cucumber-BDD we can write scenarios in understandable language. Here I have created a Scenario for OrangeHRM Login.
We can create a separate scenario for each functionality.
Every step in a Feature File describes the action we are going to perform on UI.
In the feature file, we can add specific tags for scenarios or complete feature file ex. @login, @smoke.
If you add a tag for a specific scenario then it will only execute the particular scenario. But if you add tags for the Feature It will execute all the Scenarios from the Feature File.
It will make test execution easy if you want to execute test cases for specific functionality.
Feature: Login Feature
@login
Scenario: Login to OrangeHRM
When I Visit the OrangeHRM login page
And I enter username
And I enter Password
And I click on Login button
Then I verify dashboard URL
3. LoginSteps.js:
Essentially, the purpose of the step file is to attach steps from the feature file to the page file, where actual implementation is available.
We use the “Given-When-Then” (BDD) format to write step definitions. The required statement imports the necessary modules like:
Cucumber library that contains definitions for When Then etc.
A custom LoginPage module that likely contains functions for interaction with the login page.
The steps with ‘When’ are related to the user actions like navigation, clicking on the button, and filling in the information in input boxes, etc. The steps with ‘Then’ are related to the verifications or Assertions, just like in this case I have verified if the Login is successful.
The page file contains the actual implementation of the scenario, We also defined all the functions needed for test execution. Additionally, the LoginPage class contains functions that interact with the login page elements, like navigation to the Login page, entering the username and password, clicking the login button, and verifying the dashboard URL. Moreover, the playwright provides different functions to handle the UI elements and pages. For this test case I have used goto(), click(), fill() waitFor().
process.env.WEB_URL, process.env.WEB_USERNAME and process.env.WEB_PASSWORD are the variables we are accessing from the .env file. (The use of the .env file is explained below in point no 6).
To access the .env file I have imported const path = require(‘path’);
In this file, we specify the paths of all the required files. It helps to identify files in the framework. Whenever you create any new file in the framework, you have to add a path for that new file in this Cucumber.json file so that it can be accessible during the test case execution.
.env file is a Configuration file that contains environment variables. Using a .env file is a best practice for keeping sensitive information separate from the code. Moreover, for this framework, we have stored information like URL to navigate and username and password to Login into the OrangeHRM Web application.
We are executing the ‘npx’ command to run the cucumber-js package which is a test framework for BDD. 1) –require ./steps/*.js specifies The step files from the specified path that should be loaded. 2)–tags @login specifies scenarios with @login tags are going to be executed. You can add more than one tag if needed.
3)–publish flag specifies that test results should be published to the Cucumber Cloud which is a service for storing and analyzing test results.
To start execution you can execute the command ‘npm run test’ in the terminal. Once execution is completed you can see the link for cucumber reports is available in the terminal. By clicking on this link you can see the execution report.
This is how the report looks when you click on the above URL
In conclusion, the BDD automation framework using Cucumber in JavaScript and Playwright helps improve the quality and efficiency of their testing process. This framework will help you to write test cases for any web application very efficiently, It also provides a great reusability of code.
Priyanka is an SDET with 2.5+ years of hands-on experience in Manual, Automation, and API testing. The technologies she has worked on include Selenium, Playwright, Cucumber, Appium, Postman, SQL, GitHub, and Java. Also, she is interested in Blog writing and learning new technologies.
Automating API test suite execution through CI/CD pipelines provides a significant advantage over local execution. By leveraging CI/CD, teams can obtain test results for all systems, improving the speed, quality, and reliability of tests. Manual triggering of API suite execution is not required, freeing up valuable time for team members.
In this blog post, we will guide you through the creation of a workflow file using GitHub Actions for your automated API tests. However, before diving into the creation of a CI/CD workflow, it’s essential to understand some crucial points for a better grasp of the concept.
Before we start creating a CI/CD workflow for our API tests I will suggest you first go through the API test automation framework here and also read this blog on creating a web test automation framework as it helps you to understand the different points which we all should consider before selecting the test automation framework. The API test automation framework is in Python language and has Behave library for BDD purposes.
Let’s understand some basic and important points to start with the CI/CD workflow.
What is DevOps?
DevOps is a set of practices and tools that integrate and automate tasks in the software development and IT industry. It establishes communication and collaboration between development and operations teams, enabling faster and more reliable software build, testing, and release processes. DevOps is a methodology that derives its name from the combination of “Development” and “Operations.”
The primary goal of DevOps is to bridge the gap between development and operations teams by fostering a culture of shared responsibility and collaboration. This helps to reduce the time it takes to develop, test, and deploy software while maintaining high quality and reliability standards. By automating manual processes and eliminating silos between teams, DevOps enables organizations to respond more quickly to changing market demands and customer needs.
CI/CD refers to Continuous Integration and Continuous Delivery, which are processes and practices that help to deliver code changes more frequently and reliably. These processes involve automating the building, testing, and deployment of code changes, resulting in faster and higher-quality software releases for end-users.
The CI/CD pipeline follows a workflow that starts with continuous integration (CI), followed by continuous delivery (CD). The CI process involves integrating code changes into a shared repository and automatically building and testing them to identify errors early in the development process. Once the code has been tested and approved, the CD process takes over and automates the delivery of code changes to production environments.
The CI/CD pipeline workflow helps to reduce the risks and delays associated with manual code integration and deployment while ensuring that the changes are tested and delivered quickly and reliably. This approach enables organizations to innovate faster, respond more quickly to market demands, and improve overall software quality.
Process:
What are GitHub Actions?
GitHub Actions is a feature that makes it easy to automate software workflows, including world-class CI/CD capabilities. With GitHub Actions, you can build, test, and deploy your code directly from GitHub, while also customizing code reviews, branch management, and issue-triaging workflows to suit your needs.
The GitHub platform offers integration with GitHub Actions, providing flexibility for customizing workflows to automate tasks such as building, testing, and deploying code. Developers can create custom workflows using GitHub Actions that are automatically triggered when specific events occur, such as code push, pull request merge, or as per a defined schedule.
Workflows are defined using YAML syntax, which is a human-readable data serialization language. YAML is commonly used for configuration files and in applications to store or transmit data. To learn more about YAML syntax and its history, please visit the following link.
Advantages / Benefits of using GitHub Actions for CI/CD Pipeline:
Seamless integration: GitHub Actions seamlessly integrates with GitHub repositories, making it easy to automate workflows and tasks directly from the repository.
Highly customizable: GitHub Actions offers a high degree of customization, allowing developers to create workflows that suit their specific needs.
Time-saving: GitHub Actions automates many tasks in the software development process, saving developers time and reducing the potential for errors.
Flexible: GitHub Actions can be used for a wide range of tasks, including building, testing, and deploying applications.
Workflow visualization: GitHub Actions provides a graphical representation of workflows, making it easy for developers to visualize and understand the process.
Large community: GitHub Actions has a large and active community, providing a wealth of resources, documentation, and support for developers.
Cost Saving: GitHub Actions come bundled with Github free and enterprise licenses reducing the cost of maintaining separate CI/CD tools like Jenkins
Framework Overview:
This is a BDD API automation testing framework. The reason behind choosing the BDD framework is simple it provides you the following benefits over other testing frameworks.
Improved Collaboration
Increased Test coverage
Better Test Readability
Easy Test Maintenance
Faster Feedback
Integration with Other Tools
Focus on Business Requirements
Discover what are the different types of automation testing frameworks available and why to prefer the BDD framework over others here.
Framework Explanation:
The framework is simple because we included a feature file written in the Gherkin language, as you will notice. Basically, Gherkin is a simple plain text language with a simple structure. The feature file is easy to understand for a non-technical person and that is why we prefer the BDD framework for automation. To learn more about the Gherkin language please visit the official site here https://cucumber.io/docs/gherkin/reference/. Also, we have included the POST, GET, PUT & DELETE API methods. A feature file describes all these methods using simple and understandable language.
The next component of our framework is the step file. The feature and step files are the two main and most essential parts of the BDD framework. The step file contains the implementation of the steps mentioned in the feature file. It maps the respective steps from the feature file and executes the code.We use the behave library to achieve this. The behave understands the maps of the steps with the feature file steps as both steps have the same language structure.
Then there is the utility file which contains the methods which we can use more repeatedly. There is one configuration file where we store the commonly used data. Furthermore, to install all the dependencies, we have created a requirement.txt file which contains the packages with specific versions. To install the packages from the requirement.txt file we have the following command.
pip install -r requirement.txt
The above framework is explained in detail here. I suggest you please check out the blog first and understand the framework then we can move further with the workflow detail description. A proper understanding of the framework is essential to understand how to create the CI/CD workflow file.
How to create a Workflow File?
Create a GitHub repository for your framework
Push your framework to that repository
Click on the Action Button
Click on set workflow your self option
Give a proper name to the workflow file
“Additionally, please check out the below video for a detailed step understanding.” The video will show you how to create workflow files and the steps need to follow to do so.
github actions workflow file creation
Components of CI/CD Workflow File:
Events:
Events are responsible to trigger the CI/CD workflow file. They are nothing but the actions that happen in the repository for example pushing to the branch or creating a pull request. Please check the below sample events that trigger the CI/CD workflow file.
push: This event is triggered when someone pushes code to a branch in your repository.
pull_request: This event is triggered when someone opens a new pull request or updates an existing one.
schedule: This event is triggered on a schedule that you define in your workflow configuration file.
workflow_dispatch: This event allows you to manually trigger a workflow by clicking a button in the GitHub UI.
release: This event is triggered when a new release is created in your repository.
repository_dispatch: This event allows you to trigger a workflow using a custom webhook event.
page_build: This event is triggered when GitHub Pages are built or rebuilt.
issue_comment: This event is triggered when someone comments on an issue in your repository.
pull_request_review: This event is triggered when someone reviews a pull request in your repository.
push_tag: This event is triggered when someone pushes a tag to your repository.
To know more about the events that trigger workflows please check out the GitHub official documentation here
Jobs:
After setting up the events to trigger the workflow the next step is to set up the job for the workflow. The job consists of a set of steps that performs specific tasks. For every job, there is a separate runner or we can call it a virtual machine (VM) therefore each job can run parallelly. This allows us to execute multiple tasks concurrently.
A workflow can have more than one job with a unique name and set of steps that define the actions to perform. For example, we can use a job in the workflow file to build the project, test its functionality, and deploy it to a server. The defined jobs in the workflow file can be dependent on each other. Also, they can have different requirements than the others like specific operating systems, software dependencies or packages, or environment variables.
Discover more about using jobs in a workflow from GitHub’s official documentation here
Runners:
To execute the jobs we need runners. The runners in GitHub actions are nothing but virtual machines or physical servers. GitHub categorizes them into two parts named self-hosted or provided by GitHub. Moreover, the runners are responsible for running the steps described in the job.
The self-hosed runners allow us to execute the jobs on our own system or infrastructure for example our own physical servers, virtual machines, or containers. We use self-hosted runners when we need to run jobs on specialized hardware requirements that must be met.
GitHub-hosted runners are provided by GitHub itself and can be used for free by anyone. These runners are available in a variety of configurations. Furthermore, the best thing about GitHub-hosted runners is that they automatically update with the latest software updates and security patches.
Learn more about runners for GitHub actions workflow here from GitHub’s official documentation.
Steps:
Steps in the workflow file are used to carry out particular actions. Subsequently, after adding the runner to the workflow file, we define these steps with the help of the steps property in the workflow file. Additionally, the steps consist of actions and commands to perform on the build. For example, there are steps to download the dependencies, check out the build, run the test, upload the artifacts, etc.
Learn more about the steps used in the workflow file from GitHub’s official documentation here
Actions:
In the GitHub actions workflow file, we use actions that are reusable code modules that can be shared across different workflows and repositories. One or more steps are defined under actions to perform specific tasks such as running tests, building the project, or deploying the code. We can also define the input and output parameters to the actions which help us to receive and return the data from other steps in the workflow. Developers describe the actions, and they are available on GitHub Marketplace. To use an action in the workflow, we need to use the uses property.
Find out more about actions for GitHub actions from GitHub’s official documentation here
Now we have covered all the basic topics that we need to understand before creating our CI/CD workflow file for the API automation framework. Now, let’s start explaining the workflow file.
We use the name property to give the name to the workflow file. It is a good practice to give a proper name to your workflow file. Generally, the name is related to the feature or the repository name.
name: Python API CI/CD Pipeline
Event:
Now we have to set up the event that triggers the workflow file. In this workflow, I have added two events for your reference. The pipeline will trigger the push event for the ‘main‘ branch. Additionally, I added the scheduled event to automatically trigger the workflow as per the set schedule.
The above schedule indicates that the pipeline Runs at 12:00. Action schedules run at most every 5 minutes using UTC time.
We can customize the schedule timing as per our needs. Check out the following chron specification diagram to learn how to set the schedule timing.
Job:
The job we are setting here is to build. We want to build the project and perform the required tasks as we merge new code changes.
jobs:
build:
Runner:
The runner we are using here is a GitHub-hosted runner. In this workflow, we are using a Windows-latest virtual machine. The VM will build the project, and then it will execute the defined steps.
runs-on: windows-latest
Apart from Windows-latest, there are other runners too like ubuntu-latest, macos-latest, and self-hosted. The self-hosted runner is one that we can set up on our own infrastructure, such as our own server, or virtual machine, allowing us to have more control over the environment and resources.
Steps:
The steps are the description of what are the different actions required to perform on the project build. Here, the first action we are performing is to check out the repository so that it can have the latest build code.
steps:
- uses: actions/checkout@v3
Then we are setting up the Python. As this framework is an API automation testing framework using Python and Behave so we need Python to execute the tests.
- name: Set up Python
uses: actions/setup-python@v3
with:
python-version: '3.8.9'
After we install Python, we also need to install the different packages required to run the API tests. Define these packages in the requirement.txt file, and we can install them using the following command.
As of now, we have installed all the packages and we can now run our API tests. We are running these tests with the help of the allure behave command so that once the execution is completed it will generate a Report_Json folder which is required to generate the HTML report.
- name: run test
run: behave Features -f allure_behave.formatter:AllureFormatter -o Report_Json
working-directory: .
continue-on-error: true
Here, we cannot share the generated Report_Json folder as a report. To generate the shareable report we need to convert the JSON folder to that of the HTML report.
Please find the attached GitHub repository link. I have uploaded the same project to this repository and also attached a Readme file that explains the framework and the different commands we have used so far in this project. Also, the workflow explanation is included for better understanding.
Conclusion:
In conclusion, creating a CI/CD pipeline workflow for your project using GitHub Actions streamlines the development and testing process by automating tasks such as building the project for new changes, testing the build, and deploying the code. This results in reduced time and minimized errors, ensuring that your software performance is at its best.
GitHub Actions provides a wide range of pre-built actions and the ability to create custom actions that suit your requirements. By following established practices and continuously iterating on workflows, you can ensure your software delivery is optimized and reliable.
I hope in this blog I have provided the answers to the most commonly asked question and I hope this will help you to start creating your CI/CD pipelines for your projects. Do check out the blogs on how to create a BDD framework for Web Automation and API automation for a better understanding of automation frameworks and how a robust framework can be created.
An SDET Engineer with hands-on experience in manual, automation, API, Performance, and Security Testing. The technologies I have worked on include Selenium, Playwright, Cypress, SpecFlow, Cucumber, JMeter, K6, OWASP ZAP, Appium, Postman, Maven, Behave, Pytest, BrowserStack, SQL, GitHub, Java, JavaScript, Python, C#, HTML, CSS. Also, I have hands-on experience in CI/CD the technologies I have used for CI/CD include Azure DevOps, AWS, and Github. Apart from this, I like to write blogs and explore new technologies.
