Ensuring smooth functionality and an excellent user experience for web applications is more important than ever in today’s digital world. As web applications become increasingly complex, however, traditional testing methods often struggle to meet the demands of modern development. Modern UI automation frameworks, therefore, offer powerful tools for comprehensive and reliable testing.
JavaScript, the backbone of web development, is central to many automation frameworks due to its versatility. Cypress, in fact, has gained popularity for its ease of use, powerful features, and developer-friendly approach, making it a standout in this space. It also streamlines the process of writing, executing, and maintaining automated tests, making it an essential tool for developers and testers alike.
In this blog, we’ll delve into Modern UI automation with JavaScript and Cypress, starting with the setup and then moving on to advanced features like real-time reloading and CI pipeline integration. By the end, you’ll have the knowledge to effectively automate UI testing for modern web applications, whether you’re a seasoned developer or new to automation.
Prerequisites for Modern UI Automation Framework
Before embarking on your journey with JavaScript and Cypress for Modern UI Automation, ensure you must have the following tools in your system and some basic understanding of the technologies i.e. Cypress, Automation, JavaScript and some coding knowledge.
Node.js and npm
Both Node.js and npm are essential for managing dependencies and running your Cypress tests.
VS Code offers a powerful and user-friendly environment for working with JavaScript but also seamlessly integrates with the Cypress framework for modern UI automation. It provides syntax highlighting, code completion, debugging tools, and extensions that can significantly enhance your development experience.
Familiarity with fundamental JavaScript concepts like variables, functions, and object-oriented programming will therefore crucial for writing automation scripts and interacting with the browser.
Cypress is the core framework for your end-to-end (E2E) tests; consequently, it offers a user-friendly interface and powerful capabilities for interacting with web elements.
Here, we’ve looked at the things we need before we start.
Installation for Modern UI Automation Framework
How to Install Node.js on Windows?
What is Node.js?
Node.js is a runtime environment that enables JavaScript to run outside of a web browser; consequently, it allows developers to build scalable and high-performance server-side applications. Originally, JavaScript was confined to client-side scripting in browsers, but with Node.js, it can now power the backend as well.
For testers, Node.js unlocks powerful automation capabilities but also supports tools and frameworks like WebDriver.io and Puppeteer, which automate browser interactions, manage test suites, and perform assertions. Node.js also facilitates custom test frameworks and seamless integration with testing tools. Additionally, it enables running tests in headless environments, ideal for continuous integration pipelines. Overall, Node.js enhances the effectiveness of JavaScript-based testing, improving software quality, speeding up development and UI Automation.
Key Features of Node.js
Asynchronous and Event-Driven: Node.js library APIs work asynchronously; consequently, they are non-blocking. The server moves to the next API call without waiting for the previous one to complete, therefore it using event mechanisms to handle responses efficiently.
High Speed: Built on Google Chrome’s V8 JavaScript engine, Node.js therefore executes code very quickly.
Single-Threaded but Highly Scalable: Node.js uses a single-threaded model with event looping. This event-driven architecture allows the server to respond without blocking, making it highly scalable compared to traditional servers. Unlike servers like Apache HTTP Server, which create limited threads to handle requests, Node.js can handle thousands of requests using a single-threaded program.
No Buffering: Node.js applications do not buffer data; instead they output data in chunks.
Steps to Install Node.js on Windows for UI Automation:
Double-click on the .msi installer to open the Node.js Setup Wizard.
Click “Next” on the Welcome to Node.js Setup Wizard screen.
Accept the End-User License Agreement (EULA) by checking “I accept the terms in the License Agreement” and click “Next.”
Choose the destination folder where you want to install Node.js and click “Next.”
Click “Next” on the Custom Setup screen.
When prompted to “Install tools for native modules,” click “Install.”
Wait for the installation to complete and click “Finish” when done.
Verify the Installation
Open the Command Prompt or Windows PowerShell.
Run the following command to check if Node.js was installed correctly:
node -v
If Node.js was installed successfully, the command prompt will print the version of Node.js installed.
By following these steps, you can install Node.js on your Windows system and start leveraging its capabilities for server-side scripting and automated testing.
How to Install Visual Studio Code (VS Code) on Windows?
What is Visual Studio Code (VS Code)?
Visual Studio Code (VS Code) is a free, open-source code editor developed by Microsoft. It features a user-friendly interface and powerful editing capabilities. VS Code supports a wide range of programming languages and comes with built-in features for debugging, syntax highlighting, code completion, and Git integration. It also offers a vast ecosystem of extensions to customize and extend its functionality.
Steps to Install VS Code for UI Automation
Visit the Official VS Code Website
Open any web browser like Google Chrome or Microsoft Edge.
Click the “Download for Windows” button on the website to start the download.
Open the Downloaded Installer
Once the download is complete, locate the Visual Studio Code installer in your downloads folder.
Double-click the installer icon to begin the installation process.
Accept the License Agreement
When the installer opens, you will be asked to accept the terms and conditions of Visual Studio Code.
Check “I accept the agreement” and then click the “Next” button.
Choose Installation Location
Select the destination folder where you want to install Visual Studio Code.
Click the “Next” button.
Select Additional Tasks
You may be prompted to select additional tasks, such as creating a desktop icon or adding VS Code to your PATH.
Select the options you prefer and click “Next.”
Install Visual Studio Code
Click the “Install” button to start the installation process.
The installation will take about a minute to complete.
Launch Visual Studio Code
After the installation is complete, a window will appear with a “Launch Visual Studio Code” checkbox.
Check this box and then click “Finish.”
Open Visual Studio Code
Visual Studio Code will open automatically.
You can now create a new file and start coding in your preferred programming language.
By following these steps, you have successfully installed Visual Studio Code on your Windows system. You are now ready to start your programming journey with VS Code.
Create Project for Modern UI Automation Framework
Creating a Cypress project in VS Code is straightforward. Follow these steps to get started:
Steps to Create a Cypress Project in VS Code
Open VS Code:
Launch VS Code on your computer.
Click on Files Tab:
Navigate to the top-left corner of the VS Code interface and click on the “Files” tab.
Select Open Folder Option:
From the dropdown menu, choose the “Open Folder” option. This action will prompt a pop-up file explorer window.
Choose Project Location:
Browse through the file explorer to select the location where you want to create your new Cypress project. For this example, create a new folder on the desktop and name it “CypressJavaScriptFramework”.
Open Selected Folder:
Once you’ve created the new folder, select it and click on the “Open” button. VS Code will now automatically navigate to the selected folder.
Congratulations! You’ve successfully created a new Cypress project in VS Code. On the left panel of VS Code, you’ll see your project name, and a welcome tab will appear in the editor.
Now, we are all set to start building your Cypress project in Visual Studio Code!
What is Cypress?
Cypress is a modern, open-source test automation framework designed specifically for web applications and used to UI automation also. Unlike many other testing tools that run outside of the browser and execute remote commands, Cypress operates directly within the browser. This unique architecture enables Cypress to offer fast, reliable, and easy-to-write tests, making it an invaluable tool for developers and testers.
Cypress’s architecture allows it to control the browser in real-time, providing access to every part of the application being tested. This direct control means that tests can interact with the DOM, make assertions, and simulate user interactions with unparalleled accuracy and speed.
Cypress Architecture for Modern UI Automation Framework:
Cypress automation testing operates on a NodeJS server. It uses the WebSocket protocol to create a connection between the browser and the Node.js server. WebSocket’s allow full-duplex communication, enabling Cypress to send commands and receive feedback in real time. This means Cypress can navigate URLs, interact with elements, and make assertions, while also receiving DOM snapshots, console logs, and other test-related information from the browser.
Let’s break down the components and how they interact:
User Interaction:
The process begins with a user interacting with the web application. This includes actions like clicking buttons, selecting values from drop-down menus, filling forms, or navigating through pages.
Cypress Test Scripts:
Developers write test scripts using JavaScript or TypeScript. These scripts simulate user interactions and verify that the application behaves as expected.
Cypress Runner:
The Cypress Runner executes the test scripts. It interacts with the web application, capturing screenshots and videos during the tests.
Proxy Server:
A proxy server sits between the Cypress Runner and the web application. It intercepts requests and responses, allowing developers to manipulate them.
Node.js:
Cypress runs on Node.js, providing a runtime environment for executing JavaScript or TypeScript code.
WebSocket:
The WebSocket protocol enables real-time communication between the Cypress Runner and the web application.
HTTP Requests/Responses:
HTTP requests (e.g., GET, POST) and responses are exchanged between the Cypress Runner and the application server, facilitating the testing process.
By understanding these components and their interactions, you can better appreciate how Cypress effectively automates testing for modern web applications and UI Automation.
Features of the Cypress
Time Travel: Cypress captures snapshots of your application as it runs, allowing you to hover over each command in the test runner to see what happened at every step.
Real-Time Reloads: Cypress automatically reloads tests in real-time as you make changes, providing instant feedback on your changes without restarting your test suite.
Debuggability: Cypress provides detailed error messages and stack traces, making it easier to debug failed tests. It also allows you to use browser developer tools for debugging purposes.
Automatic Waiting: Cypress automatically waits for commands and assertions before moving on, eliminating the need for explicit waits or sleeps in your test code.
Spies, Stubs, and Clocks: Cypress provides built-in support for spies, stubs, and clocks to verify and control the behavior of functions, timers, and other application features.
Network Traffic Control: Cypress allows you to control and stub network traffic, making it easier to test how your application behaves under various network conditions.
Consistent Results: Cypress runs in the same run-loop as your application, ensuring that tests produce consistent results without flaky behavior.
Cross-Browser Testing: Cypress supports testing across multiple browsers, including Chrome, Firefox, and Edge, ensuring your application works consistently across different environments.
CI/CD Integration: Cypress integrates seamlessly with continuous integration and continuous deployment (CI/CD) pipelines, enabling automated testing as part of your development workflow.
Advantages of Cypress
Easy Setup and Configuration: Cypress offers a simple setup process with minimal configuration, allowing you to start writing tests quickly without dealing with complex setup procedures.
Developer-Friendly: Cypress is designed with developers in mind, providing an intuitive API and detailed documentation that makes it easy to write and maintain tests.
Fast Test Execution: Cypress runs directly in the browser, resulting in faster test execution compared to traditional testing frameworks that operate outside the browser.
Reliable and Flake-Free: Cypress eliminates common sources of flakiness in tests by running in the same run-loop as your application, ensuring consistent and reliable test results.
Comprehensive Testing: Cypress supports a wide range of testing types, including end-to-end (E2E), integration, and unit tests, providing a comprehensive solution for testing web applications.
Rich Ecosystem: Cypress has a rich ecosystem of plugins and extensions that enhance its functionality and allow you to customize your testing setup to suit your specific needs.
Active Community and Support: Cypress has an active and growing community that provides support, shares best practices, and contributes to the development of the framework.
Seamless CI/CD Integration: Cypress integrates seamlessly with CI/CD pipelines, enabling automated testing as part of your development workflow. This integration ensures that tests are run consistently and reliably in different environments, improving the overall quality of your software.
Cypress’s unique features, reliability, and ease of use make it an ideal choice for developers and testers looking to ensure the quality and performance of their web applications.
By leveraging Cypress in your JavaScript projects, you can achieve efficient and effective UI automation, enhancing the overall development lifecycle.
Cypress Framework Structure
In a Cypress project, the folder structure is well-defined to help you organize your test code, configuration, plugins, and related files. Here’s a breakdown of the typical folders and files we will encounter:
1. cypress/ Directory
Purpose: This is the root directory where all Cypress-related files and folders reside.
2. cypress/e2e/ Directory
Purpose: This is where you should place your test files.
Details: Cypress automatically detects and runs tests from this folder. Test files typically have .spec.js or .test.js file extensions.
3. cypress/fixtures/ Directory (Optional)
Purpose: Store static data or fixture files that your tests might need.
Details: These can include JSON, CSV, or text files.
4. cypress/plugins/ Directory (Optional)
Purpose: Extend Cypress’s functionality.
Details: Write custom plugins or modify Cypress behavior through plugins.
5. cypress/support/ Directory (Optional)
Purpose: Store various support files, including custom commands and global variables.
Details:
commands.js (Optional): Define custom Cypress commands here to encapsulate frequently used sequences of actions, making your test code more concise and maintainable.
e2e.js (Optional): Include global setup and teardown code for your Cypress tests. This file runs before and after all test files, allowing you to perform tasks like setting up test data or cleaning up resources.
6. cypress.config.js File
Purpose: Customize settings for Cypress, such as the base URL, browser options, and other configurations.
Location: Usually found in the root directory of your Cypress project.
Details: You can create this file manually if it doesn’t exist or generate it using the Cypress Test Runner’s settings.
7. node_modules/ Directory
Purpose: Contains all the Node.js packages and dependencies used by Cypress and your project.
Details: Usually, you don’t need to change anything in this folder.
8. package.json File
Purpose: Defines your project’s metadata and dependencies.
Details: Used to manage Node.js packages and scripts for running Cypress tests.
9. package-lock.json File
Purpose: Ensures your project dependencies remain consistent across different environments.
Details: Automatically generated and used by Node.js’s package manager, npm.
10. README.md File (Optional)
Purpose: Include documentation, instructions, or information about your Cypress project.
11. Other Files and Folders (Project-Specific)
Purpose: Depending on your project’s needs, you may have additional files or folders for application code, test data, reports, or CI/CD configurations.
Folder Structure Overview
The folder structure is designed to keep your Cypress project organized and easy to maintain:
Main Directories:
cypress/e2e/: Where you write your tests.
cypress.config.js: Where you configure Cypress.
Optional Directories:
fixtures/: For test data.
plugins/: For extending Cypress functionality.
support/: For custom commands and utilities.
This structure helps you customize your testing environment and keep everything well-organized.
Now let’s start to install Cypress and Configure in our project.
Cypress Install and Configuration:
We’re now ready to dive into the Cypress installation and configuration process. With Node.js, VS Code, and a new project named “CypressJavaScriptFramework” set up, let’s walk through configuring Cypress step-by-step.
Open Your Project: Start by opening the “CypressJavaScriptFramework” project in VS Code.
Open a New Terminal: From the top-left corner of VS Code, open a new terminal.
Initialize Node.js Project: Verify your directory path and run the below command to initialize a new Node.js project and generate a package.json file.
npm init –y
Install Cypress: Install Cypress as a development dependency with the below command. Once installed, you’ll find Cypress listed in your package.json file. As of this writing, the latest version is 13.13.1.
npm install –save-dev cypress
Configure Cypress: To open the Cypress Test Runner, run the below command.
npx cypress open
Upon first launch, you’ll be greeted by Launchpad, which helps with initial setup and configuration.
Step 1: Choosing a Testing Type
The first decision we will make in the Launchpad is selecting the type of testing you want to perform:
E2E (End-to-End) Testing: This option runs your entire application and visits pages to test them comprehensively.
Component Testing: This option allows you to mount and test individual components of your app in isolation.
Here we must select E2E Testing.
What is E2E Testing?
End-to-End (E2E) testing is a method of testing that validates the functionality and performance of an application by simulating real user scenarios from start to end. This approach ensures that all components of the application, including the frontend and backend, work together seamlessly.
After selecting E2E Testing Configuration Screen where we just have to click on Continue button.
Step 2: Quick Configuration
Next, the Launchpad will automatically generate a set of configuration files tailored to your chosen testing type. You’ll see a list of these changes, which you can review before continuing. For detailed information about the generated configuration, you can check out the Cypress configuration reference page.
After clicking on Continue button we will notice some changes in the framework few Configuration files will be added in the Framework which are cypress.config.js cypress/ directory cypress directory Fixtures and Support directory
The description of these file s and folders we have seen in start of blog.
Step 3: Launching a Browser
Finally, the Launchpad will display a list of compatible browsers detected on your system. You can select any browser to start your testing. Don’t worry if you want to switch browsers later; Cypress allows you to change browsers at any point of time.
As in my system I have Chrome and Edge browser installed. Cypress also have the inbuild browser which is called as “Electron”
What is Electron Browser?
Electron is an open-source framework that allows developers to build cross-platform desktop applications using web technologies like HTML, CSS, and JavaScript. It combines the Chromium rendering engine and the Node.js runtime, enabling you to create desktop apps that function seamlessly across Windows, macOS, and Linux.
Key Points:
Cross-Platform Compatibility: Develop applications that work on Windows, macOS, and Linux.
Chromium-Based: Uses Chromium, the same rendering engine behind Google Chrome, for a consistent browsing experience.
Node.js Integration: Allows access to native OS functionalities via Node.js, blending web technologies with desktop capabilities.
Used by Popular Apps: Many well-known applications like Slack, Visual Studio Code, and GitHub Desktop are built using Electron.
Electron provides the flexibility to build powerful desktop applications with the familiarity and ease of web development.
Now, you’re ready to hit the start button and embark on your testing journey with Cypress!
In this article we will use chrome browser for that we have to select Chrome and click on “Start E2E Testing in Chrome”. Then we will land on Cypress runner screen here we have 2 options
Scaffold example specs: Automatically generate example test specifications to help you get started with Cypress.
Create new specs: Manually create new test specifications to tailor your testing needs and scenarios.
Here we will use Scaffold example specs.
Scaffolding Example Specs
Use: Scaffolding example specs in Cypress generates predefined test files that demonstrate how to write and structure tests.
Reason: Providing example specs helps new users quickly understand Cypress’s syntax and best practices, making it easier to start writing their own tests and ensuring they follow a proper testing framework.
Once we select Scaffold example specs option, we will notice in framework few files are added in cypress directory under e2e directory.
Finally, we have installed cypress, configured and now we can run Scaffolding Example Specs. Now we will add our own file and execute it with cypress runner and from Command Line. Before that we will go through the Cypress Testing components.
Cypress Testing Components
Let’s understand Cypress Testing Components used while automation.
describe() Block: Groups related tests and provides structure.
it() Blocks: Defines individual test cases, focusing on specific functionalities.
Hooks: Manage setup and teardown processes to maintain a consistent test environment.
Assertions: Verify that the application behaves as expected by comparing actual results to expected results.
describe() Block
The describe() block in Cypress is used to group related test cases together. It defines a test suite, making it easier to organize and manage your tests.
Purpose:
The describe() block provides a structure for your test cases, allowing you to group tests that are related to a particular feature or functionality. It helps in maintaining a clean and organized test suite, especially as your test cases grow in number.
Example:
describe('Login Functionality', () => {
// Nested describe block for more granular organization
describe('Valid Login', () => {
it('should log in successfully with valid credentials', () => {
// Valid Login Script
});
});
describe('Invalid Login', () => {
it('should display an error message with invalid credentials', () => {
// Invalid Login Script
});
});
});
it() Blocks
The it() block defines individual test cases within a describe() block. It contains the actual code for testing a specific aspect of the feature under test.
Purpose:
Each it() block should test a single functionality or scenario, making your test cases clear and focused. This helps in identifying issues quickly and understanding what each test is verifying.
Example:
describe('Form Submission', () => {
it('should successfully submit the form and show a success message', () => {
// Form Submission Script
});
});
Hooks
Hooks are special functions in Cypress that run before or after tests. They are used to set up or clean up the state and perform common tasks needed for your tests.
Types of Hooks:
before(): Executes once before all tests in a describe() block.
beforeEach(): Runs before each it() block within a describe() block.
after(): Executes once after all tests in a describe() block.
afterEach(): Runs after each it() block within a describe() block.
Purpose:
Hooks are useful for setting up test environments, preparing data, and cleaning up after tests, ensuring a consistent and reliable test environment.
Example:
describe('User Registration', () => {
before(() => {
// Runs once before all tests
});
beforeEach(() => {
// Runs before each test
});
afterEach(() => {
// Runs after each test
});
after(() => {
// Runs once after all tests
});
it('Valid Login', () => {
// Valid Login Script
});
Assertions
Assertions are statements that check if a condition is true during the test execution. They verify that the application behaves as expected and helps identify issues when the actual results differ from the expected results.
Purpose:
Assertions validate the outcomes of your test cases by comparing actual results against expected results. They help ensure that your application functions correctly and meets the defined requirements.
Example:
describe('Homepage Content', () => {
it('should display the correct page title', () => {
cy.visit('/');
cy.title().should('equal', 'Expected Page Title');
});
it('should have a visible welcome message', () => {
cy.visit('/');
cy.get('.welcome-message').should('be.visible');
cy.get('.welcome-message').should('contain', 'Welcome to our website!');
});
});
These components work together to create a comprehensive and organized test suite in Cypress, ensuring your application is thoroughly tested and reliable.
Create Test File
Before diving into test file creation, let’s define the functionalities. We will automate the Calculator.net web application and will focus on basic arithmetic operations: addition, subtraction, multiplication, and division.
Here’s a breakdown of the test scenarios:
1. Verify user able to do addition
Visit Calculator.net
Click on two numbers (e.g., 1 and 2)
Click the “+” operator
Click on another number (e.g., 1)
Click the “=” operator
Verify the result is equal to 3
Click the “reset” button
2. Verify user able to do Subtraction
Visit Calculator.net
Click on a number (e.g., 3)
Click the “-” operator
Click on another number (e.g., 1)
Click the “=” operator
Verify the result is equal to 2
Click the “reset” button
3. Verify user able to do Multiplication
Visit Calculator.net
Click on a number (e.g., 2)
Click the “*” operator
Click on another number (e.g., 5)
Click the “=” operator
Verify the result is equal to 10
Click the “reset” button
4. Verify user able to do Division
Visit Calculator.net
Click on a number (e.g., 8)
Click the “/” operator
Click on another number (e.g., 2)
Click the “=” operator
Verify the result is equal to 4
Click the “reset” button
Optimizing with Hooks:
As you noticed, visiting Calculator.net and resetting the calculator are common steps across all scenarios. To avoid code repetition, we’ll utilize Cypress hooks:
beforeEach: Execute this code before each test case. We’ll use it to visit Calculator.net.
afterEach: Execute this code after each test case. We’ll use it to reset the calculator.
Now, let’s create the test file and add the code below to Calculator.cy.js file.
/// <reference types="cypress" />
import selectors from '../fixtures/Selectors.json';
describe('Calculator Tests', () => {
before(() => {
cy.log('Tests are starting...');
});
beforeEach(() => {
cy.visit('https://www.calculator.net');
});
afterEach(() => {
cy.get(selectors.cancelButton).click();
});
after(() => {
cy.log('All tests are finished.');
});
it('Verify user able to do addition', () => {
cy.get(selectors.twoNumberButton).click();
cy.get(selectors.plusOperatorButton).click();
cy.get(selectors.oneNumberButton).click();
cy.get(selectors.equalsOperatorButton).click();
cy.get(selectors.result).should('contain.text', '3');
});
it('Verify user able to do Subtraction', () => {
cy.get(selectors.threeNumberButton).click();
cy.get(selectors.minusOperatorButton).click();
cy.get(selectors.oneNumberButton).click();
cy.get(selectors.equalsOperatorButton).click();
cy.get(selectors.result).should('contain.text', '2');
});
it('Verify user able to do Multiplication', () => {
cy.get(selectors.twoNumberButton).click();
cy.get(selectors.multiplyOperatorButton).click();
cy.get(selectors.fiveNumberButton).click();
cy.get(selectors.equalsOperatorButton).click();
cy.get(selectors.result).should('contain.text', '10');
});
it('Verify user able to do Division', () => {
cy.get(selectors.eightNumberButton).click();
cy.get(selectors.divideOperatorButton).click();
cy.get(selectors.twoNumberButton).click();
cy.get(selectors.equalsOperatorButton).click();
cy.get(selectors.result).should('contain.text', '4');
});
});
Let’s create a Selectors.json file to store all the selectors used in automation, assigning them meaningful names for better organization.
The Selector.json file is a crucial part of your test automation framework. It centralizes all the CSS selectors used in your tests, making the code more maintainable and readable. By keeping selectors in a dedicated file, you can easily update or change any element locator without modifying multiple test scripts.
Purpose:
Centralization: All element selectors are stored in one place.
Maintainability: Easy to update selectors if the application’s HTML changes.
Readability: Makes test scripts cleaner and easier to understand by abstracting the actual CSS selectors.
Add the following JSON content to your Selector.json file in the cypress/fixtures directory:
Number Buttons: Selectors for the number buttons (0-9) use the span[onclick=’r(number)’] pattern, identifying the buttons by their onclick attribute values specific to each number.
Operator Buttons: Selectors for the arithmetic operators (plus, minus, multiply, divide) use a similar pattern but include escaped quotes for the operator characters.
Equals Button: The selector for the equals button follows the same pattern, identifying it by its onclick attribute.
Result: The selector for the result display uses an ID (#sciOutPut), directly identifying the output element.
Cancel Button: The selector for the cancel button is included to reset the calculator between tests, ensuring a clean state for each test case.
By utilizing this Selector.json file, your test scripts can reference these selectors with meaningful names, thereby enhancing the clarity and maintainability of your test automation framework for UI.
Advanced Configuration In cypress.config.js:
While installing and Configration of cypress we have created cypress.config.js file. Now we will look at the Advanced configuration in cypress.config.js allows you to tailor Cypress’s behavior to fit the specific needs of your project, optimizing and enhancing the testing process.
Key Benefits:
Customization: You can set up custom configurations to suit your testing environment, such as base URL, default timeouts, viewport size, and more.
Environment Variables: Manage different environment settings, making it easy to switch between development, staging, and production environments.
Plugin Integration: Configure plugins for extended functionality, such as code coverage, visual testing, or integrating with other tools and services.
Reporter Configuration: Customize the output format of your test results, making it easier to integrate with CI/CD pipelines and other reporting tools.
Browser Configuration: Define which browsers to use for testing, including headless mode, to speed up the execution of tests.
Test Execution Control: Set up retries for flaky tests, control the order of test execution, and manage parallel test runs for better resource utilization.
Security: Configure authentication tokens, manage sensitive data securely, and control network requests and responses to mimic real-world scenarios.
This Cypress configuration file (cypress.config.js) sets various options to customize the behavior of Cypress tests. Here’s a breakdown of the configuration for modern UI Automation:
const { defineConfig } = require(“cypress”);: Import defineConfig function from Cypress, which is used to define configuration settings.
module.exports = defineConfig({ … });: Exports the configuration object, which Cypress uses to configure the test environment.
projectId: “CYFW01”: Specifies a unique project ID for identifying the test project. This is useful for organizing and managing tests in CI/CD pipeline.
downloadsFolder: “cypress/downloads”: Sets the folder where files downloaded during tests will be saved.
screenshotsFolder: “cypress/screenshots”: Defines the folder where screenshots taken during tests will be stored, particularly for failed tests.
video: true: Enables video recording of test runs, which can be useful for reviewing test execution and debugging.
screenshotOnRunFailure: true: Configures Cypress to take screenshots automatically when test fails.
chromeWebSecurity: false: Disables web security in Chrome, which can be useful for testing applications that involve cross-origin requests.
trashAssetsBeforeRuns: true: Ensures that previous test artifacts (like screenshots and videos) are deleted before running new tests, keeping the test environment clean.
viewportWidth: 1920 and viewportHeight: 1080: To simulate a screen resolution of 1920×1080 pixels, you can set the default viewport size for tests accordingly.
execTimeout: 10000: Configures the maximum time (in milliseconds) Cypress will wait for commands to execute before timing out.
pageLoadTimeout: 18000: Sets the maximum time (in milliseconds) Cypress will wait for a page to load before timing out.
defaultCommandTimeout: 10000: Defines the default time (in milliseconds) Cypress will wait for commands to complete before timing out.
retries:{ runMode: 1, openMode: 0 }:
runMode: 1: Specifies that Cypress should retry failed tests once when running in CI/CD mode (runMode).
openMode: 0: Indicates that Cypress should not retry failed tests when running interactively (openMode).
e2e: { setupNodeEvents(on, config) { … } }: Provide way to set-up Node.js event listeners for end-to-end tests. This is where you can implement custom logic or plugins to extend Cypress’s functionality.
Executing Test Cases Locally for Modern UI Automation
To run test cases for modern UI Automation, use Cypress commands in your terminal. Cypress supports both headed mode (with a visible browser window) and headless mode (where tests run in the background without displaying a browser window).
Running Test Cases in Headed Mode:
Open your terminal.
Navigate to the directory containing your Cypress tests.
Execute the tests in headed mode using the below command:
npx cypress open
This will open the Cypress Test Runner. Click on “E2E Testing,” select the browser, and run the test case from the list (e.g., calculator.cy.js). Once selected, the test case will execute, and you can see the results in real-time. Screenshots of the local test execution are provided below.
Running Test Cases in Headless Mode:
Headless mode in Cypress refers to running test cases without a visible user interface. This method allows tests to be executed entirely in the background. Here’s how you can set up and run Cypress in headless mode.
To run the test script directly from the command line, use the following command:
npx cypress run –spec “cypress\e2e\Calculator.cy.js” –browser edge
By default, Cypress executes tests in headless mode, but you can also specify it explicitly using the –headless flag:
npx cypress run — headless –spec “cypress\e2e\Calculator.cy.js” –browser edge
This enables efficient and automated test execution without launching the browser UI (UI Automation).
Conclusion
In this blog, we explored how the JavaScript and Cypress framework revolutionize modern UI automation. By leveraging Cypress’s powerful features, such as its intuitive API, robust configuration options, and seamless integration with JavaScript, we were able to effectively test complex web applications.
We delved into practical implementations of modern UI automation such as:
Creating and managing test cases with Cypress, including various operations like addition, subtraction, multiplication, and division using a calculator example.
Using advanced configuration in cypress.config.js to tailor the test environment to specific needs, from handling different environments and customizing timeouts to integrating plugins and managing network requests.
Implementing selectors through a Selector.json file to enhance test maintainability and clarity by using descriptive names for elements.
Executing tests locally in both headed and headless modes, providing insights into how to monitor test execution in real-time or run tests in the background.
By incorporating these strategies, we ensure that our web applications not only function correctly but also provide a seamless and reliable user experience. Cypress’s modern approach to UI testing simplifies the automation process, making it easier to handle the dynamic nature of contemporary web applications while maintaining high standards of quality and performance.
I am an SDET Engineer proficient in manual, automation, API, Performance, and Security Testing. My expertise extends to technologies such as Selenium, Cypress, Cucumber, JMeter, OWASP ZAP, Postman, Maven, SQL, GitHub, Java, JavaScript, HTML, and CSS. Additionally, I possess hands-on experience in CI/CD, utilizing GitHub for continuous integration and delivery. My passion for technology drives me to constantly explore and adapt to new advancements in the field.
KPIs for Test Automation are measurable criteria that demonstrate how effectively the automation testing process supports the organization’s objectives. These metrics assess the success of automation efforts and specific activities within the testing domain. KPIs for test automation are crucial for monitoring progress toward quality goals, evaluating testing efficiency over time, and guiding decisions based on data-driven insights. They encompass metrics tailored to ensure thorough testing coverage, defect detection rates, testing cycle times, and other critical aspects of testing effectiveness.
Importance of KPIs
Performance Measurement: Key performance indicators (KPIs) offer measurable metrics to gauge the performance and effectiveness of automated testing efforts. They monitor parameters such as test execution times, test coverage, and defect detection rates, providing insights into the overall efficacy of the testing process KPIs will help your team improve testing skills
Identifying Challenges and Problems: Key performance indicators (KPIs) assist in pinpointing bottlenecks or challenges within the test automation framework. By monitoring metrics such as test error rates, script consistency, and resource allocation, KPIs illuminate areas needing focus or enhancement to improve the dependability and scalability of automated testing.
Optimizing Resource Utilization: Key performance indicators (KPIs) facilitate improved allocation of resources by pinpointing areas where automated efforts are highly effective and where manual intervention might be required. This strategic optimization aids in maximizing the utilization of testing resources and minimizing costs associated with testing activities.
Facilitating Ongoing Enhancement: Key performance indicators (KPIs) support continual improvement by establishing benchmarks and objectives for testing teams. They motivate teams to pursue elevated standards in automation scope, precision, and dependability, fostering a culture of perpetual learning and refinement of testing proficiency.
Benefits of KPIs:
Test Coverage clear objective: KPIs will help an unbiased view of the effectiveness of automation testing you with the help
Process Enhancement: KPIs highlight the areas for improvement while doing automation testing processes. So you can achieve continuous enhancement & efficiency.
Executive Insight: Sharing KPIs with the team will have transparency & a better understanding of what test automation can achieve
Process Tracking: Regular monitoring of KPIs tracks the status and progress of automated testing, ensuring alignment with goals and timelines.
KPIs For Test Automation:
1. Test Coverage:
Description: Test coverage refers to the proportion of your application code that is tested. It ensures that your automated testing encompasses all key features and functions. Achieving high test coverage is crucial for reducing the risk of defects reaching production and can also reduce manual efforts.
Examples of Measurements:
Requirements Traceability Matrix (RTM): Maps test cases to requirements to ensure that all requirements are covered by tests.
User Story Coverage: Measures the percentage of user stories that have been tested.
Description: This performance metric gauges the time required to run a test suite. Effective automation testing, indicated by shorter execution times, is critical for the deployment of software in a DevOps setting. Efficient test execution supports seamless continuous integration and continuous delivery (CI/CD) workflows, ensuring prompt software releases and updates.
Examples of Measurements:
Total Test Execution Time: Total time taken to execute all test cases in a test suite.
Average Execution Time per Test Case: Average time taken to execute an individual test case.
Description: This metric in automation measures the percentage of test cases that fail during a specific build or over a set period. It is determined by dividing the number of failed tests by the total number of tests executed and multiplying the result by 100 to express it as a percentage. Tracking this rate helps identify problematic areas in the code or test environment, facilitating timely fixes and enhancing overall software quality. Maintaining a low failure rate is essential for ensuring the stability and reliability of the application throughout the testing lifecycle.
Examples of Measurements:
Failure Rate Per Build: Percentage of test cases that fail in each build.
Historical Failure Trends: Trends in test failure rates over time.
Description: Active defects represent the present state of issues, encompassing new, open, or resolved defects, guiding the team in determining appropriate resolutions. The team sets a threshold for monitoring these defects, taking immediate action on those that surpass this limit.
Examples of Measurements:
Defect Count: Number of active defects at any given time.
Defect Aging: Time taken to resolve defects from the time they were identified.
Tools to Measure Active Defects:
Defect Tracking Tools: Jira, Bugzilla, HP ALM
Test Management Tools: TestRail, Zephyr, QTest
5. Build Stability:
Description: Build stability in automation helps measure the reliability and consistency of application builds. You can check how frequently builds pass or fail during automation. Monitoring build stability helps your team identify failures early, and maintaining build stability is necessary for continuous delivery (CI/CD) workflows.
Examples of Measurements:
Pass/Fail Rate: Percentage of builds that pass versus those that fail.
Mean Time to Recovery (MTTR): Average time taken to fix a failed build.
Description: Defect density measures the number of defects found in a module or piece of code per unit size (e.g., lines of code, function points). It helps in identifying areas of the code that are more prone to defects.
Examples of Measurements:
Defects per KLOC (Thousand Lines of Code): Number of defects found per thousand lines of code.
Defects per Function Point: Number of defects found per function point.
Description: Test case effectiveness measures how well the test cases are able to detect defects. It is calculated by the number of defects detected divided by the total number of defects.
Examples of Measurements:
Defects Detected by Tests: Number of defects detected by automated tests.
Total Defects: Total number of defects detected including those found in production.
Tools to Measure Test Case Effectiveness:
Test Management Tools: TestRail, Zephyr, QTest
Defect Tracking Tools: Jira, Bugzilla, HP ALM
8. Test Automation ROI (Return on Investment):
Description: This KPI measures the financial benefit gained from automation versus the cost incurred to implement and maintain it. It helps in justifying the investment in test automation.
Examples of Measurements:
Cost Savings from Reduced Manual Testing: Savings from reduced manual testing efforts.
Automation Implementation Costs: Costs incurred in implementing and maintaining automation.
Tools to Measure Test Automation ROI:
Project Management Tools: MS Project, Smartsheet, Asana
Test Management Tools: TestRail, Zephyr, QTest
9. Test Case Reusability:
Description: This metric measures the extent to which test cases can be reused across different projects or modules. Higher reusability indicates efficient and modular test case design.
Examples of Measurements:
Reusable Test Cases: Number of test cases reused in multiple projects.
Total Test Cases: Total number of test cases created.
Description: Defect leakage measures the number of defects that escape to production after testing. Lower defect leakage indicates more effective testing.
Examples of Measurements:
Defects Found in Production: Number of defects found in production.
Total Defects Found During Testing: Total number of defects found during testing phases.
Description: This KPI measures the effort required to maintain and update automated tests. Lower maintenance effort indicates more robust and adaptable test scripts.
Examples of Measurements:
Time Spent on Test Maintenance: Total time spent on maintaining and updating test scripts.
Number of Test Scripts Updated: Number of test scripts that required updates.
Tools to Measure Automation Test Maintenance Effort:
Key Performance Indicators (KPIs) are crucial for ensuring the quality and reliability of applications. Metrics like test coverage, test execution time, test failure rate, active defects, and build stability offer valuable insights into the testing process. By following these KPIs, teams can detect defects early and uphold high software quality standards. Implementing and monitoring these metrics supports effective development cycles and facilitates seamless integration and delivery in CI/CD workflows.
Click here for more blogs on software testing and test automation.
As a Junior SDET with 2 years of hands-on experience, I specialize in both manual and automation testing for web and mobile applications. I have worked with a variety of technologies, including Selenium, Playwright, Cucumber, Appium, SQL, Java, JavaScript, and Python, to deliver comprehensive test solutions. My expertise covers both functional and regression testing, with a focus on ensuring quality across different platforms.
This blog explores how we can use AI capabilities to automate our test case generation tasks for web applications and APIs, focusing on AI-assisted Test Case Generation for Web & API. Before diving into this topic, let’s first understand why automating test case generation is important. But before that, let’s clarify what a test case is: a test case is a set of steps or conditions used by a tester or developer to verify and validate whether a software application meets customer and business requirements. Now that we understand what a test case is, let’s explore why we create them.
What is the need for test case creation?
To ensure quality: Test cases help identify defects and ensure the software meets requirements.
To improve efficiency: Well-structured test cases streamline the testing process.
To facilitate regression testing: You can reuse test cases to verify that new changes haven’t introduced defects.
To improve communication: Test cases serve as a common language between developers and testers.
To measure test coverage: Test cases help assess the extent to which the software has been tested.
When it comes to manual test case creation some limitations, disadvantages, or challenges impact the efficiency and effectiveness of the testing process such as:
What are the limitations of manual test case generation?
Time-Consuming: Manual test case writing is a time-consuming process as each test case requires detailed planning and documentation to ensure the coverage of requirements and expected output.
Resource Intensive: Creating manual test cases requires significant resources and skilled personnel. Testers must thoroughly understand the application and its related requirements to write effective test cases. This process demands a substantial allocation of human resources, which could be better utilized in other critical areas.
Human Error: Any task that needs human interactions is prone to error because that is a human tendency and manual test case creation is no exception. Mistakes can occur in documenting the steps, and expected results, or even in understanding the requirements. Which could result in inaccurate test cases that lead to undetected bugs and defects.
Expertise Dependency: Creating high-quality test cases that cover all the requirements and results into high test coverage requires a certain level of expertise and domain knowledge. This creates a limitation especially if those individuals are unavailable or if there is a high turnover rate.
These are just some of the challenges that I have mentioned but there could be more. Comment down your thoughts on this one. If you have any other challenges then you can share them in the comment section. Now that we have understood why we create a test case and what value it adds in testing along with the limitations for manual test case generation let’s see what are the benefits of automating this test case generation process.
Benefits of automated test case generation:
Efficiency and Speed: Automated test case generation significantly improves the efficiency and speed of test case writing. As tools and algorithms drive the process instead of manual efforts, it creates test cases faster and quickly updates them whenever there are changes in the application, ensuring that testing keeps pace with development.
Increased Test Coverage: Automated test case generation eliminates or reduces the chances of compromising the test coverage. This process generates a wide range of test cases, including those that manual testing might overlook. By covering various scenarios, such as edge cases, it ensures thorough testing.
Accuracy and Consistency: Automating test case generation ensures accurate and consistent creation of test cases every time. This consistency is crucial for maintaining the integrity of the testing process and applying the same standards across all test cases.
Improved Collaboration: By standardizing the test case generation process, automated test case generation promotes improved collaboration among cross-functional teams. It ensures that all team members, including developers, testers, and business analysts, are on the same page.
Again, these are just a few advantages that I have listed down. You can share more in the comment section and let me know what the limitations of automated test case generation are as well.
Before we move ahead it is essential to understand what is AI and how it works. This understanding of AI will help us to design and build our algorithms and tools to get the desired output.
What is AI?
AI (Artificial Intelligence) simulates human intelligence in machines, programming them to think, learn, and make decisions. AI systems mimic cognitive functions such as learning, reasoning, problem-solving, perception, and language understanding.
How does AI work?
AI applications work based on a combination of algorithms, computational models, and large datasets. We divide this process into several steps as follows.
1. Data Collection and Preparation:
Data Collection: AI system requires vast amounts of data to learn from. You can collect this data from various sources such as sensors, databases, and user interactions.
Data Preparation: We clean, organize, and format the collected data to make it suitable for training AI models. This step often involves removing errors, handling missing values, and normalizing the data.
2. Algorithm Selection:
Machine Learning (ML): Algorithms learn from data and improve over time without explicit programming. Examples include decision trees, support vector machines, and neural networks.
Deep Learning: A subset of machine learning that uses neural networks with many layers (deep neural networks) to analyze complex patterns in data. It is particularly effective for tasks such as image and speech recognition.
3. Model Training:
Training: During training, the AI model learns to make predictions or decisions by analyzing the training data. The model adjusts its parameters to minimize errors and improve accuracy.
Validation: We test the model on a separate validation dataset to evaluate its performance and fine-tune its parameters.
4. Model Deployment:
Once the team trains and validates the AI model, they deploy it to perform its intended tasks in a real-world environment. This could involve making predictions, classifying data, or automating processes.
5. Inference and Decision-Making:
Inference is the process of using the trained AI model to make decisions or predictions based on new, unseen data. The AI system applies the learned patterns and knowledge to provide outputs or take actions.
6. Feedback and Iteration:
AI systems continuously improve through feedback loops. By analyzing the outcomes of their decisions and learning from new data, AI models can refine their performance over time. This iterative process helps in adapting to changing environments and evolving requirements.
Note: We are using Open AI to automate the test case generation process. For this, you need to create an API key for your Open AI account. Check this Open AI API page for more details.
Automated Test Case Generation for Web:
Prerequisite:
Open AI account and API key
Node.js installed on the system
Approach:
For web test case generation using AI the approach I have followed is to scan the DOM structure of the web page analyze the tag and attribute present and then use this as input data to generate the test case.
Step 1: Web Scrapping
Web scrapping will provide us the DOM structure information of the web page. We will store this and then pass this to the next process which is analyzing this stored DOM structure.
Install Puppeteer npm package using npm i puppeteer We are using Puppeteer to launch the browser and visit the web page.
Next, we have an async function scrapeWebPage This function requires the web URL. Once you pass the web URL then it stores the tags and attributes from the DOM content.
This function will return the structure and at last will return the web elements.
Step 2: Analyze elements
In this step, we are analyzing the elements that we got from our first step and based on that we will define what action to take on those elements.
function analyzePageStructure(pageStructure) {
const actions = [];
pageStructure.forEach(element => {
const { tagName, attributes } = element;
if (tagName === 'input' && (attributes.includes('type="text"') || attributes.includes('type="password"'))) {
actions.push(`Fill in the ${tagName} field`);
} else if (tagName === 'button' && attributes.includes('type="submit"')) {
actions.push('Click the submit button');
}
});
console.log("Actions are: ", actions);
return actions;
}
module.exports = analyzePageStructure;
Code Explanation:
Here the function analyzePageStructure takes pageStrucure as a parameter, which is nothing but the elements that we got using web scraping.
We are declaring the action array here to store all the actions that we will define to perform.
In this particular code, I am only considering two types i.e. text and submit and tagNames i.e. input and button.
For type text and tag name input, I am adding an action to enter the data.
For type submit and tag name submit I am adding an action to click.
At last, this function will return the actions array.
Step 3: Generate Test Cases
This is the last step of this approach. Till here we have our actions and the elements as well. Now, we are ready to generate the test cases for the entered web page.
const axios = require('axios');
async function generateBddTestCases(actions, apiKey) {
const prompt = `
Generate BDD test cases using Gherkin syntax for the following login page actions: ${actions.join(', ')}. Include test cases for:
1. Functional Testing: Verify each function of the software application.
2. Boundary Testing: Test boundaries between partitions.
3. Equivalence Partitioning: Divide input data into valid and invalid partitions.
4. Error Guessing: Anticipate errors based on experience.
5. Performance Testing: Ensure the software performs well under expected workloads.
6. Security Testing: Identify vulnerabilities in the system.
`;
const headers = {
'Content-Type': 'application/json',
'Authorization': `Bearer ${apiKey}`
};
const data = {
model: 'gpt-3.5-turbo',
prompt,
max_tokens: 1000,
n: 1,
stop: ['\n'],
};
try {
const response = await axios.post('https://api.openai.com/v1/completions', data, { headers });
return response.data.choices[0].text.trim();
} catch (error) {
console.error('Error generating test cases:', error.response ? error.response.data : error.message);
return null;
}
}
module.exports = generateBddTestCases;
Code Explanation:
The function generateBddTestCases takes two parameters actions and apikey (Open AI API key)
We have added a prompt along with the actions and API key to generate the test cases.
The API used in the above code is provided by Open AI.
Output:
Feature: Login functionality
# Functional Testing
Scenario: Successful login with valid credentials Given the user is on the login page When the user fills in the username field with “user123” And the user fills in the password field with “password123” And the user clicks the submit button Then the user should be redirected to the dashboard
Scenario: Unsuccessful login with invalid credentials Given the user is on the login page When the user fills in the username field with “invalidUser” And the user fills in the password field with “invalidPass” And the user clicks the submit button Then an error message “Invalid username or password” should be displayed
# Boundary Testing
Scenario: Login with username at boundary length Given the user is on the login page When the user fills in the username field with “user12345678901234567890” # Assume max length is 20 And the user fills in the password field with “password123” And the user clicks the submit button Then the user should be redirected to the dashboard
Scenario: Login with password at boundary length Given the user is on the login page When the user fills in the username field with “user123” And the user fills in the password field with “password12345678901234567890” # Assume max length is 20 And the user clicks the submit button Then the user should be redirected to the dashboard
# Equivalence Partitioning
Scenario: Login with invalid username partition Given the user is on the login page When the user fills in the username field with “invalidUser!” And the user fills in the password field with “password123” And the user clicks the submit button Then an error message “Invalid username format” should be displayed
Scenario: Login with invalid password partition Given the user is on the login page When the user fills in the username field with “user123” And the user fills in the password field with “short” And the user clicks the submit button Then an error message “Password is too short” should be displayed
# Error Guessing
Scenario: Login with SQL injection attempt Given the user is on the login page When the user fills in the username field with “admin’–“ And the user fills in the password field with “password123” And the user clicks the submit button Then an error message “Invalid username or password” should be displayed And no unauthorized access should be granted
# Performance Testing
Scenario: Login under load Given the system is under heavy load When the user fills in the username field with “user123” And the user fills in the password field with “password123” And the user clicks the submit button Then the login should succeed within acceptable response time
# Security Testing
Scenario: Login with XSS attack Given the user is on the login page When the user fills in the username field with “<script>alert(‘XSS’)</script>” And the user fills in the password field with “password123” And the user clicks the submit button Then an error message “Invalid username format” should be displayed And no script should be executed
Automated Test Case Generation for API:
Approach:
To effectively achieve AI Test Case Generation for APIs, we start by passing the endpoint and the URI. Subsequently, we attach files containing the payload and the expected response. With these parameters in place, we can then leverage AI, specifically OpenAI, to generate the necessary test cases for the API.
Step 1: Storing the payload and expected response json files in the resources folder
We are going to use the POST API for this and for POST APIs we need payload.
The payload is passed through json file stored in the resources folder.
We also need to pass the expected response of this POST API so that we can create effective test cases.
The expected response json file will help us to create multiple test case to ensure maximum test coverage.
Step 2: Generate Test Cases
In this step, we will use the stored payload, and expected response json files along with the API endpoint.
const fs = require('fs');
const axios = require('axios');
// Step 1: Read JSON files
const readJsonFile = (filePath) => {
try {
return JSON.parse(fs.readFileSync(filePath, 'utf8'));
} catch (error) {
console.error(`Error reading JSON file at ${filePath}:`, error);
throw error;
}
};
const payloadPath = 'path_of_payload.json';
const expectedResultPath = 'path_of_expected_result.json';
const payload = readJsonFile(payloadPath);
const expectedResult = readJsonFile(expectedResultPath);
console.log("Payload:", payload);
console.log("Expected Result:", expectedResult);
// Step 2: Generate BDD Test Cases
const apiKey = 'your_api_key';
const apiUrl = 'https://reqres.in';
const endpoint = '/api/login';
const callType = 'POST';
const generateApiTestCases = async (apiUrl, endpoint, callType, payload, expectedResult, retries = 3) => {
const prompt = `
Generate BDD test cases using Gherkin syntax for the following API:
URL: ${apiUrl}${endpoint}
Call Type: ${callType}
Payload: ${JSON.stringify(payload)}
Expected Result: ${JSON.stringify(expectedResult)}
Include test cases for:
1. Functional Testing: Verify each function of the API.
2. Boundary Testing: Test boundaries for input values.
3. Equivalence Partitioning: Divide input data into valid and invalid partitions.
4. Error Guessing: Anticipate errors based on experience.
5. Performance Testing: Ensure the API performs well under expected workloads.
6. Security Testing: Identify vulnerabilities in the API.
`;
try {
const response = await axios.post('https://api.openai.com/v1/completions', {
model: 'gpt-3.5-turbo',
prompt: prompt,
max_tokens: 1000,
n: 1,
stop: ['\n'],
}, {
headers: {
'Authorization': `Bearer ${apiKey}`,
'Content-Type': 'application/json'
}
});
const bddTestCases = response.data.choices[0].text.trim();
// Check if bddTestCases is a valid string before writing to file
if (typeof bddTestCases === 'string') {
fs.writeFileSync('apiTestCases.txt', bddTestCases);
console.log("BDD test cases written to apiTestCases.txt");
} else {
throw new Error('Invalid data received for BDD test cases');
}
} catch (error) {
if (error.response && error.response.status === 429 && retries > 0) {
console.log('Rate limit exceeded, retrying...');
await new Promise(resolve => setTimeout(resolve, 2000)); // Wait for 2 seconds before retrying
return generateApiTestCases(apiUrl, endpoint, callType, payload, expectedResult, retries - 1);
} else {
console.error('Error generating test cases:', error.response ? error.response.data : error.message);
throw error;
}
}
};
generateApiTestCases(apiUrl, endpoint, callType, payload, expectedResult)
.catch(error => console.error('Error generating test cases:', error));
Code Explanation:
Firstly we are reading the two json files from the resources folder i.e. payload.json and expected_result.json
Next, use your API key, specify the API URL and endpoint along with callType
Write a prompt for generating the test cases.
Use the same Open AI API to generate the test cases.
Output:
Feature: Login API functionality
# Functional Testing
Scenario: Successful login with valid credentials Given the API endpoint is “https://reqres.in/api/login” When a POST request is made with payload:
“”” { “email”: “eve.holt@reqres.in”, “password”: “cityslicka” } “”” Then the response status should be 200 And the response should be: “”” { “token”: “QpwL5tke4Pnpja7X4” } “””
Scenario: Unsuccessful login with missing password Given the API endpoint is “https://reqres.in/api/login” When a POST request is made with payload:
“”” { “email”: “eve.holt@reqres.in” } “”” Then the response status should be 400 And the response should be: “”” { “error”: “Missing password” } “””
Scenario: Unsuccessful login with missing email Given the API endpoint is “https://reqres.in/api/login” When a POST request is made with payload:
“”” { “password”: “cityslicka” } “”” Then the response status should be 400 And the response should be: “”” { “error”: “Missing email” } “””
# Boundary Testing
Scenario: Login with email at boundary length Given the API endpoint is “https://reqres.in/api/login” When a POST request is made with payload:
“”” { “email”: “eve.holt@reqres.in.this.is.a.very.long.email.address”, “password”: “cityslicka” } “”” Then the response status should be 200 And the response should be: “”” { “token”: “QpwL5tke4Pnpja7X4” } “””
Scenario: Login with password at boundary length Given the API endpoint is “https://reqres.in/api/login” When a POST request is made with payload:
“”” { “email”: “eve.holt@reqres.in”, “password”: “thisisaverylongpasswordthatexceedstypicallength” } “”” Then the response status should be 200 And the response should be: “”” { “token”: “QpwL5tke4Pnpja7X4” } “””
# Equivalence Partitioning
Scenario: Login with invalid email format Given the API endpoint is “https://reqres.in/api/login” When a POST request is made with payload:
“”” { “email”: “eve.holt@reqres”, “password”: “cityslicka” } “”” Then the response status should be 400 And the response should be: “”” { “error”: “Invalid email format” } “””
Scenario: Login with invalid password partition Given the API endpoint is “https://reqres.in/api/login” When a POST request is made with payload:
“”” { “email”: “eve.holt@reqres.in”, “password”: “short” } “”” Then the response status should be 400 And the response should be: “”” { “error”: “Password is too short” } “””
# Error Guessing
Scenario: Login with SQL injection attempt Given the API endpoint is “https://reqres.in/api/login” When a POST request is made with payload:
“”” { “email”: “admin’–“, “password”: “cityslicka” } “”” Then the response status should be 401 And the response should be: “”” { “error”: “Invalid email or password” } “”” And no unauthorized access should be granted
# Performance Testing
Scenario: Login under load Given the API endpoint is “https://reqres.in/api/login” When the system is under heavy load And a POST request is made with payload:
“”” { “email”: “eve.holt@reqres.in”, “password”: “cityslicka” } “”” Then the response status should be 200 And the login should succeed within acceptable response time
# Security Testing
Scenario: Login with XSS attack in email Given the API endpoint is “https://reqres.in/api/login” When a POST request is made with payload:
“”” { “email”: “<script>alert(‘XSS’)</script>”, “password”: “cityslicka” } “”” Then the response status should be 400 And the response should be: “”” { “error”: “Invalid email format” } “”” And no script should be executed
Conclusion:
Automating test case generation using AI capabilities will help to ensure total test coverage. It will also enhance the process by addressing the limitations mentioned above of manual test case creation. The use of AI tools like Open AI significantly improves efficiency, increases test coverage, ensures accuracy, and promotes consistency.
The code implementation shared in this blog demonstrates a practical way to leverage OpenAI for automating AI Test Case Generation. I hope you find this information useful and encourage you to explore the benefits of AI in your testing processes. Feel free to share your thoughts and any additional challenges in the comments. Happy testing!
Click here for more blogs on software testing and test automation.
An SDET Engineer with hands-on experience in manual, automation, API, Performance, and Security Testing. The technologies I have worked on include Selenium, Playwright, Cypress, SpecFlow, Cucumber, JMeter, K6, OWASP ZAP, Appium, Postman, Maven, Behave, Pytest, BrowserStack, SQL, GitHub, Java, JavaScript, Python, C#, HTML, CSS. Also, I have hands-on experience in CI/CD the technologies I have used for CI/CD include Azure DevOps, AWS, and Github. Apart from this, I like to write blogs and explore new technologies.
ADB (Android Debug Bridge) is an invaluable tool for developers, testers and Android enthusiasts alike. It allows you to interact with your Android device from a computer, opening new possibilities for application development and usability improvement. Explore ADB commands for android device manipulation in this blog.
Android enthusiasts, developers and testers find ADB to be of great utility. It provides several features essential for managing devices and developing apps.
ADB uses a command-line interface (CLI) to function. This implies that instead of interacting with it through a graphical user interface (GUI), you do it by typing instructions into a terminal or command prompt.
ADB establishes a connection between your Android smartphone and PC. You can use your computer to control your gadget using this connection and use ADB Commands for Android Device interaction.
Below are few of the benefits of using the ADB commands for Android Device Control
The tool makes it possible to do a variety of interactions with your Android device, which can be useful for both everyday use, development and testing.
By enabling fast access to a device from a development environment, ADB helps developers to build and testers to test applications more efficiently.
By providing access to more complex system capabilities, ADB can assist users in personalizing their devices or troubleshooting problems.
ADB allows you to install apps straight from your development environment onto the device, saving you the trouble of manually moving them there. With the ability to perform diagnostic commands and retrieve error logs, it also aids in troubleshooting applications.
System logs (logcat), which are essential for debugging apps and comprehending system events, can be retrieved by ADB.
When you need to remotely manage a device or perform automated testing, it can mimic human activities like key presses and touch events.
We’ll go over the most crucial ADB commands in this tutorial, along with their advantages and uses and explore ADB command for Android Device.
But before you start using the ADB commands to manipulate the Android device, you’ll first need to set up your device and computer to enable communication.
Explore Following ADB Commands to manipulate Android Device
Device Connectivity
Getting comprehensive details on your connected device is one of the most basic things you can do with ADB. For developers and testers who must comprehend the context in which their apps are operating, this is essential which can be achieved by executing the subsequent ADB command:
adb devices
This command displays a list of every device that is either networked or USB-connected to your computer. It provides a list of device serial numbers and their status, which can be
1. “unauthorized” (meaning the device is connected but not authorized for ADB communication),
2. “offline” (meaning the device is connected but not ready for commands), or
3. “device” (meaning the device is connected and ready for commands). Making sure your device is correctly connected and prepared to communicate with ADB starts with doing this.
Device Information
It is critical to understand the characteristics of your Android device. The following is a straightforward shell command that provides detailed information about the currently connected device.
adb shell getprop
This command provides a list of exclusive device properties, including the phone number in store memory, the handle number for GSM radio standard communications, the operating system version, the device model, hardware details, and even kernel panics, if necessary. You can think of it as an inside perspective, opening a revealing window that can help identify problems and combine them with public report papers on security bugs to quickly ensure compatibility.
Battery Power
Battery management is a crucial part of the app development and testing process. You can simulate different battery conditions and examine how your program behaves in response by setting the battery level using adb shell dumpsys.
adb shell dumpsys battery set level 100
This command sets the battery level to 100%, but you can alter it to any percentage you wish. This is quite handy for evaluating your app’s response to low battery conditions, power saving settings, and charging state.
Bluetooth Enable
Many applications rely on Bluetooth capability, particularly those involving IoT devices, wearables, and music streaming. Use the following command to see if Bluetooth is enabled:
adb shell settings get global bluetooth_on
A ‘1’ in the terminal response indicates Bluetooth is enabled, whereas a ‘0’ indicates Bluetooth is turned off.
To turn on Bluetooth, use
adb shell am start -a android.bluetooth.adapter.action.REQUEST_ENABLE
To turn off Bluetooth:
adb shell am start -a android.bluetooth.adapter.action.REQUEST_DISABLE
Using these commands involves significantly less hardware labor when testing the Bluetooth functionality of your own software.
WiFi Control
Many apps require a Wi-Fi connection. To control the Wi-Fi status on your device, enter the following information:
– Enable Wi-Fi:
adb shell svc wifi enable
– Disable Wi-Fi:
adb shell svc wifi disable
This is particularly handy when evaluating how your app functions without internet connectivity and providing different network situation simulations.
Airplane Mode
When in airplane mode, all wireless connections are cut off, including Bluetooth, cellular, and Wi-Fi. You can use this to test the offline functionality of your app. Use these commands to switch between Airplane mode ON and Off:
To enable Airplane mode:
adb shell settings put global airplane_mode_on 1 OR adb shell am broadcast -a android.intent.action.AIRPLANE_MODE --ez state true
To disable Airplane mode:
adb shell settings put global airplane_mode_on 0 OR adb shell am broadcast -a android.intent.action.AIRPLANE_MODE --ez state false
These commands help you test your app’s behavior in various connectivity scenarios.
Installing an App
Deploying your app to a physical device is a crucial step in development. To install an APK file on your device, use:
adb install path-to-apk-file\sample-app.apk
Replace the ‘path-to-apk-file’ with the location of your APK file. This command simplifies the installation process across multiple devices, ensuring consistency.
Putting an App in the Background
Testing how your app behaves when sent to the background and brought back to the foreground is essential for a smooth user experience. To put an app in the background, use:
adb shell input keyevent 3
adb shell input keyevent 187
The first command simulates pressing the home button, sending the app to the background. The second command brings up the recent apps screen, allowing you to simulate app switching.
Relaunching an App
To relaunch an app, use the `monkey` tool, which sends random events to the system. This is useful for stress testing and ensuring your app can handle unexpected inputs:
Replace ` app-package ` with your app’s package name. This command launches the specified app, helping you automate the process of starting your app from a clean state.
Uninstalling an App
To uninstall an app from your device, use:
adb uninstall app-package
Replace ` app-package` with the package name of the app you want to uninstall. This command is useful for cleaning up after tests and ensuring no residual data remains on the device.
Making a Call
For apps that interact with the phone’s calling capabilities, being able to initiate calls programmatically is important. To initiate a call from your device, use:
adb shell am start -a android.intent.action.CALL -d tel:1231231234
Replace `1231231234` with the phone number you want to call. This command is useful for testing call-related features and interactions.
Rebooting the Device
Sometimes, a fresh start is needed. To reboot your device, simply use:
adb reboot
This command restarts your device, ensuring it is in a clean state before starting tests or after making significant changes.
Conclusion
These ADB instructions are but a sample of what may be accomplished using ADB by developers and testers. They undoubtedly offer a great deal of assistance with your work, including a better way to manage and debug native applications on your device and steps toward automation. These commands can be used in conjunction with your development and testing workflow to improve efficiency and ensure that, for example, an app’s behavior at various signal strengths is tested instantly. This eliminates the need for hours-long testing, which is particularly useful when a developer wants to upgrade or downgrade firmware based on how it will affect performance over varying time periods. This tool couldn’t be more powerful!
Whether you’re a seasoned developer, tester or just starting out, ADB is the only tool you need with its many helpful commands that can handle anything from making the screen as bright or obscure as you like to completely changing the way Android can be used.
Gaining control of these ADB commands will benefit you in a variety of testing and development situations, allowing your app to provide a seamless and enjoyable user experience. Thus, give them a go! I hope that learning about these commands will add a little bit of fun to your Android development and testing experience!
Click here to read more blogs like this and learn new tricks and techniques of software testing.
Apurva is a Test Engineer, with 3+ years of experience in Manual and automation testing. Having hands-on experience in testing Web as well as Mobile applications in Selenium and Playwright BDD with Java. And loves to explore and learn new tools and technologies.
When we hear the word Arduino, the first thing that comes to mind is a microprocessor chip assembled on a single board to help us to create an IOT friendly environment and IOT projects using Arduino communication.
This chip helps us create various projects and electronic devices, and moreover, facilitates physical interaction with the world of technology. Although it’s not only Arduino which is used in IOT Testing, other devices like raspberry pi are also used but Arduino is preferred first due to its cost and ease of handling.
However, Arduino is not just limited to a microprocessor chip. It is also a company that designs, manufactures, and supports electronic hardware and software and help to develop Arduino communication for IOT Application.
Arduino, moreover, makes the interaction between advanced technologies and the physical world easy and fun.
Additionally, Arduino is open-source, allowing anyone to contribute to and expand its capabilities.
Today in this blog we will cover Arduino and its different types of communication.
How Does Arduino communication using for IOT Application:
The main key to creating a project using any microprocessor is, therefore, to have an IoT-enabled environment.
To have an IOT enabled environment using any microprocessor is to be able to communicate or to be able to program and manage it accordingly, so is important in Arduino communication for IOT App.
Arduino communication is classified into two parts: wired and wireless, each with further subdivisions.
1. Wired Communication
Wired communication is, therefore, the most reliable and straightforward way to establish a connection. However, we won’t limit Arduino to simple wired communication. Instead, we’ll expand this wired communication into three distinct categories to develop IOT project/application using Arduino.
These 3 categories are explain below for Arduino Communication for IOT App:
Serial Communication:
UART communication is another name for serial communication. UART stands for universal Asynchronous receiver and transmitter.
It is mostly used for communication between Arduino and computers, but it can be applied in various other ways depending on our requirements and develop IOT application using Arduino. To establish the connection easily, we typically use the ‘Serial library’ in Arduino.
Examples for Serial Communication:
For an example let’s try to have UART communication between 2 Arduino.
Hardware Set-up
Connect the ground pin of both the Arduino.
Connect the RX pin of first Arduino with TX pin of second Arduino and TX pin of first Arduino to second Arduino.
Code Example :
Master Arduino Code:
void setup() {
Serial.begin(9600); // Initialize serial communication at 9600 bits per second
}
void loop() {
Serial.println("Hello from Master!"); // Send a message
delay(1000); // Wait for a second
}
Slave Arduino Code:
void setup() {
Serial.begin(9600); // Initialize serial communication at 9600 bits per second
}
void loop() {
if (Serial.available() > 0) { // Check if data is available to read
String message = Serial.readString(); // Read the incoming data
Serial.println("Received: " + message); // Print the received message
}
}
Here we go with a successful Arduino communication using UART
SPI Communication
SPI communication stands for Serial Peripheral Interface, which is usually used for short-distance communication and IOT projects.
The difference between serial communication and SPI communication is that, in contrast, SPI communication is synchronous and uses a clock cycle to communicate. Additionally, SPI is full-duplex communication.
In SPI you will find that there is a master slave relation where SPI has 4 wires to communicate which is MOSI (Master out Slave in) MISO (Master in Slave out) SCL (Serial clock) SS (slave select). Here, the first master sends the signal to all slaves with the identity number; consequently, the slave who has the same identity number reverts back to the master. After hearing back from the slave master, then send data to the particular slave. After the master sends the data, it then sends a clock signal to indicate that the data transfer is complete.
Examples for SPI Communication:
For example, let’s try to have SPI communication between two Arduino boards.
Hardware Set-up
First define the Pins and include SPI.H library in the code
Here let’s consider 10, 11, 12, 13 digital pins SS, MOSI, MISO, SCK on Arduino UNO.
Now let’s initialize the pins in the code:
void setup() {
SPI.begin(115200);
// Set pin modes for SS, MOSI, MISO, and SCK
pinMode(SS, OUTPUT);
pinMode(MOSI, OUTPUT);
pinMode(MISO, INPUT);
pinMode(SCK, OUTPUT);
// Set slave select (SS) pin high to disable the slave device
digitalWrite(SS, HIGH);
}
I2C Communication stands for Inter Integrated Circuit, and it is a half-duplex communication that works on a 2-wire protocol. Additionally, it has an extra overhead with start and stop bits.
I2C has an acknowledgment bit after every byte of transfer. It has a pullup resistor requirement. I2c is slower but comes with clock stretch functionality.
Examples for I2C Communication:
For an example let’s try to have I2C communication between 2 Arduino
Hardware Set-up
I2C pins on the Arduino Uno are SDA (A4) and SCL (A5).
Connect the Ground pin of Both Arduinos.
Connect the SDA pin of Master to the SDA pin of slave.
Connect the SCL pin of Master to the SCL pin of Master.
Master Arduino Code:
#include <Wire.h>
void setup() {
Wire.begin(); // Join I2C bus as master
}
void loop() {
Wire.beginTransmission(8); // Begin transmission to device with address 8
Wire.write("Hello from Master!"); // Send a message
Wire.endTransmission(); // End transmission
delay(1000); // Wait for a second
}
Slave Arduino Code:
#include <Wire.h>
void setup() {
Wire.begin(8); // Join I2C bus with address 8
Wire.onReceive(receiveEvent); // Register a function to be called when data is received
}
void loop() {
// Do nothing, everything is handled in receiveEvent
}
void receiveEvent(int howMany) {
while (Wire.available()) { // Loop through all received bytes
char c = Wire.read(); // Read each byte
Serial.print(c); // Print the received message
}
Serial.println();
}
PFB Image of Connection
2. Wireless Communication
Sometime we want to develop the projects which need to have the flexibility due which we need to have wireless connection. Don’t worry Arduino covered you there too Arduino do have 4 types wireless communication for IOT project using Arduino.
Bluetooth: As the name is very familiar to us, let’s directly move to the components which help us to attain Bluetooth connectivity. By using HC-05 and HC-06 we can attain Bluetooth connectivity using Arduino. However Bluetooth provides connectivity to a short distance.
Wifi: Sometimes we don’t need short distance coverage then we can switch to the wifi or in cases where we want the internet accessibility for some of our projects we can move to wifi connection. You can use the ESP32 and ESP8266 modules for the same purpose.
RF (Radio Frequency): Uses radio waves for medium-distance communication, with modules like nRF24L01 and RF433.
LoRa (Long Range): Great for long-distance, low-power communication, ideal for remote monitoring systems.
Conclusion
In conclusion, Arduino offers a versatile platform for various types of communication, allowing users to create innovative and interactive projects in IOT. Through serial, SPI, and I2C communications, Arduino facilitates reliable and efficient data exchange, each method catering to different needs and scenarios for exploring Arduino communication for IOT App.
Whether you are connecting multiple Arduinos or interfacing with other devices, understanding these communication protocols enhances your ability to leverage Arduino’s full potential in the IOT world.
Deepti Rana is a passionate profession keen to learn new technologies and exceling them with expertise. She thrives to bridge the gap between real time problems and technical world. With a background in IT she has taken steps in robotics too. Her background in IT doesn’t stop her to explore different fields, that can be seen through her projects where see has used GSM, GPRS, ESP32, Arduino, PS2, line sensor, ultrasonic sensors, and fingerprint sensor. She does love to share her knowledge, which she accomplishes by volunteering in Bhumi NGO.
