Software testing mistakes to fix using AI — software testing isn’t just about finding bugs — it’s about ensuring that the product delivers value, reliability, and confidence to both the business and the end-users. Yet, even experienced QA engineers and teams fall into common traps that undermine the effectiveness of their testing efforts, which include Software testing mistakes to fix using AI.
If you’ve ever felt like you’re running endless test cycles but still missing critical defects in production, chances are one (or more) of these mistakes is happening in your process. Let’s break down the 7 most common software testing mistakes to fix using AI.
1. Treating Testing as a Last-Minute Activity
The mistake:
In many organizations, testing still gets pushed to the very end of the development lifecycle. The team develops features for weeks or months, and once deadlines are looming, QA is told to “quickly test everything.” This leaves little time for proper planning, exploratory testing, or regression checks. Rushed testing almost always results in overlooked bugs.
How to avoid it:
Adopt a shift-left testing mindset: bring QA into the earliest stages of development. Testers can review requirements, user stories, and wireframes to identify issues before code is written.
Integrate testing into each sprint if you’re following Agile. Don’t wait until the release phase — test incrementally.
Encourage developers to write unit tests and practice TDD (Test-Driven Development), so defects are caught as early as possible.
Early involvement means fewer surprises at the end and a smoother release process.
Fix this with AI:
AI-powered requirement analysis tools can review user stories and design docs to automatically highlight ambiguities or missing edge cases. Generative AI can also generate preliminary test cases as soon as requirements are written, helping QA get started earlier without waiting for code. Predictive analytics can forecast potential high-risk areas of the codebase so testers prioritize them early in the sprint.
2. Lack of Clear Test Objectives
The mistake:
Testing without defined goals is like shooting in the dark. Some teams focus only on “happy path” tests that check whether the basic workflow works, but skip edge cases, negative scenarios, or business-critical paths. Without clarity, QA may spend a lot of time running tests that don’t actually reduce risk.
How to avoid it:
Define testing objectives for each cycle: Are you validating performance? Checking for usability? Ensuring compliance.
Collaborate with product owners and developers to write clear acceptance criteria for user stories.
Maintain a test strategy document that outlines what kinds of tests are required (unit, integration, end-to-end, performance, security).
Having clear objectives ensures testing isn’t just about “checking boxes” but about delivering meaningful coverage that aligns with business priorities.
Fix this with AI:
Use NLP-powered tools to automatically analyze user stories and acceptance criteria, flagging ambiguous or missing requirements. This ensures QA teams can clarify intent before writing test cases, reducing gaps caused by unclear objectives. AI-driven dashboards can also track coverage gaps in real time, so objectives don’t get missed.
3. Over-Reliance on Manual Testing
The mistake:
Manual testing is valuable, but if it’s the only approach, teams end up wasting effort on repetitive tasks. Regression testing, smoke testing, and large datasets are prone to human error when done manually. Worse, it slows down releases in fast-paced CI/CD pipelines.
How to avoid it:
Identify repetitive test cases that can be automated and start small — login flows, form submissions, and critical user journeys.
Use frameworks like Selenium, Cypress, Playwright, Appium, or Pytest for automation, depending on your tech stack.
Balance automation with manual exploratory testing. Automation gives speed and consistency, while human testers uncover usability issues and unexpected defects.
Think of automation as your assistant, not your replacement. The best testing strategy combines the efficiency of automation with the creativity of manual exploration.
Fix this with AI:
AI-driven test automation tools can generate, maintain, and even self-heal test scripts automatically when the UI changes, reducing maintenance overhead. Machine learning models can prioritize regression test cases based on historical defect data and usage analytics, so you test what truly matters.
4. Poor Test Data and Environment Management
The mistake:
It’s common to hear: “The bug doesn’t happen in staging but appears in production.” This usually happens because test environments don’t mimic production conditions or because test data doesn’t reflect real-world complexity. Incomplete or unrealistic data leads to false confidence in test results.
How to avoid it:
Create production-like environments for staging and QA. Use containerization (Docker, Kubernetes) to replicate conditions consistently.
Use synthetic but realistic test data that covers edge cases (e.g., very large inputs, special characters, boundary values).
Refresh test data regularly, and anonymize sensitive customer data if you use production datasets.
Remember, if your test environment doesn’t reflect reality, your tests won’t either.
Fix this with AI:
AI-driven test data generators can automatically craft rich, production-like datasets that simulate real user behavior and edge cases without exposing sensitive data. Machine learning models can identify missing coverage areas by analyzing historical production incidents and system logs, ensuring your tests anticipate future issues—not just past ones.
5. Ignoring Non-Functional Testing
The mistake:
Too many teams stop at “the feature works.” But does it scale when thousands of users log in at once? Does it remain secure under malicious attacks? Does it deliver a smooth experience on low network speeds? Ignoring non-functional testing creates systems that “work fine” in a demo but fail in the real world.
How to avoid it:
Integrate performance testing into your pipeline using tools like JMeter or Locust to simulate real-world traffic.
Run security tests (SQL injection, XSS, broken authentication) regularly — don’t wait for a penetration test once a year. ZAP Proxy passive and active scans can help!
Conduct usability testing with actual users or stakeholders to validate that the software isn’t just functional, but intuitive.
A product that functions correctly but performs poorly or feels insecure still damages user trust. Non-functional testing is just as critical as functional testing.
Fix this with AI:
AI can elevate non-functional testing from reactive to predictive. Machine learning models can simulate complex user patterns across diverse devices, geographies, and network conditions—pinpointing performance bottlenecks before they appear in production.
AI-driven security testing tools constantly evolve with new threat intelligence, automatically generating attack scenarios that mirror real-world exploits such as injection attacks, authentication bypasses, and API abuse.
For usability, AI-powered analytics and vision models can evaluate screen flows, identify confusing layouts, and detect design elements that slow user interaction. Instead of waiting for manual feedback cycles, development teams get continuous, data-backed insights to refine performance, security, and experience in tandem.
6. Inadequate Test Coverage and Documentation
The mistake:
Incomplete or outdated test cases often lead to critical gaps. Some QA teams also skip documentation to “save time,” but this creates chaos later — new team members don’t know what’s been tested, bugs get repeated, and regression cycles lose effectiveness.
How to avoid it:
Track test coverage using tools that measure which parts of the codebase are covered by automated tests.
Keep documentation lightweight but structured: test charters, bug reports, acceptance criteria, and coverage reports. Avoid bloated test case repositories that nobody reads.
Treat documentation as a living artifact. Update it continuously, not just during release crunches.
Good documentation doesn’t have to be lengthy — it has to be useful and easy to maintain.
Fix this with AI:
AI can transform documentation and coverage management from a manual chore into a continuous, intelligent process. By analyzing code commits, test execution results, and requirements, AI tools can automatically generate and update test documentation, keeping it synchronized with the evolving product.
Machine learning models can assess coverage depth, correlate it with defect history, and flag untested or high-risk code paths before they cause production issues. AI-powered assistants can also turn static documentation into dynamic knowledge engines, allowing testers to query test cases, trace feature impacts, or uncover reusable scripts instantly.
This ensures documentation stays accurate, context-aware, and actionable — supporting faster onboarding and more confident releases.
7. Not Learning from Production Defects
The mistake:
Bugs escaping into production are inevitable. But the bigger mistake is when teams only fix the bug and move on, without analyzing why it slipped through. This leads to the same categories of defects reappearing release after release.
How to avoid it:
Run root cause analysis for every critical production defect. Was it a missed requirement? An incomplete test case? An environment mismatch?
Use post-mortems not to blame but to improve processes. For example, if login bugs frequently slip through, strengthen test coverage around authentication.
Feed learnings back into test suites, automation, and requirements reviews. developers to write unit tests and practice TDD (Test-Driven Development), so defects are caught as early as possible.
Great QA teams don’t just find bugs — they learn from them, so they don’t happen again.
Fix this with AI:
AI can turn every production defect into a learning opportunity for continuous improvement. By analyzing production logs, telemetry, and historical bug data, AI systems can uncover hidden correlations—such as which modules, code changes, or dependencies are most prone to introducing similar defects. Predictive analytics models can forecast which areas of the application are most at risk in upcoming releases, guiding QA teams to focus their regression tests strategically. AI-powered Root Cause Analysis tools can automatically cluster related issues, trace them to their originating commits, and even propose preventive test cases or test data refinements to avoid repeating past mistakes.
Instead of reacting to production failures, AI helps teams proactively strengthen their QA process with data-driven intelligence and faster feedback loops.
Conclusion: Building a Smarter QA Practice with AI
Software testing is not just a phase in development — it’s a mindset. It requires curiosity, discipline, and continuous improvement. Avoiding these seven mistakes can transform your QA practice from a bottleneck into a true enabler of quality and speed.
Software testing mistakes to fix using AI. Here’s the truth: quality doesn’t happen by accident. It’s the result of planning, collaboration, and constant refinement. By involving QA early, setting clear objectives, balancing manual and automated testing, managing data effectively, and learning from past mistakes, your team can deliver not just working software, but software that delights users and stands the test of time.
AI takes this one step further — with predictive analytics to catch risks earlier, self-healing test automation that adapts to change, intelligent test data generation, and AI-powered RCA (Root Cause Analysis) that learns from production. Instead of chasing bugs, QA teams can focus on engineering intelligent, resilient, and user-centric quality.
Strong QA isn’t about finding more bugs — it’s about building more confidence. And with AI, that confidence scales with every release.
I’m Sr. Digital Marketing Executive with a strong interest in content strategy, SEO, and social media marketing. She is passionate about building brand presence through creative and analytical approaches. In her free time, she enjoys learning new digital trends and exploring innovative marketing tools.
API Automation Testing Framework – In Today’s fast-paced digital ecosystem, almost every modern application relies on APIs (Application Programming Interfaces) to function seamlessly. Whether it’s a social media integration pulling live updates, a payment gateway processing transaction, or a data service exchanging real-time information, APIs act as the invisible backbone that connects various systems together.
Because APIs serve as the foundation of all interconnected software, ensuring that they are reliable, secure, and high performing is absolutely critical. Even a minor API failure can impact multiple dependent systems; consequently, it may cause application downtime, data mismatches, or even financial loss.
That’s where API automation testing framework comes in. Unlike traditional UI testing, API testing validates the core business logic directly at the backend layer, which makes it faster, more stable, and capable of detecting issues early in the development cycle — even before the frontend is ready.
In this blog, we’ll walk through the process of building a complete API Automation Testing Framework using a combination of:
Java – as the main programming language
Maven – for project and dependency management
Cucumber – to implement Behavior Driven Development (BDD)
RestAssured – for simplifying RESTful API automation
Playwright – to handle browser-based token generation
The framework you’ll learn to build will follow a BDD (Behavior-Driven Development) approach, enabling test scenarios to be written in simple, human-readable language. This not only improves collaboration between developers, testers, and business analysts but also makes test cases easier to understand, maintain, and extend.
Additionally, the API automation testing framework will be CI/CD-friendly, meaning it can be seamlessly integrated into automated build pipelines for continuous testing and faster feedback.
By the end of this guide, you’ll have a scalable, reusable, and maintainable API testing framework that brings together the best of automation, reporting, and real-time token management — a complete solution for modern QA teams.
What is API?
An API (Application Programming Interface) acts as a communication bridge between two software systems, allowing them to exchange information in a standardized way. In simpler terms, it defines how different software components should interact — through a set of rules, protocols, and endpoints.
Think of an API as a messenger that takes a request from one system, delivers it to another system, and then brings back the response. This interaction, therefore, allows applications to share data and functionality without exposing their internal logic or database structure.
Let’s take a simple example: When you open a weather application on your phone, it doesn’t store weather data itself. Instead, it sends a request to a weather server API, which processes the request and sends back a response — such as the current temperature, humidity, or forecast. This request-response cycle is what makes APIs so powerful and integral to almost every digital experience we use today.
Most modern APIs follow the REST (Representational State Transfer) architectural style. REST APIs use the HTTP protocol and are designed around a set of standardized operations, including:
HTTP Method
Description
Example Use
GET
Retrieve data from the server
Fetch a list of users
POST
Create new data on the server
Add a new product
PUT
Update existing data
edit user details
DELETE
Remove data
Delete a record
The responses returned by API’s are typically in JSON (JavaScript Object Notation) format – a lightweight, human-readable, and machine-friendly data format that’s easy to parse and validate.
In essence, API’s are the digital glue that holds modern applications together — enabling smooth communication, faster integrations, and a consistent flow of information across systems.
What is API Testing?
API Testing is the process of verifying that an API functions correctly and performs as expected — ensuring that all its endpoints, parameters, and data exchanges behave according to defined business rules.
In simple terms, it’s about checking whether the backend logic of an application works properly — without needing a graphical user interface (UI). Since APIs act as the communication layer between different software components, testing them helps ensure that the entire system remains reliable, secure, and efficient.
API testing typically focuses on four main aspects:
Functionality – Does the API perform the intended operation and return the correct response for valid requests?
Reliability – Does it deliver consistent results every time, even under different inputs and conditions?
Security – Is the API protected from unauthorized access, data leaks, or token misuse?
Performance – Does it respond quickly and remain stable under heavy load or high traffic?
Unlike traditional UI testing, which validates the visual and interactive parts of an application, API testing operates directly at the business logic layer. This makes it:
Faster – Since it bypasses the UI, execution times are much shorter.
More Stable – UI changes (like a button name or layout) don’t affect API tests.
Proactive – Tests can be created and run even before the front-end is developed.
In essence, API testing ensures the heart of your application is healthy. By validating responses, performance, and security at the API level, teams can detect defects early, reduce costs, and deliver more reliable software to users.
Why is API Testing Important?
API Testing plays a vital role in modern software development because APIs form the backbone of most applications. A failure in an API can affect multiple systems and impact overall functionality.
Here’s why API testing is important:
Ensures Functionality: Verifies that endpoints return correct responses and handle errors properly.
Enhances Security: Detects vulnerabilities like unauthorized access or token misuse.
Validates Data Integrity: Confirms that data remains consistent across APIs and databases.
Improves Performance: Checks response time, stability, and behavior under load.
Detects Defects Early: Allows early testing right after backend development, saving time and cost
Supports Continuous Integration: Easily integrates with CI/CD pipelines for automated validation.
In short, API testing ensures your system’s core logic is reliable, secure, and ready for real-world use.
Tools for Manual API Testing
Before jumping into automation, it’s essential to explore and understand APIs manually. Manual testing helps you validate endpoints, check responses, and get familiar with request structures.
Here are some popular tools used for manual API testing:
Postman: The most widely used tool for sending API requests, validating responses, and organizing test collections [refer link – https://www.postman.com/.
SoapUI: Best suited for testing both SOAP and REST APIs with advanced features like assertions and mock services.
Insomnia: A lightweight and user-friendly alternative to Postman, ideal for quick API exploration.
cURL: A command-line tool perfect for making fast API calls or testing from scripts.
Fiddler: Excellent for capturing and debugging HTTP/HTTPS traffic between client and server.
Using these tools helps testers understand API behavior, request/response formats, and possible edge cases — forming a strong foundation before moving to API automation.
Tools for API Automation Testing
After verifying APIs manually, the next step is to automate them using reliable tools and libraries. Automation helps improve test coverage, consistency, and execution speed.
Here are some popular tools used for API automation testing:
RestAssured: A powerful Java library designed specifically for testing and validating RESTful APIs.
Cucumber: Enables writing test cases in Gherkin syntax (plain English), making them easy to read and maintain.
Playwright: Automates browser interactions; in our framework, it will be used for token generation or authentication flows.
Postman + Newman: Allows you to run Postman collections directly from the command line — ideal for CI/CD integration.
JMeter: A robust tool for performance and load testing of APIs under different conditions.
In this blog, our focus will be on building a framework using RestAssured, Cucumber, and Playwright — combining functional, BDD, and authentication automation into one cohesive setup.
Framework Overview
We’ll build a Behavior-Driven API Automation Testing Framework that combines multiple tools for a complete testing solution. Here’s how each component fits in:
Cucumber – Manages the BDD layer, allowing test scenarios to be written in simple, readable feature files.
RestAssured – Handles HTTP requests and responses for validating RESTful APIs.
Playwright – Automates browser-based actions like token generation or authentication.
Maven – Manages project dependencies, builds, and plugins efficiently.
Cucumber HTML Reports – Automatically generates detailed execution reports after each run.
The framework follows a modular structure, with separate packages for step definitions, utilities, configurations, and feature files — ensuring clean organization, easy maintenance, and scalability.
In this, we will be creating a feature file for API Automation Testing Framework. A feature file consists of steps. These steps are mentioned in the gherkin language. The feature is easy to understand and can be written in the English language so that a non-technical person can understand the flow of the test scenario. In this framework we will be automating the four basic API request methods i.e. POST, PUT, GET and DELETE.
We can assign tags to our scenarios mentioned in the feature file to run particular test scenarios based on the requirement. The key point you must notice here is the feature file should end with .feature extension. We will be creating four different scenarios for the four different API methods.
Feature: All Notes API Validation
@api
Scenario Outline: Validate POST Create Notes API Response for "<scenarioName>" Scenario
When User sends "<method>" request to "<url>" with headers "<headers>" and query file "<queryFile>" and requestDataFile "<bodyFile>"
Then User verifies the response status code is <statusCode>
And User verifies the response body matches JSON schema "<schemaFile>"
Then User verifies fields in response: "<contentType>" with content type "<fields>"
Examples:
| scenarioName | method | url | headers | queryFile | bodyFile | statusCode | schemaFile | contentType | fields |
| Valid create Notes | POST | /api/v1/loan-syndications/{dealId}/investors/{investorId}/notes | NA | NA | Create_Notes_Request | 200 | NA | NA | NA |
Scenario Outline: Validate GET Notes API Response for "<scenarioName>" Scenario
When User sends "<method>" request to "<url>" with headers "<headers>" and query file "<queryFile>" and requestDataFile "<bodyFile>"
Then User verifies the response status code is <statusCode>
And User verifies the response body matches JSON schema "<schemaFile>"
Then User verifies fields in response: "<contentType>" with content type "<fields>"
Examples:
| scenarioName | method | url | headers | queryFile | bodyFile | statusCode | schemaFile | contentType | fields |
| Valid Get Notes | GET | /api/v1/loan-syndications/{dealId}/investors/{investorId}/notes | NA | NA | NA | 200 | Notes_Schema_200 | json | note=This is Note 1 |
Scenario Outline: Validate Update Notes API Response for "<scenarioName>" Scenario
When User sends "<method>" request to "<url>" with headers "<headers>" and query file "<queryFile>" and requestDataFile "<bodyFile>"
Then User verifies the response status code is <statusCode>
And User verifies the response body matches JSON schema "<schemaFile>"
Then User verifies fields in response: "<contentType>" with content type "<fields>"
Examples:
| scenarioName | method | url | headers | queryFile | bodyFile | statusCode | schemaFile | contentType | fields |
| Valid update Notes | PUT | /api/v1/loan-syndications/{dealId}/investors/{investorId}/notes/{noteId}/update-notes | NA | NA | Update_Notes_Request | 200 | NA | NA | NA |
Scenario Outline: Validate DELETE Create Notes API Response for "<scenarioName>" Scenario
When User sends "<method>" request to "<url>" with headers "<headers>" and query file "<queryFile>" and requestDataFile "<bodyFile>"
Then User verifies the response status code is <statusCode>
And User verifies the response body matches JSON schema "<schemaFile>"
Then User verifies fields in response: "<contentType>" with content type "<fields>"
Examples:
| scenarioName | method | url | headers | queryFile | bodyFile | statusCode | schemaFile | contentType | fields |
| Valid delete | DELETE | /api/v1/loan-syndications/{dealId}/investors/{investorId}/notes/{noteId} | NA | NA | NA | 200 | NA | NA | NA |
Step 4: Creating a Step Definition File
Unlike the automation framework which we have built in the previous blog, we will be creating a single-step file for all the feature files. In the BDD framework, the step files are used to map and implement the steps described in the feature file. Rest Assured library is very accurate to map the steps with the steps described in the feature file. We will be describing the same steps in the step file as they have described in the feature file so that behave will come to know the step implementation for the particular steps present in the feature file.
package org.Spurqlabs.Steps;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.When;
import io.restassured.response.Response;
import org.Spurqlabs.Core.TestContext;
import org.Spurqlabs.Utils.*;
import org.json.JSONArray;
import org.json.JSONObject;
import java.io.File;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.HashMap;
import java.util.Map;
import static io.restassured.module.jsv.JsonSchemaValidator.matchesJsonSchemaInClasspath;
import static org.Spurqlabs.Utils.DealDetailsManager.replacePlaceholders;
import static org.hamcrest.Matchers.equalTo;
public class CommonSteps extends TestContext {
private Response response;
@When("User sends {string} request to {string} with headers {string} and query file {string} and requestDataFile {string}")
public void user_sends_request_to_with_query_file_and_requestDataFile (String method, String url, String headers, String queryFile, String bodyFile) throws IOException {
String jsonString = Files.readString(Paths.get(FrameworkConfigReader.getFrameworkConfig("DealDetails")), StandardCharsets.UTF_8);
JSONObject storedValues = new JSONObject(jsonString);
String fullUrl = FrameworkConfigReader.getFrameworkConfig("BaseUrl") + replacePlaceholders(url);
Map<String, String> header = new HashMap<>();
if (!"NA".equalsIgnoreCase(headers)) {
header = JsonFileReader.getHeadersFromJson(FrameworkConfigReader.getFrameworkConfig("headers") + headers + ".json");
} else {
header.put("cookie", TokenManager.getToken());
}
Map<String, String> queryParams = new HashMap<>();
if (!"NA".equalsIgnoreCase(queryFile)) {
queryParams = JsonFileReader.getQueryParamsFromJson(FrameworkConfigReader.getFrameworkConfig("Query_Parameters") + queryFile + ".json");
for (String key : queryParams.keySet()) {
String value = queryParams.get(key);
for (String storedKey : storedValues.keySet()) {
value = value.replace("{" + storedKey + "}", storedValues.getString(storedKey));
}
queryParams.put(key, value);
}
}
Object requestBody = null;
if (!"NA".equalsIgnoreCase(bodyFile)) {
String bodyTemplate = JsonFileReader.getJsonAsString(
FrameworkConfigReader.getFrameworkConfig("Request_Bodies") + bodyFile + ".json");
for (String key : storedValues.keySet()) {
String placeholder = "{" + key + "}";
if (bodyTemplate.contains(placeholder)) {
bodyTemplate = bodyTemplate.replace(placeholder, storedValues.getString(key));
}
}
requestBody = bodyTemplate;
}
response = APIUtility.sendRequest(method, fullUrl, header, queryParams, requestBody);
response.prettyPrint();
TestContextLogger.scenarioLog("API", "Request sent: " + method + " " + fullUrl);
if (scenarioName.contains("GET Notes") && response.getStatusCode() == 200) {
DealDetailsManager.put("noteId", response.path("[0].id"));
}
}
@Then("User verifies the response status code is {int}")
public void userVerifiesTheResponseStatusCodeIsStatusCode(int statusCode) {
response.then().statusCode(statusCode);
TestContextLogger.scenarioLog("API", "Response status code: " + statusCode);
}
@Then("User verifies the response body matches JSON schema {string}")
public void userVerifiesTheResponseBodyMatchesJSONSchema(String schemaFile) {
if (!"NA".equalsIgnoreCase(schemaFile)) {
String schemaPath = "Schema/" + schemaFile + ".json";
response.then().assertThat().body(matchesJsonSchemaInClasspath(schemaPath));
TestContextLogger.scenarioLog("API", "Response body matches schema");
} else {
TestContextLogger.scenarioLog("API", "Response body does not have schema to validate");
}
}
@Then("User verifies field {string} has value {string}")
public void userVerifiesFieldHasValue(String jsonPath, String expectedValue) {
response.then().body(jsonPath, equalTo(expectedValue));
TestContextLogger.scenarioLog("API", "Field " + jsonPath + " has value: " + expectedValue);
}
@Then("User verifies fields in response: {string} with content type {string}")
public void userVerifiesFieldsInResponseWithContentType(String contentType, String fields) throws IOException {
// If NA, skip verification
if ("NA".equalsIgnoreCase(contentType) || "NA".equalsIgnoreCase(fields)) {
return;
}
String responseStr = response.getBody().asString().trim();
try {
if ("text".equalsIgnoreCase(contentType)) {
// For text, verify each expected value is present in response
for (String expected : fields.split(";")) {
expected = replacePlaceholders(expected.trim());
if (!responseStr.contains(expected)) {
throw new AssertionError("Expected text not found: " + expected);
}
TestContextLogger.scenarioLog("API", "Text found: " + expected);
}
} else if ("json".equalsIgnoreCase(contentType)) {
// For json, verify key=value pairs
JSONObject jsonResponse;
if (responseStr.startsWith("[")) {
JSONArray arr = new JSONArray(responseStr);
jsonResponse = !arr.isEmpty() ? arr.getJSONObject(0) : new JSONObject();
} else {
jsonResponse = new JSONObject(responseStr);
}
for (String pair : fields.split(";")) {
if (pair.trim().isEmpty()) continue;
String[] kv = pair.split("=", 2);
if (kv.length < 2) continue;
String keyPath = kv[0].trim();
String expected = replacePlaceholders(kv[1].trim());
Object actual = JsonFileReader.getJsonValueByPath(jsonResponse, keyPath);
if (actual == null) {
throw new AssertionError("Key not found in JSON: " + keyPath);
}
if (!String.valueOf(actual).equals(String.valueOf(expected))) {
throw new AssertionError("Mismatch for " + keyPath + ": expected '" + expected + "', got '" + actual + "'");
}
TestContextLogger.scenarioLog("API", "Validated: " + keyPath + " = " + expected);
}
} else {
throw new AssertionError("Unsupported content type: " + contentType);
}
} catch (AssertionError | Exception e) {
TestContextLogger.scenarioLog("API", "Validation failed: " + e.getMessage());
throw e;
}
}
Step 5: Creating API
Till now we have successfully created a feature file and a step file now in this step we will be creating a utility file. Generally, in Web automation, we have page files that contain the locators and the actions to perform on the web elements but in this framework, we will be creating a single utility file just like the step file. The utility file contains the API methods and the endpoints to perform the specific action like, POST, PUT, GET, or DELETE. The request body i.e. payload and the response body will be captured using the methods present in the utility file. So the reason these methods are created in the utility file is that we can use them multiple times and don’t have to create the same method over and over again.
package org.Spurqlabs.Utils;
import io.restassured.RestAssured;
import io.restassured.http.ContentType;
import io.restassured.response.Response;
import io.restassured.specification.RequestSpecification;
import java.io.File;
import java.util.Map;
public class APIUtility {
public static Response sendRequest(String method, String url, Map<String, String> headers, Map<String, String> queryParams, Object body) {
RequestSpecification request = RestAssured.given();
if (headers != null && !headers.isEmpty()) {
request.headers(headers);
}
if (queryParams != null && !queryParams.isEmpty()) {
request.queryParams(queryParams);
}
if (body != null && !method.equalsIgnoreCase("GET")) {
if (headers == null || !headers.containsKey("Content-Type")) {
request.header("Content-Type", "application/json");
}
request.body(body);
}
switch (method.trim().toUpperCase()) {
case "GET":
return request.get(url);
case "POST":
return request.post(url);
case "PUT":
return request.put(url);
case "PATCH":
return request.patch(url);
case "DELETE":
return request.delete(url);
default:
throw new IllegalArgumentException("Unsupported HTTP method: " + method);
}
}
Step 6: Create a Token Generation using Playwright
In this step, we automate the process of generating authentication tokens using Playwright. Many APIs require login-based tokens (like cookies or bearer tokens), and managing them manually can be difficult — especially when they expire frequently.
The TokenManager class handles this by:
Logging into the application automatically using Playwright.
Extracting authentication cookies (OauthHMAC, OauthExpires, BearerToken).
Storing the token in a local JSON file for reuse.
Refreshing the token automatically when it expires.
This ensures that your API tests always use a valid token without manual updates, making the framework fully automated and CI/CD ready.
package org.Spurqlabs.Utils;
import java.io.*;
import java.nio.file.*;
import java.time.Instant;
import java.util.HashMap;
import java.util.Map;
import com.google.gson.Gson;
import com.google.gson.reflect.TypeToken;
import com.microsoft.playwright.*;
import com.microsoft.playwright.options.Cookie;
public class TokenManager {
private static final ThreadLocal<String> tokenThreadLocal = new ThreadLocal<>();
private static final ThreadLocal<Long> expiryThreadLocal = new ThreadLocal<>();
private static final String TOKEN_FILE = "token.json";
private static final long TOKEN_VALIDITY_SECONDS = 30 * 60; // 30 minutes
public static String getToken() {
String token = tokenThreadLocal.get();
Long expiry = expiryThreadLocal.get();
if (token == null || expiry == null || Instant.now().getEpochSecond() >= expiry) {
// Try to read from a file (for multi-JVM/CI)
Map<String, Object> fileToken = readTokenFromFile();
if (fileToken != null) {
token = (String) fileToken.get("token");
expiry = ((Number) fileToken.get("expiry")).longValue();
}
// If still null or expired, fetch new
if (token == null || expiry == null || Instant.now().getEpochSecond() >= expiry) {
Map<String, Object> newToken = generateAuthTokenViaBrowser();
token = (String) newToken.get("token");
expiry = (Long) newToken.get("expiry");
writeTokenToFile(token, expiry);
}
tokenThreadLocal.set(token);
expiryThreadLocal.set(expiry);
}
return token;
}
private static Map<String, Object> generateAuthTokenViaBrowser() {
String bearerToken;
long expiry = Instant.now().getEpochSecond() + TOKEN_VALIDITY_SECONDS;
int maxRetries = 2;
int attempt = 0;
Exception lastException = null;
while (attempt < maxRetries) {
try (Playwright playwright = Playwright.create()) {
Browser browser = playwright.chromium().launch(new BrowserType.LaunchOptions().setHeadless(true));
BrowserContext context = browser.newContext();
Page page = context.newPage();
// Robust wait for login page to load
page.navigate(FrameworkConfigReader.getFrameworkConfig("BaseUrl"), new Page.NavigateOptions().setTimeout(60000));
page.waitForSelector("#email", new Page.WaitForSelectorOptions().setTimeout(20000));
page.waitForSelector("#password", new Page.WaitForSelectorOptions().setTimeout(20000));
page.waitForSelector("button[type='submit']", new Page.WaitForSelectorOptions().setTimeout(20000));
// Fill a login form
page.fill("#email", FrameworkConfigReader.getFrameworkConfig("UserEmail"));
page.fill("#password", FrameworkConfigReader.getFrameworkConfig("UserPassword"));
page.waitForSelector("button[type='submit']:not([disabled])", new Page.WaitForSelectorOptions().setTimeout(10000));
page.click("button[type='submit']");
// Wait for either dashboard element or flexible URL match
boolean loggedIn;
try {
page.waitForSelector(".dashboard, .main-content, .navbar, .sidebar", new Page.WaitForSelectorOptions().setTimeout(20000));
loggedIn = true;
} catch (Exception e) {
// fallback to URL check
try {
page.waitForURL(url -> url.startsWith(FrameworkConfigReader.getFrameworkConfig("BaseUrl")), new Page.WaitForURLOptions().setTimeout(30000));
loggedIn = true;
} catch (Exception ex) {
// Both checks failed
loggedIn = false;
}
}
if (!loggedIn) {
throw new RuntimeException("Login did not complete successfully: dashboard element or expected URL not found");
}
// Extract cookies
String oauthHMAC = null;
String oauthExpires = null;
String token = null;
for (Cookie cookie : context.cookies()) {
switch (cookie.name) {
case "OauthHMAC":
oauthHMAC = cookie.name + "=" + cookie.value;
break;
case "OauthExpires":
oauthExpires = cookie.name + "=" + cookie.value;
if (cookie.expires != null && cookie.expires > 0) {
expiry = cookie.expires.longValue();
}
break;
case "BearerToken":
token = cookie.name + "=" + cookie.value;
break;
}
}
if (oauthHMAC != null && oauthExpires != null && token != null) {
bearerToken = oauthHMAC + ";" + oauthExpires + ";" + token + ";";
} else {
throw new RuntimeException("❗ One or more cookies are missing: OauthHMAC, OauthExpires, BearerToken");
}
browser.close();
Map<String, Object> map = new HashMap<>();
map.put("token", bearerToken);
map.put("expiry", expiry);
return map;
} catch (Exception e) {
lastException = e;
System.err.println("[TokenManager] Login attempt " + (attempt + 1) + " failed: " + e.getMessage());
attempt++;
try { Thread.sleep(2000); } catch (InterruptedException ignored) {}
}
}
throw new RuntimeException("Failed to generate auth token after " + maxRetries + " attempts", lastException);
}
private static void writeTokenToFile(String token, long expiry) {
try {
Map<String, Object> map = new HashMap<>();
map.put("token", token);
map.put("expiry", expiry);
String json = new Gson().toJson(map);
Files.write(Paths.get(TOKEN_FILE), json.getBytes());
} catch (IOException e) {
e.printStackTrace();
}
}
private static Map<String, Object> readTokenFromFile() {
try {
Path path = Paths.get(TOKEN_FILE);
if (!Files.exists(path)) return null;
String json = new String(Files.readAllBytes(path));
return new Gson().fromJson(json, new TypeToken<Map<String, Object>>() {}.getType());
} catch (IOException e) {
return null;
}
}
}
Step 7: Create Framework Config File
A good tester is one who knows the use and importance of config files. In this framework, we are also going to use the config file. Here, we are just going to put the base URL in this config file and will be using the same in the utility file over and over again. The config file contains more data than just of base URL when you start exploring the framework and start automating the new endpoints then at some point, you will realize that some data can be added to the config file.
Additionally, the purpose of the config files is to make tests more maintainable and reusable. Another benefit of a config file is that it makes the code more modular and easier to understand as all the configuration settings are stored in a separate file and it makes it easier to update the configuration settings for all the tests at once.
At this stage, we create the TestRunner class, which serves as the entry point to execute all Cucumber feature files. It uses TestNG as the test executor and integrates Cucumber for running BDD-style test scenarios.
The @CucumberOptions annotation defines:
features → Location of all .feature files.
glue → Packages containing step definitions and hooks.
plugin → Reporting options like JSON and HTML reports.
After execution, Cucumber automatically generates:
Cucumber.json → For CI/CD and detailed reporting.
Cucumber.html → A user-friendly HTML report showing test results.
This setup makes it easy to run all API tests and view clean, structured reports for quick analysis.
Once the framework is set up, you can execute your API automation suite directly from the command line using Maven. Maven handles compiling, running tests, and generating reports automatically.
Run All Tests –
To run all Cucumber feature files:
mvn clean test
clean → Deletes old compiled files and previous reports for a fresh run.
test → Executes all test scenarios defined in your project.
After running this command, Maven will trigger the Cucumber TestRunner, execute all scenarios, and generate reports in the test-output folder.
Run Tests by Tag –
Tags allow you to selectively run specific test scenarios or features. You can add tags like @api1, @smoke, or @regression in your .feature files to categorize tests.
Example:
@api1
Scenario: Verify POST API creates a record successfully
Given User sends "POST" request to "/api/v1/create" ...
Then User verifies the response status code is 201
To execute only scenarios with a specific tag, use:
mvn clean test -Dcucumber.filter.tags="@api1"
The framework will run only those tests that have the tag @api1.
You can combine tags for more flexibility:
@api1 or @api2 → Runs tests with either tag.
@smoke and not @wip → Runs smoke tests excluding work-in-progress scenarios.
This is especially useful when running specific test groups in CI/CD pipelines.
View Test Reports
API Automation Testing Framerwork Report – After the execution, Cucumber generates detailed reports automatically in the test-output directory:
Cucumber.html → User-friendly HTML report showing scenario results and logs.
Cucumber.json → JSON format report for CI/CD integrations or analytics tools.
You can open the report in your browser:
project-root/test-output/Cucumber.html
This section gives testers a clear understanding of how to:
API automation testing framework ensures that backend services are functioning properly before the application reaches the end user. Therefore, by integrating Cucumber, RestAssured, and Playwright, we have built a flexible and maintainable test framework that:
Supports BDD style scenarios.
Handles token-based authentication automatically.
Provides reusable utilities for API calls.
Generates rich HTML reports for easy analysis.
This hybrid setup helps QA engineers achieve faster feedback, maintain cleaner code, and enhance the overall quality of the software.
I am a Jr. SDET Engineer with experience in Manual and Automation Testing (UI & API). Skilled in Selenium, Playwright, Cucumber, Java, Postman, and SQL for developing and executing automated test cases. Proficient with GitHub, Bitbucket, and CI/CD tools like GitHub Actions, with hands-on experience in Regression testing, Defect tracking, and delivering high-quality software solutions.
Manual Testing with Playwright MCP – Have you ever felt that a simple manual test should be less manual?
For years, quality assurance relied on pure human effort to explore, click, and record. But what if you could perform structured manual and exploratory testing, generate detailed reports, and even create test cases—all inside your Integrated Development Environment (IDE), using zero code?
I’ll tell you this: there’s a tool that can help us perform manual testing in a much more structured and easy way inside the IDE: Playwright MCP.
Section 1: End the Manual Grind – Welcome to AI-Augmented QA
The core idea is to pair a powerful AI assistant (like GitHub Copilot) with a tool that can control a real browser (Playwright MCP). This simple setup is done in only a few minutes.
The Essential Setup for Manual Testing with Playwright MCP: Detailed Steps
For this setup, you will integrate Playwright MCP as a tool that your AI agent can call directly from VS Code.
1. Prerequisites (The Basics)
VS Code installed in your system.
Node.js (LTS version recommended) installed on your machine.
2. Installing GitHub Copilot (The AI Client)
Open Extensions: In VS Code, navigate to the Extensions view (Ctrl+Shift+X or Cmd+Shift+X).
Search and Install: Search for “GitHub Copilot” and “GitHub Copilot Chat” and install both extensions.
Authentication: Follow the prompts to sign in with your GitHub account and activate your Copilot subscription.
GitHub Copilot is an AI-powered code assistant that acts almost like an AI pair programmer.
After successful installation and Authentication, you see something like below
3. Installing the Playwright MCP Server (The Browser Tool)
Playwright MCP (Model Context Protocol): This is the bridge that provides browser automation capabilities, enabling the AI to interact with the web page.
The most direct way to install the server and configure the agent is via the official GitHub page:
Navigate to the Source: Open your browser and search for the Playwright MCP Server official GitHub page (https://github.com/microsoft/playwright-mcp).
The One-Click Install: On the GitHub page, look for the Install Server VSCode button.
Launch VS Code: Clicking this button will prompt you to open Visual Studio Code.
Final Step: Inside VS Code, select the “Install server” option from the prompt to automatically add the MCP entry to your settings.
To verify successful installation and configuration, follow these steps:
Click on “Configure Tool” icon
After clicking on the “configure tool “ icon, you see the tools of Playwright MCP as shown in the below image.
After clicking on the “Settings” icon, you see the “Configuration (JSON)” file of Playwright MCP, where you start, stop, and restart the server as shown in image below
After the Playwright MCP Server is successfully configured and installed, you will see the output as shown below.
2. Stop and Restart Server
This complete setup allows the Playwright MCP Server to act as the bridge, providing browser automation capabilities and enabling the GitHub Copilot Agent to interact with the web page using natural language.
Section 2: Phase 1: Intelligent Exploration and Reporting
The first, most crucial step is to let the AI agent, powered by the Playwright MCP, perform the exploratory testing and generate the foundational report. This immediately reduces the tester’s documentation effort.
Instead of manually performing steps, you simply give the AI Agent your test objective in natural language.
The Exploration Workflow:
Exploration Execution: The AI uses discrete Playwright MCP tools (like browser_navigate, browser_fill, and browser_click) to perform each action in a real browser session.
Report Generation: Immediately following execution, the AI generates an Exploratory Testing Report. This report is generated on the basis of the exploration, summarizing the detailed steps taken, observations, and any issues found.
Our focus is simple: Using Playwright MCP, we reduce the repetitive tasks of a Manual Tester by automating the recording and execution of manual steps.
Execution Showcase: Exploration to Report
Input (The Prompt File for Exploration)
This prompt directs the AI to execute the manual steps and generate the initial report.
Prompt for Exploratory Testing
Exploratory Testing: (Use Playwright MCP)
Navigate to https://www.demoblaze.com/. Use Playwright MCP Compulsory for Exploring the Module <Module Name> and generate the Exploratory Testing Report in a .md file in the Manual Testing/Documentation Directory.
Output (The Generated Exploration Report) The AI generates a structured report summarizing the execution.
Live Browser Snapshot from Playwright MCP Execution
Once the initial Exploration Report is generated, QA teams move to design specific, reusable assets based on these findings.
1. Test Case Design (on basis of Exploration Report)
The Exploration Report provides the evidence needed to design formal Test Cases. The report’s observations are used to create the Expected Results column in your CSV or Test Management Tool.
The focus is now on designing reusable test cases, which can be stored in a CSV format.
These manually designed test cases form the core of your execution plan.
We need to provide the Exploratory Report for References at the time of design test Cases.
Drag and drop the Exploratory Report File as context as shown in the image below.
Input (Targeted Execution Prompt)
This prompt instructs the AI to perform a single, critical verification action from your Test Case.
Role: Act as a QA Engineer.
Based on Exploratory report Generate the Test cases in below of Format of Test Case Design Template
=======================================
🧪 TEST CASE DESIGN TEMPLATE For CSV File
=======================================
Test Case ID – Unique identifier for the test case (e.g., TC_001)
Test Case Title / Name – Short descriptive name of what is being tested
Preconditions / Setup – Any conditions that must be met before test execution
Test Data – Input values or data required for the test
Test Steps – Detailed step-by-step instructions on how to perform the test
Expected Result – What should happen after executing the steps
Actual Result – What happened (filled after execution)
Status – Pass / Fail / Blocked (result of the execution)
Priority – Importance of the test case (High / Medium / Low)
Severity – Impact level if the test fails (Critical / Major / Minor)
Test Type – (Optional) e.g., Functional, UI, Negative, Regression, etc.
Execution Date – (Optional) When the test was executed
Executed By – (Optional) Name of the tester
Remarks / Comments – Any additional information, observations, or bugs found
Output (The Generated Test cases)
The AI generates structured test cases.
2. Test Plan Creation
The created test cases are organized into a formal Test Plan document, detailing the scope, environment, and execution schedule.
Input (Targeted Execution Prompt)
This prompt instructs the AI to perform a single, critical verification action from your Test Case. 2
Role: Act as a QA Engineer. - Use clear, professional language. - Include examples where relevant. - Keep the structure organized for documentation. - Format can be plain text or Markdown. - Assume the project is a web application with multiple modules. generate Test Cases in Form Of <Module Name >.txt in Manual Testing/Documentation Directory Instructions for AI: - Generate a complete Test Plan for a software project For Our Test Cases - Include the following sections: 1. Test Plan ID 2. Project Name 3. Module/Feature Overview 4. Test Plan Description 5. Test Strategy (Manual, Automation, Tools) 6. Test Objectives 7. Test Deliverables 8. Testing Schedule / Milestones 9. Test Environment 10. Roles & Responsibilities 11. Risk & Mitigation 12. Entry and Exit Criteria 13. Test Case Design Approach 14. Metrics / Reporting 15. Approvals
Output (The Generated Test plan)
The AI generates structured test plan of designed test cases.
3. Test Cases Execution
This is where the Playwright MCP delivers the most power: executing the formal test cases designed in the previous step.
Instead of manually clicking through the steps defined in the Test Plan, the tester uses the AI agent to execute the written test case (e.g., loaded from the CSV) in the browser.
The Playwright MCP ensures the execution of those test cases is fast, documented, and accurate.
Any failures lead to immediate artifact generation (e.g., defect reports).
Input (Targeted Execution Prompt)
This prompt instructs the AI to perform a single, critical verification action from your Test Case.
Use Playwright MCP to Navigate “https://www.demoblaze.com/” and Execute Test Cases attached in context and Generate Test Execution Report.
First, Drag and drop the test case file for references as shown in the image below.
Live Browser Snapshot from Playwright MCP Execution
Output (The Generated Test Execution report)
The AI generates structured test execution report of designed test cases.
4. Defect Reporting and Tracking
If a Test Case execution fails, the tester immediately leverages the AI Agent and Playwright MCP to generate a detailed defect report, which is a key task in manual testing.
Execution Showcase: Formal Test Case Run (with Defect Reporting)
We will now execute a Test Case step, intentionally simulating a failure to demonstrate the automated defect reporting capability.
Input (Targeted Execution Prompt for Failure)
This prompt asks the AI to execute a check and explicitly requests a defect report and a screenshot if the assertion fails.
Refer to the test cases provided in the Context and Use Playwright MCP to execute the test, and if there is any defect, then generate a detailed defect Report. Additionally, I would like a screenshot of the defect for evidence.
Output (The Generated Defect report and Screenshots as Evidence)
The AI generates a structured defect report of designed test cases.
Conclusion: Your Role is Evolving, Not Ending
Manual Testing with Playwright MCP is not about replacing the manual tester; it’s about augmenting their capabilities. It enables a smooth, documented, and low-code way to perform high-quality exploratory testing with automated execution.
Focus on Logic: Spend less time on repetitive clicks and more time on complex scenario design.
Execute Instantly: Use natural language prompts to execute tests in the browser.
Generate Instant Reports: Create structured exploratory test reports from your execution sessions.
Future-Proof Your Skills: Learn to transition seamlessly to an AI-augmented testing workflow.
It’s time to move beyond the traditional—set up your Playwright MCP today and start testing with the power of an AI-pair tester!
QA Engineer exploring the future of AI in software testing. Working with Playwright, modern automation frameworks, LLMs, and Agentic AI to build smarter, efficient testing solutions. Interested in AI-driven automation, intelligent QA systems, self-healing tests, and modern testing architectures.
Automation always comes with surprises. Recently, I stumbled upon one such challenge while working on a scenario that required automating PDF download using Playwright to verify a PDF download functionality. Sounds straightforward, right? At first, I thought so too. But the web application I was dealing with had other plans.
The Unexpected Complexity
Instead of a simple file download, the application displayed the report PDF inside an iframe. Looking deeper, I noticed a blob source associated with the PDF. Initially, it felt promising—maybe I could just fetch the blob and save it. But soon, I realized the blob didn’t actually contain the full PDF file. It only represented the layout instructions, not the content itself.
Things got more interesting (and complicated) when I found out that the entire PDF was rendered inside a canvas. The content wasn’t static—it was dynamically displayed page by page. This meant I couldn’t directly extract or save the file from the DOM.
At this point, downloading the PDF programmatically felt like chasing shadows.
The Print Button Dilemma
To make matters trickier, the only straightforward option available on the page was the print button. Clicking it triggered the system’s file explorer dialog, asking me to manually pick a save location. While that works fine for an end-user, for automation purposes it was a dealbreaker.
I didn’t want my automation scripts to depend on manual interaction. The whole point of this exercise was to make the process seamless and repeatable.
Digging Deeper: A Breakthrough
After exploring multiple dead ends, I finally turned my focus back to Playwright itself. That’s when I discovered something powerful—Playwright’s built-in capability to generate PDFs directly from a page.
The key was:
Wait for the report to open in a new tab (triggered by the app after selecting “Print View”).
Bring this new page into focus and make sure all content was fully rendered.
Use Playwright’s page.pdf() function to export the page as a properly styled PDF file.
The Solution in Action
Here’s the snippet that solved it:
// Wait for new tab to open and capture it
const [newPage] = await Promise.all([
context.waitForEvent("page"),
event.Click("(//span[text()='OK'])[1]", page), // triggers tab open
]);
global.secondPage = newPage;
await global.secondPage.bringToFront();
await global.secondPage.waitForLoadState("domcontentloaded");
// Use screen media for styling
await global.secondPage.emulateMedia({ media: "screen" });
// Path where you want the file saved
const downloadDir = path.resolve(__dirname, "..", "Downloads", "Reports");
if (!fs.existsSync(downloadDir)) fs.mkdirSync(downloadDir, { recursive: true });
const filePath = path.join(downloadDir, "report.pdf");
// Save as PDF
await global.secondPage.pdf({
path: filePath,
format: "A4",
printBackground: true,
margin: {
top: "1cm",
bottom: "1cm",
left: "1cm",
right: "1cm",
},
});
console.log(`✅ PDF saved to: ${filePath}`);
Key Highlights of the Implementation
Capturing the New Tab The Print/PDF Report option opened the report in a new browser tab. Instead of losing control, we captured it with context.waitForEvent(“page”) and stored it in a global variable global.secondPage. This ensured smooth access to the report tab for further processing.
Switching to Print View The dropdown option was switched to Print View to ensure the PDF was generated in the correct layout before proceeding with export.
Emulating Screen Media To preserve the on-screen styling (instead of print-only styles), we used page.emulateMedia({ media: “screen” }). This allowed the generated PDF to look exactly like what users see in the browser.
Saving the PDF to a Custom Path A custom folder structure was created dynamically using Node.js path and fs modules. The PDFs were named systematically and stored under Downloads/ImageTrend/<date>/, ensuring organized storage.
Full-Page Export with Print Background Using Playwright’s page.pdf() method, we captured all pages of the report (not just the visible one), along with background colors and styles for accurate representation.
Clean Tab Management Once the PDF was saved, the secondary tab (global.secondPage) was closed, bringing the focus back to the original tab for processing the next incident report.
What I Learned
This challenge taught me something new: PDFs in web apps aren’t always what they seem. Sometimes they’re iframes, sometimes blob objects, and in trickier cases, dynamically rendered canvases. Trying to grab the raw file won’t always work.
But with Playwright, there’s a smarter way. By leveraging its ability to generate PDFs from a live-rendered page, I was able to bypass the iframe/blob/canvas complexity entirely and produce consistent, high-quality PDF files.
Conclusion:
What started as a simple “verify PDF download” task quickly turned into a tricky puzzle of iframes, blobs, and canvases. But the solution I found—automating PDF download using Playwright with its built-in PDF generation—was not just a fix, it was an eye-opener.
It reminded me once again that automation isn’t just about tools; it’s about understanding the problem deeply and then letting the tools do what they do best.
This was something new I learned, and I wanted to share it with all of you. Hopefully, it helps the next time you face a similar challenge.
Python for Test Automation: Best Libraries and Frameworks. Indeed, automated testing is at the heart of modern software development, ensuring reliability, rapid delivery, and continuous improvement. Moreover, Python shines in this landscape, offering a mature ecosystem, ease of use, and tools that cater to every type of testing, from back-end APIs to eye-catching web UIs. Let’s dig deeper into the leading Python solutions for test automation, with code snippets and extra insights.
Specifically, Pytest is an open-source framework known for its elegant syntax, allowing developers to write tests using plain Python assert statements, and for its extensible design that accommodates unit, integration, and even complex functional test suites. Its fixture system allows reusable setup and teardown logic, making your tests both DRY (Don’t Repeat Yourself) and powerful. Additionally, a vast ecosystem of plugins supports reporting, parallelization, coverage, mocking, and more.
How it helps:
Plain assert syntax: Write readable tests without specialized assertions.
Powerful fixtures system: Enables reusable setup/teardown logic and dependency injection.
Parameterization: Run the same test with multiple inputs easily.
Plugin ecosystem: Extends capabilities (parallel runs, HTML reporting, mocking, etc.).
Auto test discovery: Finds tests in files and folders automatically.
What makes it useful:
Extremely easy for beginners, yet scalable for large and complex projects.
Fast feedback and parallel test execution.
Integrates well with CI/CD pipelines and popular Python libraries.
Large, active community and abundant documentation.
Meanwhile, Unittest, or PyUnit, is Python’s default, xUnit-inspired testing framework. It leverages class-based test suites and is included with Python by default, so there’s no installation overhead. Specifically, its structure—using setUp() and tearDown() methods—supports organized, reusable testing flows ideal for legacy systems or developers experienced with similar frameworks like JUnit.
How it helps:
Standard library: Ships with Python, zero installation required.
Class-based organization: Supports test grouping and reusability via inheritance.
Flexible test runners: Customizable, can generate XML results for CI.
Rich assertion set: Provides detailed validation of test outputs.
What makes it useful:
Good fit for legacy code or existing xUnit users.
Built-in and stable, making it ideal for long-term projects.
Well-structured testing process with setup/teardown methods.
Easy integration with other Python tools and editors.
import unittest
def add(a, b):
return a + b
class TestCalc(unittest.TestCase):
def setUp(self):
# Code to set up preconditions, if any
pass
def test_add(self):
self.assertEqual(add(2, 3), 5)
def tearDown(self):
# Cleanup code, if any
pass
if __name__ == '__main__':
unittest.main()
3. Selenium – World’s top Browser Automation tool
What it solves:
Selenium automates real browsers (Chrome, Firefox, Safari, and more); moreover, from Python, it simulates everything a user might do—clicks, form inputs, navigation, and more. Indeed, this framework is essential for end-to-end UI automation and cross-browser testing, and it integrates easily with Pytest or Unittest for reporting and assertions. Pair it with cloud services (such as Selenium Grid or BrowserStack) for distributed, real-device testing at scale.
How it helps:
Cross-browser automation: Supports Chrome, Firefox, Safari, Edge, etc.
WebDriver API: Simulates user interactions as in real browsers.
End-to-end testing: Validates application workflows and user experience.
Selectors and waits: Robust element selection and waiting strategies.
What makes it useful:
De facto standard for browser/UI automation.
Integrates with Pytest/Unittest for assertions and reporting.
Supports distributed/cloud/grid testing for broad coverage.
Community support and compatibility with cloud tools (e.g., BrowserStack).
4. Behave – Behavior-Driven Development (BDD) Framework
What it solves:
Behave lets you express test specs in Gherkin (Given-When-Then syntax), bridging the gap between technical and non-technical stakeholders. Ultimately, this encourages better collaboration and living documentation. Moreover, Behave is ideal for product-driven development and client-facing feature verification, as test cases are easy to read and validate against business rules.
How it helps:
Gherkin syntax: Uses Given/When/Then statements for business-readable scenarios.
Separation of concerns: Business rules (features) and code (steps) remain synced.
Feature files: Serve as living documentation and acceptance criteria.
What makes it useful:
Promotes collaboration between dev, QA, and business stakeholders.
Easy for non-coders and clients to understand and refine test cases.
Keeps requirements and test automation in sync—efficient for agile teams.
Feature: Addition
Scenario: Add two numbers
Given I have numbers 2 and 3
When I add them
Then the result should be 5
Step Definition
from behave import given, when, then
@given('I have numbers {a:d} and {b:d}')
def step_given_numbers(context, a, b):
context.a = a
context.b = b
@when('I add them')
def step_when_add(context):
context.result = context.a + context.b
@then('the result should be {expected:d}')
def step_then_result(context, expected):
assert context.result == expected
5. Robot Framework – Keyword-Driven and Extensible
What it solves:
Similarly, Robot Framework uses simple, human-readable, keyword-driven syntax to create test cases. Furthermore, it’s highly extensible, with libraries for web (SeleniumLibrary), API, database, and more, plus robust reporting and log generation. In particular, Robot is perfect for acceptance testing, RPA (Robotic Process Automation), and scenarios where non-developers need to write or understand tests.
How it helps:
Keyword-driven: Tests written in tabular English syntax, easy for non-coders.
*** Settings ***
Library SeleniumLibrary
*** Test Cases ***
Open Google And Check Title
Open Browser https://www.google.com Chrome
Title Should Be Google
Close Browser
6. Requests – HTTP for Humans
What it solves:
Python’s requests library is a developer-friendly HTTP client for RESTful APIs, and when you combine it with Pytest’s structure, you get a powerful and expressive way to test every aspect of an API: endpoints, status codes, headers, and response payloads. This pair is beloved for automated regression suites and contract testing.
How it helps:
Clean HTTP API: Requests library makes REST calls intuitive and readable.
Combine with Pytest: Gets structure, assertions, fixtures, and reporting.
Easy mocking and parameterization: Fast feedback for API contract/regression tests.
What makes it useful:
Rapid API test development and high maintainability.
Efficient CI integration for validating code changes.
Very flexible—supports HTTP, HTTPS, form data, authentication, etc.
Specifically, Locust is a modern load-testing framework that allows you to define user behavior in pure Python. Moreover, it excels at simulating high-traffic scenarios, monitoring system performance, and visualizing results in real time. Consequently, its intuitive web UI and flexibility make it the go-to tool for stress, spike, and endurance testing APIs or backend services.
How it helps:
Python-based user flows: Simulate realistic load scenarios as Python code.
Web interface: Live, interactive test results with metrics and graphs.
Distributed architecture: Scalable to millions of concurrent users.
What makes it useful:
Defines custom user behavior for sophisticated performance testing.
Real-time monitoring and visualization.
Lightweight, scriptable, and easy to integrate in CI pipelines.
from locust import HttpUser, task, between
class WebsiteUser(HttpUser):
wait_time = between(1, 3)
@task
def load_main(self):
self.client.get("/")
@task
def load_about(self):
self.client.get("/about")
@task
def load_contact(self):
self.client.get("/contact")
8. Allure and HTMLTestRunner – Reporting Tools
What it solves:
Visual reports are essential to communicate test results effectively. Notably, Allure generates clean, interactive HTML reports with test status, logs, screengrabs, and execution timelines—welcomed by QA leads and management alike. Similarly, HTMLTestRunner produces classic HTML summaries for unittest runs, showing pass/fail totals, stack traces, and detailed logs. These tools greatly improve visibility and debugging.
9. Playwright for Python – Modern Browser Automation
What it solves:
Playwright is a relatively new but powerful framework for fast, reliable web automation. It supports multi-browser, multi-context testing, handles advanced scenarios like network mocking and file uploads, and offers built-in parallelism for rapid test runs. Its robust architecture and first-class Python API make it a preferred choice for UI regression, cross-browser validation, and visual verification in modern web apps.
How it helps:
Multi-browser/multi-context: Automates Chromium, Firefox, and WebKit with a single API.
Auto-waiting and fast execution: Eliminates common flakiness in web UI tests.
from playwright.sync_api import sync_playwright
def test_example():
with sync_playwright() as p:
browser = p.chromium.launch(headless=True)
page = browser.new_page()
page.goto("https://example.com")
assert page.title() == "Example Domain"
browser.close()
Summary Table of Unique Features and Advantages
Every framework has a unique fit—pair them based on your team’s needs, tech stack, and test goals! Python libraries and frameworks for test automation.
Frameworks
Unique Features
Advantages
Pytest
Fixtures, plugins, assert syntax, auto discovery
Scalable, beginner-friendly, fast, CI/CD ready
Unittest
Std. library, class structure, flexible runner
Stable, built-in, structured
Selenium
Cross-browser UI/WebDriver, selectors, waits
UI/E2E leader, flexible, cloud/grid compatible
Behave
Gherkin/business syntax, feature/step separation
BDD, collaboration, readable, requirement sync
Robot Framework
Keyword-driven, extensible, RPA, reporting
Low code, reusable, logs, test visibility
Request
Simple API calls, strong assertions, fast feedback
Rapid API testing, CI ready, flexible
Locust
Python load flows, real-time web UI, scalable
Powerful perf/load, code-defined scenarios
Allure
Interactive HTML reports, attachments, logs
Stakeholder visibility, better debugging
Playwright
Multi-browser, auto-waiting, advanced scripting
Modern, fast, reliable, JS-app friendly
Conclusion
Python for Test Automation: Each of these frameworks has a unique niche, whether it’s speed, readability, extensibility, collaboration, or robustness. When selecting tools, consider your team’s familiarity, application complexity, and reporting/auditing needs—the Python ecosystem will almost always have a perfect fit for your automation challenge.
Indeed, the Python ecosystem boasts tools for every test automation challenge. Whether you’re creating simple smoke tests or orchestrating enterprise-grade BDD suites, there’s a Python library or framework ready to accelerate your journey. In fact, for every domain—unit, API, UI, performance, or DevOps pipeline, Python keeps testing robust, maintainable, and expressive.
I’m Sr. Digital Marketing Executive with a strong interest in content strategy, SEO, and social media marketing. She is passionate about building brand presence through creative and analytical approaches. In her free time, she enjoys learning new digital trends and exploring innovative marketing tools.
Pytest Vs Unittest: Testing forms the backbone of reliable software development, and in Python, two major frameworks stand out to get the job done: Unittest and Pytest. While both aim to ensure code correctness, maintainability, and robustness, they take very different approaches. Moreover, Python includes Unittest as a built-in framework, offering a familiar class-based testing style without requiring extra dependencies. Pytest, on the other hand, is a modern, feature-rich alternative that emphasizes simplicity, readability, and powerful capabilities like parametrization and fixtures.
In this blog, we’ll break down the key differences, advantages, and practical examples of both frameworks—helping you decide when to stick with the reliability of Unittest and when to embrace the flexibility of Pytest for your projects. Let’s see the Pytest vs Unittest: Which Python Testing Framework to Choose?
Step 1: Understanding the Fundamentals of Pytest Vs Unittest
What is Unittest?
Unittest comes bundled with Python as part of its standard library. Therefore, it ensures immediate availability and compatibility across different environments without requiring extra dependencies. Moreover, the seamless integration across environments makes Unittest convenient to use without the need for additional packages. To begin with, unit testing represents the first level of software testing, where testers examine the smallest parts of a program to ensure each unit functions as designed.
Example:
import unittest
class SimpleTest(unittest.TestCase):
def test_example(self):
self.assertTrue(True)
if __name__ == '__main__':
unittest.main()
For example, this is the basic test code using the Unittest framework, which contains a single test. This test() method will fail if True is ever false.
Output:
OOps concepts supported by unittest framework:
Text Fixture: A test fixture provides a baseline for running the tests. It basically provides the prerequisites needed for executing one or more tests and any clean up or temporary database generation running the process with all these functionality handled by text fixture.
Test Case: A set of cases defines the conditions that determine whether a system under test works correctly. It is a collection of unit tests
Test Suite: In addition, a test suite is a collection of test cases used to verify that a software program exhibits a specified set of behaviors by executing the aggregated tests together.
Test Runner: Similarly, a test runner is a component that sets up the execution of tests and provides the outcomes to the user. Furthermore, the runner may use a graphical interface, a text-based interface, or return a special value to indicate the results of executing tests.
Sample example of Unit test fixture:
import unittest
class SimpleTest(unittest.TestCase):
def setUp(self):
# This is the fixture. Runs before every test.
self.data = [1, 2, 3]
def tearDown(self):
# Clean up here (optional). Runs after every test.
self.data = None
def test_sum(self):
self.assertEqual(sum(self.data), 6)
def test_max(self):
self.assertEqual(max(self.data), 3)
if __name__ == '__main__':
unittest.main()
What is Pytest?
Overall, Pytest is a robust testing framework for Python that makes it easier to write simple and scalable test cases. In fact, Pytest’s simple syntax lets developers get started quickly with minimal boilerplate code. In addition, it supports fixtures, parametrization, and numerous plugins, making it a versatile and powerful tool for writing and organizing test cases.
Example:
import pytest
@pytest.mark.smoke
def test_function_one():
print('inside test function test_function_one')
num = 10
assert num !=12
Output:
Pytest Text Fixture:
Here’s a list of some of the most popular pytest fixtures you’ll often see used:
tmp_path / tmpdir: Provides a temporary directory unique to the test run.
monkeypatch: Allows you to modify or “patch” functions or environment variables for the duration of a test.
capfd / capsys: Captures output to file descriptors/stdout/stderr.
request: Gives access to the test context for parametrization, data, etc.
db (often custom): Sets up and tears down a database connection.
client: Creates a test client for web applications.
autouse fixtures: Moreover, Pytest automatically applies fixtures without requiring you to declare them in a test function.
parametrized fixtures: Moreover, you can deliver different values to tests using the same fixture code, enabling you to run tests against multiple inputs.
Step 3: Writing tests(Automation using Pytest Vs Unittest)
Writing tests using unittest
To begin with, create a project and add a Python package named business_logic. Inside this package, create two Python files: calculator.py and login.py.
Login.py:
USER = "Admin"
PASS = "Admin123"
def authenticate_user(username,password):
if username:
if password:
if USER==USER and PASS==password:
return 'Login Successful'
else:
return 'Invalid Credentials'
else:
return 'Password Cannot Be Empty...'
else:
return 'Username cannot be Empty...'
For example, the above simple code authenticates the user with a valid username and password. If the entered credentials match the predefined ‘Admin’ and ‘Pass’, the user successfully logs in to the application. If it’s not matching the criteria it will give a warning message popup.
Calculator.py:
def addition(n1,n2):
if type(n1) in [int,float,complex] and type(n2) in [int,float,complex]:
if n1<=0 or n2<=0:
return 'Number shud be greater than zero'
return n1+n2
else:
return 'Invalid Input'
In above code a simple calculator method is used for calculator additional functionality where n1 and n2 are could be [int,float,complex] if n1 or n2 are <=0 it will return warning popup message ‘Number shud be greater than zero’ and when n1 or n2>0 it will return addition of n1 and n2 else it will give warning popup message as ‘Invalid Input’.
Test_login_scenario.py:
import unittest
from business_logic.login import authenticate_user
class TestLogin(unittest.TestCase):
def test_valid_username_and_password(self):
if (authenticate_user('user0','pass0'))==True:
return True
print('inside test_valid_username_and_password')
def test_invalid_username_and_password(self):
print('Inside test_invalid_username_and_password')
self.assertEqual(10,20)
For instance, the above unit test verifies the login functionality for both positive and negative scenarios using Python’s built-in library.
Writing tests using pytest
from business_logic.calculator import addition
import pytest
import threading
@pytest.mark.parametrize("n1,n2,expected_result",[
(10,20,30),
(10,"A","Invalid Input"),
(0, "A", "Invalid Input"),
(0,10, "Number shud be greater than zero"),
(0,0,"Number shud be greater than zero"),
(10,-2,"Number shud be greater than zero"),
(2,4,6)
])
def test_calculator(n1,n2,expected_result):
print(n1,n2,expected_result)
result = addition(n1,n2)
assert result == expected_result
Similarly, in the above code, we have used Pytest parameterization to test the calculator’s addition functionality with the Pytest library.
Step 4: Run code through Command line
Unittest important commands:
Python -m unittest —> This is use to search entire test cases Example – python -m unittest tests.module.testclass
Python -m unittest -v test_module —> Here -v is used for more details
Python -m unittest -h —> -h is used for all command line help options
-f —> -f is used to stop the test run on the first error or failure
-k —> It is use to run the test methods and classes that matches the pattern or substring
Pytest important commands:
Pytest test_module() —> This is used to run tests in module
Pytest tests/ —> This is used to run tests in directory
f – failed
E – error
s – skipped
x – xfailed
X – xpassed
p – passed
P – passed with output
Step 5: Advantages of using Pytest and Unittest
Advanced Features of Unittest
Test discovery: Automatically finds and runs tests.
Test suites: Group multiple tests together.
Mocking capabilities: Use unittest.mock for mocking objects.
Advanced Features of Pytest
Parametrization: Easily run a test with multiple sets of parameters.
Plugins: A rich ecosystem of plugins to extend functionality.
Step 6: Key comparison between unittest and pytest
Aspect
Unittest
Pytest
Included with Python
Yes (Standard Libraries)
No (third-party package, install needed)
Syntax
More verbose, class-based
Simple, concise, function-based
Test Discovery
Requires strict naming and class structure
Automatic, flexible
Fixtures
Limited to setUp/tearDown methods
Powerful, modular fixtures with scopes
Parameterization
No built-in support (needs custom handling)
Built-in @pytest.mark.parametrize
Assertions
Assertion methods (e.g., assertEqual)
Plain assert with detailed introspection
Plugins
Few, limited support
Large rich ecosystem
Test Execution Speed
Sequential by default
Supports parallel execution
Mocking
Uses unittest.mock
Compatible with unittest.mock and plugins
Learning Curve
Easier for beginners
Moderate due to more features
Community
Standard library with stable adoption
Large and active community
Conclusion
Both Unittest and Pytest help you write reliable, maintainable tests—but they serve different needs. On the other hand, Unittest is lightweight, built-in, and well-suited for straightforward or legacy projects. In contrast, Pytest is modern, concise, and equipped with powerful features like fixtures, plugins, and parametrization—making it ideal for larger or more complex testing needs.
If you want simplicity with no extra setup, go with Unittest. If you want flexibility, readability, and speed, choose Pytest.
Jyotsna is a Jr SDET which have expertise in manual and automation testing for web and mobile both. She has worked on Python, Selenium, Mysql, BDD, Git, HTML & CSS. She loves to explore new technologies and products which put impact on future technologies.
