How to handle Web tables using Java and Selenium Webdriver
What are Web Tables?
Web tables, also known as HTML tables, are a widely used format for displaying data on web pages. They allow for a structured representation of information in rows and columns, making it easy to read and manipulate data. Selenium WebDriver, a powerful tool for web browser automation, provides the functionality to interact with these tables programmatically. This capability is beneficial for tasks like web scraping, automated testing, and data validation. In this blog, we will see how to extract data from Web tables in Java-Selenium.
Identify web table from your webpage:
To effectively identify and interact with web tables using Selenium, it’s crucial to understand the HTML structure of tables and the specific tags used. Here’s an overview of the key table-related HTML tags
A typical HTML table consists of several tags that define its structure:
- <table>: The main container for the table.
- <thead>: Defines the table header, which contains header rows (<tr>).
- <tbody>: Contains the table body, which includes the data rows.
- <tr>: Defines a table row.
- <th>: Defines a header cell in a table row.
- <td>: Defines a standard data cell in a table row.
How to Identify Web Tables?
As we have got an idea of what is Web Table and how to identify WebTables on the webpage, now we will see how to extract the table data.
We will be using “https://www.globalsqa.com/angularJs-protractor/WebTable/”
As a demo website, here you will get a sample WebTable with fields like first name, last name, email, etc. Here we have applied a filter for email to minimize the size of the table.
We will be starting by launching the browser and navigating to the webpage. We have applied a filter for the email “PolGermain@whatever.com”, you can change it as per your requirement.
WebDriverManager.chromedriver().setup();
WebDriver driver =new ChromeDriver();
driver.get("https://www.globalsqa.com/angularJs-protractor/WebTable/");
driver.manage().window().maximize();
WebElement global_search = driver.findElement(By.xpath("//input[@type='search' and @placeholder='global search']"));
global_search.sendKeys("PolGermain@whatever.com");
global_search.sendKeys(Keys.ENTER);Once we get the filtered data from the table, now we need to locate the table and get the number of rows. The table will have multiple rows so, we need to use a list to store all the rows.
List<WebElement> row =driver.findElements(By.xpath("//table[@class='table table-striped']/tbody/tr"));As we have stored all the rows in the list, now we need to iterate through each rows to fetch the columns and store the column data in another list.
Example :
| Abc | 1 |
| Xyz | 2 |
When we are iterating through the 1st row we will get data as Abc and 1 and store it in the list ’as rowdata[Abc, 1] similarly data from the 2nd row will be stored as rowdata[Xyz, 2].When we are iterating through the 2nd row the data from the 1st row will be overwritten.
That’s why we will need one more list ‘webRows ’ to store all the rows.
In the below code snippet, here we are iterating through all the columns from each row one by one and finally storing all the rows in the list WebRows.
List<WebElement> row =driver.findElements(By.xpath("//table[@class='table table-striped']/tbody/tr"));
List<String> rowdata =new ArrayList<>();
for(int i=0;i<row.size();i++){
List<List<String>> webRows = new ArrayList<>();
List<WebElement> values = driver.findElements(By.xpath("//table[@class='table table-striped']/tbody/tr["+(i+1)+"]/td"));
for(int j=0;j<values.size();j++){
rowdata.add(values.get(j).getText());
}
System.out.println("rowdata--"+rowdata);
webRows.add(rowdata);
}How to access table data with the column index?
We have successfully extracted the table data now you can use this data as per your requirement
To do this we need to iterate through the list ‘webRows’ where we have our table data stored. We will be accessing all the columns by their index. In this case, you should know the column index you want to access. The column index always starts from 0.
for (int s = 0; s < webRows.size(); s++) {
List<String> row = webRows.get(s);
System.out.println(row.get(1));
System.out.println(row);
}Below is the complete code snippet for the above-mentioned steps. You need to update related Xpaths in case you are not able to access the rows and columns with the given Xpaths.
package Selenium;
import io.github.bonigarcia.wdm.WebDriverManager;
import org.openqa.selenium.*;
import org.openqa.selenium.chrome.ChromeDriver;
import java.util.ArrayList;
import java.util.List;
public class Webtable {
public static void main(String[] args) throws InterruptedException {
WebDriverManager.chromedriver().setup();
WebDriver driver =new ChromeDriver();
driver.get("https://www.globalsqa.com/angularJs-protractor/WebTable/");
driver.manage().window().maximize();
WebElement global_search = driver.findElement(By.xpath("//input[@type='search' and @placeholder='global search']"));
global_search.sendKeys("PolGermain@whatever.com");
//global_search.sendKeys("Pol");
global_search.sendKeys(Keys.ENTER);
Thread.sleep(3000);
List<WebElement> row =driver.findElements(By.xpath("//table[@class='table table-striped']/tbody/tr"));
List<List<String>> webRows = new ArrayList<>();
for(int i=0;i<row.size();i++){
List<String> rowdata =new ArrayList<>();
List<WebElement> values = driver.findElements(By.xpath("//table[@class='table table-striped']/tbody/tr["+(i+1)+"]/td"));
for(int j=0;j<values.size();j++){
rowdata.add(values.get(j).getText());
}
webRows.add(rowdata);
}
for (int s = 0; s < webRows.size(); s++) {
List<String> row1 = webRows.get(s);
System.out.println(row1);
System.out.println(row1.get(1));
}
}
}
When you execute the above code, you will get output in the below format
[Pol, Germain, 49, PolGermain@whatever.com, 1020.1597184937436]
Germain
[Pol, Germain, 62, PolGermain@whatever.com, 911.4520444579008]
Germain
[Pol, Germain, 10, PolGermain@whatever.com, 2809.911328973954]
Germain
Instead of accessing data by the index, you can access it using the column index also, and to do that you need to use the HashMaps instead of lists. HashMap will help to store column headers as keys and column data as values
Example:
| Name | Id |
| Abc | 1 |
| Xyz | 2 |
Here Name and ID will be your keys and Abc, 1 and Xyz, 2 will be the values.
How to store and access table data using HashMap?
The code snippet below shows how to use HashMap to store data in key-value format.
package Selenium;
import io.github.bonigarcia.wdm.WebDriverManager;
import org.openqa.selenium.*;
import org.openqa.selenium.chrome.ChromeDriver;
import java.util.ArrayList;
import java.util.List;
public class Webtable_Blog {
public static void main(String[] args) throws InterruptedException {
WebDriverManager.chromedriver().setup();
WebDriver driver = new ChromeDriver();
driver.get("https://www.globalsqa.com/angularJs-protractor/WebTable/");
driver.manage().window().maximize();
WebElement global_search = driver.findElement(By.xpath("//input[@type='search' and @placeholder='global search']"));
global_search.sendKeys("PolGermain@whatever.com");
// global_search.sendKeys("Pol");
global_search.sendKeys(Keys.ENTER);
Thread.sleep(5000);
List<WebElement> rows = driver.findElements(By.xpath("//table[@class='table table-striped']/tbody/tr"));
System.out.println("size-"+rows.size());
List<Map<String, String>> webRows = new ArrayList<>();
for (int i = 0; i < rows.size(); i++) {
List<WebElement> keys = driver.findElements(By.xpath("//table[@class='table table-striped']/thead/tr[1]/th"));
List<WebElement> values = driver.findElements(By.xpath("//table[@class='table table-striped']/tbody/tr["+(i+1)+"]/td"));
Map<String, String> webColumn = new HashMap<>();
try {
for (int j = 0; i < keys.size(); j++) {
webColumn.put(keys.get(j).getText(), values.get(j).getText());
}
} catch (Exception e) {
}
webRows.add(webColumn);
}
for (int s = 0; s < webRows.size(); s++) {
System.out.println(webRows.get(s).get("lastName"));
System.out.println(webRows.get(s));
}
}
}
Output-
size-4
Germain
{firstName=Pol, lastName=Germain, balance=1527.3558523201625, age=28, email=PolGermain@whatever.com}
Germain
{firstName=Pol, lastName=Germain, balance=250.18122282042322, age=20, email=PolGermain@whatever.com}
Germain
{firstName=Pol, lastName=Germain, balance=274.9486946306141, age=3, email=PolGermain@whatever.com}
Germain
{firstName=Pol, lastName=Germain, balance=1176.6629976866143, age=10, email=PolGermain@whatever.com}
Refer to the following GitHub repository for How to automate web tables in Java-Selenium.
https://github.com/priyanka1970/WebTables_with_Java_Selenium
Conclusion-
In this blog, we’ve delved into the powerful capabilities of Selenium WebDriver for handling web tables in Java. WebTables are a crucial part of web applications, often used to display large amounts of data in an organized manner. In Java Selenium, handling these WebTables efficiently is a key skill for any test automation engineer. Throughout this blog, we’ve explored various techniques to interact with WebTables, including locating tables, accessing rows and cells, iterating through table data, and performing actions like sorting and filtering.
Click here for more blogs on software testing and test automation.
Priyanka is an SDET with 2.5+ years of hands-on experience in Manual, Automation, and API testing. The technologies she has worked on include Selenium, Playwright, Cucumber, Appium, Postman, SQL, GitHub, and Java. Also, she is interested in Blog writing and learning new technologies.