Appium and Python Visual Testing

Appium and Python Visual Testing – Have you ever wondered how your app’s toggle switches look on different devices? Visual elements like toggle buttons play a crucial role in the user experience, and verifying their color states is more than just cosmetic; it’s a matter of functionality, accessibility, and trust.

In this blog, we’ll explore how to verify toggle colors on real Android and iOS devices using Appium and Python—for visual testing, a practical guide for mobile automation testers who want to ensure their apps don’t just work, but look right too.

We’ll dive into:

Why is color detection essential in domains like e-commerce, healthcare, gaming, and automotive?
Three powerful techniques for verifying toggle states:
- Accessibility Identifiers
- Image Comparison
- Pixel-Level RGB Color Extraction

Step-by-step examples for both Android and iOS devices.

Importance of Color Detection in Visual Testing

Color detection plays a crucial role in image verification across various domains, where visual accuracy directly impacts user experience, brand integrity, and functionality. Below are some key applications:

E-commerce:
Accurate color representation of products is vital for online shopping platforms. Image verification ensures product photos match real-life appearances, reducing return rates and increasing customer trust.

Advertising and Marketing:
Consistent brand identity depends on precise color reproduction in ads, banners, and promotional content. Image testing helps maintain visual alignment with brand guidelines across different platforms and formats.
Gaming:
Visual elements like character designs, backgrounds, and effects contribute to the immersive quality of a game. Testing ensures that color schemes, contrasts, and visuals meet design standards and enhance gameplay.
Healthcare and Medical Imaging:
In medical diagnostics, accurate color detection in images like X-rays, MRIs, and pathology slides is critical. Image verification supports precise interpretation, leading to better patient outcomes.
Automotive:
Vehicle interfaces and design previews rely on color-accurate visuals. Testing ensures that dashboards, infotainment systems, and design prototypes reflect real-world colors and improve user experience.

So, let’s dive into verifying toggle colors on Android and iOS app step by step

Set your system for Appium and Python Visual Testing for real Android and IOS testing

For Android testing configuration, use the blog below for reference
How to configure Windows Desktop for Android App Automation using Appium

For IOS testing configuration, use the blog below for reference
How to configure macOS for iOS Mobile App Automation using Appium

Color detection ways in automation:

Using the Accessibility Identifiers: Utilizing accessibility identifiers (e.g., accessibility_id, content-desc, checked attribute from XPath) to determine the toggle’s state. These identifiers provide semantic information about the element, which is more reliable than relying solely on visual appearance.

Implementation: Use driver.find_element(AppiumBy.XPATH) or similar methods to locate the toggle based on its accessibility identifier. Check the element’s properties or state attributes (if available) to determine whether it’s “checked” or “unchecked.”

Advantages: Most reliable and maintainable approach, as it relies on semantic information rather than visual appearance.

Using Image Comparison: Capture a screenshot of the toggle in its “On” state and another in its “Off” state. Then, compare the screenshot of the actual toggle with the stored “On” and “Off” images.

Implementation: Use image comparison libraries like scikit-image or opencv-python to calculate similarity metrics (e.g., pixel-wise difference, structural similarity index). Determine the state based on the highest similarity score with the stored “On” or “Off” images. The below snippet will check whether the actual and ideal images are same or not.

img1 = imageio.imread(ideal_image_path)
img2 = imageio.imread(actual_image_path_repo_root)
if img1.shape != img2.shape:
  	 print("Both images should have the same dimensions")
  	 raise selenium.common.NoSuchElementException('No such element present')
diff = np.sum(np.abs(img1 - img2))
avg = diff / (img1.shape[0] * img1.shape[1] * img1.shape[2])
percentage = (avg / 255) * 100
	
if percentage == 0:
  	 return True
else:
 	  return False

In the above snippet, using opencv library from Python we are comparing the images first using the size of both the images, then calculating the average difference per Pixel for both the images.

Advantages: More robust to minor color variations compared to pixel color extraction.

Considerations: Maintaining a library of reference images for different toggle states is required.

Using Pixel Color Extraction:

The RGB (Red, Green, Blue) color model is one of the most widely used systems in digital image processing and display technologies. It is an additive color model, meaning it combines the intensity of these three primary colors(RGB) to create a broad spectrum of colors. Each color in this model is represented as a combination of Red, Green, and Blue values, ranging from 0 to 255 for 8-bit images.

For example:
- (255, 0, 0) represents pure red.
- (0, 255, 0) represents pure green.
- (0, 0, 255) represents pure blue.
- (255, 255, 255) represents white.
- (0, 0, 0) represents black.

How RGB Detection Works:

RGB detection involves extracting the Red (R), Green (G), and Blue (B) intensity values of individual pixels from digital media such as images or videos. Each pixel acts as a building block of the media, storing its color as a combination of these three values.

For image comparison in Python install pillow package using – from PIL import Image
Load the image – image = Image.open(‘example.jpg’)
Access the pixel at any location – rgb = image.getpixel((50, 50))

This will return the RGB value for that particular point. Open this website https://www.rapidtables.com/web/color/RGB_Color.html. Here you can find the color type according to RGB values, like if this method is returning the (255,215,0), which means it’s GOLD color.

By entering these values, you can find the color. Also like by entering 0,0,0 you can find the black color.

For demo purposes, let’s open the settings of android→connections→wifi toggle and check whether it’s turned ON or OFF.

Use code below for reference of color detection on a real Android device (Python Visual Testing)

Pre-setup for Android Device

Start Appium server using the below command, or you can use Appium GUI as well
- appium -a 127.0.0.1 -p 4723
Check connected adb devices using the below command, and you should be able to see a connected device with the device UDID
- adb devices
pip install Pillow

import time
from PIL import Image
import io

def app_init_for_android():
   from appium import webdriver
   from appium.webdriver.common.appiumby import AppiumBy

   desired_caps = {
       "appium:deviceName": "my_samsung",
       "appium:udid": "R9ZRskddjk0CHT",
       "platformName": "Android",
       "appium:platformVersion": "13",
       "appium:appPackage": "com.android.settings",  # Settings app package
       "appium:appActivity": "com.android.settings.Settings",  # Main Settings activity
       "automationName": "UiAutomator2"
   }

   # Initialize the Appium driver for the real iOS device
   driver = webdriver.Remote('http://127.0.0.1:4723', desired_caps)
   time.sleep(5)

   driver.find_element(
       AppiumBy.XPATH,
       "//androidx.recyclerview.widget.RecyclerView[@resource-id='com.android.settings:id/recycler_view']/android.widget.LinearLayout[2]"
   ).click()
   time.sleep(5)

   # To install pillow package use: pip install pillow
   element_1 = driver.find_element(AppiumBy.XPATH, "//android.widget.Switch[@content-desc='Wi-Fi']")
   time.sleep(5)
get_rgb_colors(element_1)
def get_rgb_colors(locator):
   element_location = locator.location
   element_size = locator.size

   screenshot = driver.get_screenshot_as_png()
   screenshot_image = Image.open(io.BytesIO(screenshot))
   width, height = screenshot_image.size
   center_x = width // 2
   center_y = height // 2 
   background_color = screenshot_image.getpixel((center_x , center_y))
   return background_color

Let’s break down the code

Install the required packages image, io, and time
Initialize the Android driver using the correct capability JASON
Inspect the locators to navigate to the wi-fi toggle present page
- You can refer blog below on how to find locators in Android and IOS devices – Blog on locator finding strategy in real Android and IOS devices
Find the center coordinates of the toggle using the locators method
Passing those coordinates to the getpixel method will give us the RGB value of that particular pixel
Open the website https://www.rapidtables.com/web/color/RGB_Color.html
Here you can find the color type according to RGB values

Use the code below for reference of colour detection on a real iOS device (Appium Visual Testing)

Pre-setup for iOS Device

Start Appium server using the below command, or you can use Appium GUI as well
- appium -a 127.0.0.1 -p 4723
pip install Pillow

We have to use build command to build our project and start our testing on real iOS. For IOS id→xcode→Window→device and simulators→Identifier
(e.g. –xcodebuild -project (path_for_WebDriverAgent.xcodeproj) -scheme WebDriverAgentRunner -destination ‘platform=iOS,id=(id_of_connected_ios) test)

For demo purposes let’s open settings of i-Phone→wifi settings

Consider the code below for color detection on iOS automation

import time
from PIL import Image
import io

def app_init_for_ios():
   from appium import webdriver
   from appium.webdriver.common.appiumby import AppiumBy

desired_caps =
{
   "platformName": "iOS",
   "platformVersion": "ios_devie_version",
   "deviceName": "ios_device_name",
   "udid": "ios_device_udid",
   "bundleId": "com.apple.Preferences",
   "automationName": "XCUITest",
   "xcodeSigningId": "iPhone Developer",
   "xcodeOrgId": "your_xcode_id",
   "autoAcceptAlerts": true,
   "newCommandTimeout": 10000
}

   # Initialize the Appium driver for the real iOS device
   driver = webdriver.Remote('http://127.0.0.1:4723', desired_caps)
   time.sleep(5)
driver.find_element(
       AppiumBy.XPATH,
       "//XCUIElementTypeStaticText[@name="WIFI"]"
   ).click()
   time.sleep(5)

   # To install pillow package use: pip install pillow
   element_1 = driver.find_element(AppiumBy.XPATH, "//XCUIElementTypeSwitch[@name="Wi‑Fi"]")
   time.sleep(5)
get_rgb_colors(element_1)
def get_rgb_colors(locator):
   element_location = locator.location
   element_size = locator.size

   screenshot = driver.get_screenshot_as_png()
   screenshot_image = Image.open(io.BytesIO(screenshot))
   width, height = screenshot_image.size
   center_x = width // 2
   center_y = height // 2 
   background_color = screenshot_image.getpixel((center_x , center_y))
   return background_color

Let’s break down the code – If you see the code, it’s similar to the Android color verification code The two key differences are like first one is the capabilities are different for iOS, and the locator finding strategy is different.

Conclusion

There are three primary methods for verifying toggle colors using Appium and Python Visual Testing

1. Accessibility Identifiers:

This is the most straightforward and reliable approach. Mobile apps often include labels or attributes (like accessibility_id or content-desc) that indicate the current state of a toggle. This method requires no image processing, as it leverages metadata provided by developers—making it both efficient and robust.

2. Image Comparison:

This technique involves capturing screenshots of the toggle in both “on” and “off” states and comparing them to reference images. Tools like OpenCV or scikit-image help analyze visual similarity, accounting for minor differences due to lighting or device variations. It’s especially useful when you need to validate the UI’s visual accuracy.

3. Pixel Color Extraction:

By extracting specific RGB values from toggle regions using libraries like Pillow, this method offers precision at the pixel level. It’s ideal for verifying exact color codes, and the extracted values can be cross-referenced with tools like RapidTables for further validation.
While Android and iOS may differ slightly in setup and element location, the core strategies remain consistent. Depending on your testing needs, you can use these methods individually or in combination to ensure your app displays the correct colors—ultimately contributing to a seamless and visually consistent user experience.

Click here to read more blogs like this.

Ajay Salunkhe

Junior Software Development Engineer in Test (JR. SDET) with 1 year of hands-on experience in automating and testing mobile applications using Python and Appium. Proficient in Selenium and Java, with a solid understanding of real device testing on both iOS and Android platforms. Adept at ensuring the quality and performance of applications through thorough manual and automated testing. Skilled in SQL and API testing using Postman.

Visual Testing: How to Verify Toggle Colors on Real Devices with Appium and Python

Importance of Color Detection in Visual Testing

Set your system for Appium and Python Visual Testing for real Android and IOS testing

Color detection ways in automation:

Using Pixel Color Extraction:

How RGB Detection Works:

Use code below for reference of color detection on a real Android device (Python Visual Testing)

Use the code below for reference of colour detection on a real iOS device (Appium Visual Testing)

For demo purposes let’s open settings of i-Phone→wifi settings

Conclusion

There are three primary methods for verifying toggle colors using Appium and Python Visual Testing

Recent Posts

Visit Us

Our Blogs

A Beginner’s Guide to Fast, Reliable Web Testing with CodeceptJS & Puppeteer

Cypress and TypeScript: A Dynamic Duo for Web Application & API Automation

Cypress Cucumber Framework: A Complete Guide to BDD Automation for Efficient Testing using JavaScript

Contact Us