In this Blog, I’ll walk you through the process of fetching an email link and OTP from Email using Python. Learn how to fetch links & OTP from email efficiently with simple steps. Email and OTP (One-Time Password) verification commonly ensure security and verify user identity in various scenarios.
Some typical scenarios include:
User Registration
Password Reset
Two-factor authentication (2FA)
Transaction Verification
Subscription Confirmation
We’ll leverage the imap_tools library to interact with Gmail’s IMAP server. We’ll securely manage our credentials using the dotenv library. This method is efficient and ensures that your email login details remain confidential.
Store Credentials Securely to Fetch OTP from Email:
A .env file is typically used to store environment variables. This file contains key-value pairs of configuration settings and sensitive information that your application needs to run but which you do not want to hard-code into your scripts for security and flexibility reasons.
Create a .env file in your project directory to store your Gmail credentials.
EMAIL_USER=your-email@gmail.com
EMAIL_PASS=your-password
How to Create and Use an App Password for Gmail
To securely fetch emails using your Gmail account in a Python script, you should use an App Password.
This is especially important if you have two-factor authentication (2FA) enabled on your account.
Here’s a step-by-step guide on how to generate an App Password in Gmail:
Go to your Google Account settings.
Select “Security” from the left-hand menu.
Enable Two-Factor Authentication:
Go to your Google Account Security Page.
Under the “Signing in to Google” section, ensure that 2-Step Verification is turned on. If it’s not enabled, click on “2-Step Verification” and follow the instructions to set it up.
Generate an App Password:
Once 2-step Verification is enabled, return to the Google Account Security Page.
Under the “Signing in to Google” section, you will now see an option for “App passwords.” Click on it.
You might be prompted to re-enter your Google account password.
In the “Select app” dropdown, choose “Mail” or “Other (Custom name)” and provide a name (e.g., “Python IMAP”).
In the “Select device” dropdown, choose the device you’re generating the password for, or select “Other (Custom name)” and enter a name (e.g., “My Computer”).
Click on “Generate.”
Google will provide you with a 16-character password. Note this password down securely, as you’ll need it for your Python script.
Load Environment Variables:
In your Python script, use the dotenv library to load these credentials securely. Here’s how you can do it:
from dotenv import load_dotenv
from imap_tools import MailBox, AND
import os
# Load .env file
load_dotenv()
# Read variables
email_user = os.getenv('EMAIL_USER')
email_pass = os.getenv('EMAIL_PASS')
Loading Environment Variables:
The dotenv library is used to load the email username and password from the .env file. This approach keeps your credentials secure and out of your source code.
Connect to Gmail and Fetch Emails:
We will create a function to connect to Gmail’s IMAP server and fetch the latest unread email. The function will look like this:
def check_latest_email():
# Connect to Gmail's IMAP server
with MailBox('imap.gmail.com').login(email_user, email_pass, 'INBOX') as mailbox:
# Fetch the latest unread email
emails = list(mailbox.fetch(AND(seen=False), limit=1, reverse=True))
if len(emails) == 0:
return None, None, None # No Emails Found
return emails[0]
if __name__ == "__main__":
email = check_latest_email()
if email:
print("Email subject: ", email.subject)
print("Email text: ", email.text)
print("Email from: ", email.from_)
else:
print("No new emails found.")
Connecting to Gmail’s IMAP Server:
Using the imap_tools library, we connect to Gmail’s IMAP server.
The MailBox class handles the connection.
The login method authenticates using your email and password.
Fetching the Latest Unread Email:
The fetch method retrieves emails based on specified criteria.
AND(seen=False) ensures we only get unread emails.
limit=1 fetches the latest one.
reverse=True sorts the emails in descending order.
Handling Email Data:
The function check_latest_email returns the most recent unread email’s subject, text, and sender.
If no new emails are found, it returns None.
By following these steps, you can efficiently fetch the latest unread email from your Gmail inbox using Python.
This method is not only secure but also straightforward, making it easy to integrate into your projects.
Fetching the link from email:
def extract_link(email_text):
# Regex pattern to match URLs
url_pattern = re.compile(r'https?://[^\s]+')
match = url_pattern.search(email_text)
if match:
return match.group()
return None
#Example to fetch link from email content:
link = extract_link(email.text)
if link:
print("Extracted Link: ", link)
else:
print("No link found in the email content.")
Fetching OTP from email:
Create a function to extract the OTP from the email content using a regular expression. This assumes the OTP is a 6-digit number, which is common for many services:
def extract_otp(email_text):
# Regex pattern to match a 6-digit number
otp_pattern = re.compile(r'\b\d{6}\b')
match = otp_pattern.search(email_text)
if match:
return match.group()
return None
#Example to extract otp from email
otp = extract_otp(email.text)
if otp:
print("Extracted OTP: ", otp)
else:
print("No OTP found in the email content.")
Refer to the following GitHub repository for instructions on how to fetch links and OTPs from Gmail.
Fetching links and OTPs from email content programmatically is essential for enhancing security, improving user experience, and increasing operational efficiency. Automation ensures timely and accurate processing, reducing the risk of errors and phishing attacks while providing a seamless user experience. This approach allows businesses to scale their operations, maintain compliance, and focus on strategic activities.
Click here for more blogs on software testing and test automation.
Harish is an SDET with expertise in API, web, and mobile testing. He has worked on multiple Web and mobile automation tools including Cypress with JavaScript, Appium, and Selenium with Python and Java. He is very keen to learn new Technologies and Tools for test automation. His latest stint was in TestProject.io. He loves to read books when he has spare time.
Understanding 2FA Authenticator Apps and Time-based One-Time Passwords (TOTP)
2FA Login using TOTP: In an era where cybersecurity is paramount, two-factor authentication (2FA) has become a cornerstone of online security. Authenticator apps and Time-based One-Time Passwords (TOTP) offer an additional layer of protection beyond traditional password-based security measures. This guide aims to elucidate the concept of 2FA authenticator apps, delve into the workings of TOTP, and provide insights into their importance in digital security.
What are 2FA Authenticator Apps?
Authenticator apps generate time-based one-time passwords (TOTPs) on mobile devices for authentication. They serve as a secure means of implementing 2FA, requiring users to provide both something they know (their password) and something they have (their mobile device with the authenticator app installed).
How Do 2FA Authenticator Apps Work?
Initialization: When setting up 2FA for an online account, users typically scan a QR code or manually enter a secret key provided by the service into their authenticator app.
Code Generation: Once initialized, the authenticator app generates TOTPs based on a shared secret key and the current time. These TOTPs are typically six-digit codes that change every 30 seconds.
Authentication: During login, users enter the current TOTP from their authenticator app along with their password. The service verifies the entered TOTP against the expected value based on the shared secret key.
Time-based One-Time Passwords (TOTP):
TOTP generates short-lived authentication codes using a shared secret key and the current time. The algorithm combines the secret key with the current time interval to produce a unique, one-time password that is valid only for a brief period, typically 30 seconds. TOTP ensures that even if an attacker intercepts a generated code, it quickly becomes obsolete, enhancing security.
Importance of Authenticator Apps and TOTP:
Enhanced Security: Authenticator apps provide an additional layer of security beyond passwords, significantly reducing the risk of unauthorized access to online accounts.
Protection Against Phishing: Because TOTP codes generate locally on your device and change with time, they resist phishing attacks that aim at stealing static passwords.
Convenience: Authenticator apps offer a convenient and user-friendly method of implementing 2FA, eliminating the need to rely on SMS-based authentication methods that may be vulnerable to SIM swapping attacks.
Authenticator apps and Time-based One-Time Passwords (TOTP) play a crucial role in safeguarding online accounts against unauthorized access and cyber threats. By incorporating 2FA with authenticator apps into their security protocols, individuals and organizations can significantly enhance their digital security posture in an increasingly interconnected world.
The protocol (otpauth) signals that an authenticator app should open this URL (DOMAIN).
the type (totp)
a label (USERNAME) that is a colon-separated combination of issuer and username
a secret (SECRET)
the issuer
The key element for logging in and automating the TOTP process is the supplied secret.
However, what’s the utilization method for this secret?
When you scan the QR code with an authenticator app, it combines the secret with the current time to create a unique password. The app then stores the second-factor secret, shows the issuer and username, and generates a new passcode every 30 seconds (though this time frame can be changed).That’s the gist of it!
You can then employ the passcode as a secondary login factor, providing assurance to services that it is indeed you are accessing your account.
Automating Login with Playwright for 2FA-Secured Websites
Automating the login process for websites secured with two-factor authentication (2FA) can streamline testing and administrative tasks. Playwright, a powerful automation library, provides the tools necessary to automate interactions with web pages, including those requiring 2FA.
Prerequisites:
Install Node.js: Make sure Node.js is installed on your system. You can download it from nodejs.org.
Set Up Playwright: Install Playwright using npm, the Node.js package manager, by running the following command in your terminal:
// Navigate to login page
await page.goto('https://example.com/login');
Enter Username and Password:
// Enter username and password
await page.fill('#username', 'your_username');
await page.fill('#password', 'your_password');
Click on the Login Button:
// Click on the login button
await page.click('#loginButton');
Handle 2FA Authentication:
Playwright supports interacting with elements on the page to handle 2FA. You can wait for the 2FA input field to appear and then fill it with the code.
The new OTPAuth.TOTP() syntax indicates that an instance of the TOTP class from the OTPAuth library is being created.
This instance is configured with various parameters such as issuer, label, algorithm, digits, period, and secret, which are used to generate and validate one-time passwords for authentication purposes.
In essence, new OTPAuth.TOTP() initializes a new TOTP (Time-based One-Time Password) object, allowing the application to generate OTPs for user authentication.
totp.generate() is a method call on the totp object. This method is provided by the OTPAuth library and is used to generate a one-time password (OTP) based on the TOTP algorithm.
The generate() method computes the current OTP value based on the TOTP parameters configured earlier, such as the secret key, time period, and algorithm. This OTP is typically used for user authentication purposes.
Once token holds the generated OTP, it can be used in the subsequent lines of code for filling an authentication code input field and submitting it for authentication.
Here’s the final code.
import * as OTPAuth from 'otpauth';
export class LoginPage {
readonly page: Page;
private readonly loginButton: Locator;
private readonly userNameInput: Locator;
private readonly passwordInput: Locator;
private readonly emailDisplay: Locator;
private userEmail: any;
private readonly authCode: Locator;
private readonly submitAuthCode: Locator;
constructor(page: Page) {
this.page = page;
this.loginButton = page.getByTestId(AppUniqueId.login_submitButton);
this.userNameInput = page.getByTestId(AppUniqueId.login_username);
this.passwordInput = page.getByTestId(AppUniqueId.login_password);
this.emailDisplay = page.getByTestId(AppUniqueId.personalDetails_email);
this.authCode = page.locator('#root > div > div > section > main > div > div > div > input');
this.submitAuthCode = page.locator('#root > div > div > section > main > div > div > button');
}
async waitFor(ms: number) {
return new Promise(resolve => setTimeout(resolve, ms));
}
async loginUserUI(childUser: User) {
let totp = new OTPAuth.TOTP({
issuer: "ISSUER",
label: "LABLE",
algorithm: "SHA1",
digits: 6,
period: 30,
secret: "XXXXXXXXXX”
});
await this.userNameInput.fill(await childUser.username);
await this.passwordInput.fill(await childUser.password);
await this.loginButton.click();
try{
let token = totp.generate()
await this.authCode.fill(token, { timeout: 5000});
await this.submitAuthCode.click()
}catch(err)
{
}
}
async verifyLoginUserEmail(childUser: User) {
this.userEmail = childUser.username;
try{
await this.page.locator("div[class='col-sm-12 d-flex impersonation-bar'] a").click()
}catch(err)
{
}
}
}
Conclusion:
Automating login for 2FA-secured websites using Playwright can enhance efficiency and productivity in various scenarios, from testing to administrative tasks. By following the steps outlined above, you can create robust automation scripts tailored to your specific requirements. https://github.com/apurvakolse/playwright-typescript-totp
Feel free to customize and expand upon these steps to suit your needs and the specific requirements of the website you’re working with.
Disclaimer: Ensure that you have the necessary permissions to automate interactions with websites, and use this knowledge responsibly and ethically.
Apurva is a Test Engineer, with 3+ years of experience in Manual and automation testing. Having hands-on experience in testing Web as well as Mobile applications in Selenium and Playwright BDD with Java. And loves to explore and learn new tools and technologies.
