Automating OTP retrieval for testing with the Gmail API

I had to encounter a situation where I needed to do testing or automate a task but there was multi-factor authentication in place and I needed an automated way to retrieve OTPs for internal testing reasons. This is how you can handle that situation using the Gmail API — which is free to use for standard accounts and integrates well with Selenium test scripts.

Why use the Gmail API for automated testing

When you’re writing end-to-end tests that involve accounts protected by OTPs, manually retrieving codes breaks automation. The Gmail API provides a programmatic, auditable way to read test emails so your automation can continue, provided you follow best practices: use test accounts or domain-delegated service accounts, use OAuth properly.

High-level approach

• Create dedicated test email accounts (or use a G Suite / Google Workspace test domain). Never use real users’ accounts for automated OTP retrieval.
• Use OAuth2 or a service account with domain-wide delegation (for Workspace) to grant read access to the test inboxes — no password scraping, no bypassing MFA.
• Implement a small helper module that uses the Gmail API to search recent emails for the OTP message, parse the message body, extract the code, and return it to your test flow.
• Integrate with your Selenium tests so the test waits/polls the helper for the OTP, then proceeds to enter it into the UI.
• Audit and rotate credentials regularly and restrict scopes to https://www.googleapis.com/auth/gmail.readonly only.

Setup Steps

1. Enable Gmail API on Google Cloud Console.
2. Download your OAuth credentials JSON (credentials.json).
3. Place it in your project and define paths for:
   • CREDENTIALS_FILE → path to your OAuth client JSON
   • TOKEN_FILE → token file (created automatically after first login)
   • Install dependencies:pip install google-auth google-auth-oauthlib google-auth-httplib2 google-api-python-client beautifulsoup4 lxml
4. Run the script once manually — a browser window will open for authorization. A token file will be created and reused later for automated runs.

Key Idea

The approach is simple:

• Use Gmail API (with OAuth) to read your own test inbox.
• Query the inbox for recent messages from a specific sender and subject.
• Extract the OTP using regex patterns.
• Return the OTP to your test framework (e.g., Selenium).

This ensures secure and auditable access — no “bypassing” MFA — and allows you to test login workflows end-to-end.

Full Working Python Script

import os
import re
import base64
import datetime
from bs4 import BeautifulSoup
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.discovery import build

# Define constants
SCOPES = ['https://www.googleapis.com/auth/gmail.readonly']
CREDENTIALS_FILE = 'credentials.json'
TOKEN_FILE = 'token.json'  # Created automatically after first OAuth run

# Regex patterns for OTPs (adjust as needed)
OTP_PATTERNS = [
    r'\b(\d{6})\b',            # six digits
    r'\b(\d{4})\b',            # four digits
    r'\b(\d{5})\b',            # five digits
    r'code[:\s]*([0-9]{4,8})', # "code: 123456"
    r'one[-\s]?time[-\s]?password[:\s]*([0-9]{4,8})',
]

# Step 1: Authenticate and create Gmail API service

def get_gmail_service():
    creds = None
    if os.path.exists(TOKEN_FILE):
        creds = Credentials.from_authorized_user_file(TOKEN_FILE, SCOPES)
    if not creds or not creds.valid:
        if creds and creds.expired and creds.refresh_token:
            creds.refresh(Request())
        else:
            flow = InstalledAppFlow.from_client_secrets_file(CREDENTIALS_FILE, SCOPES)
            creds = flow.run_local_server(port=0)
        with open(TOKEN_FILE, 'w') as token:
            token.write(creds.to_json())
    return build('gmail', 'v1', credentials=creds)

# Step 2: Gmail query helpers

def search_messages(service, query, max_results=1):
    response = service.users().messages().list(userId='me', q=query, maxResults=max_results).execute()
    return response.get('messages', [])

def get_message(service, msg_id):
    return service.users().messages().get(userId='me', id=msg_id, format='full').execute()

# Step 3: Extract and decode email body

def extract_text_from_part(part):
    data = part.get('body', {}).get('data')
    if not data:
        return ''
    return base64.urlsafe_b64decode(data.encode('ASCII')).decode('utf-8', errors='ignore')

def get_message_body(msg):
    payload = msg.get('payload', {})
    mime_type = payload.get('mimeType', '')
    if mime_type == 'text/plain':
        return extract_text_from_part(payload)
    if mime_type == 'text/html':
        html = extract_text_from_part(payload)
        return BeautifulSoup(html, 'lxml').get_text(separator='\n')
    texts = []
    for part in payload.get('parts', []):
        ptype = part.get('mimeType', '')
        if ptype == 'text/plain':
            texts.append(extract_text_from_part(part))
        elif ptype == 'text/html':
            html = extract_text_from_part(part)
            texts.append(BeautifulSoup(html, 'lxml').get_text(separator='\n'))
    return '\n'.join(texts)

# Step 4: Regex-based OTP finder

def find_otp_in_text(text):
    for pat in OTP_PATTERNS:
        m = re.search(pat, text, re.IGNORECASE)
        if m:
            return m.group(1)
    return None

def pretty_headers(headers):
    return {h['name']: h['value'] for h in headers}

# Step 5: Main OTP extractor

def hs_otp():
    service = get_gmail_service()
    today_date = datetime.datetime.now().strftime("%Y/%m/%d")

    specific_sender = "no-reply@xyz.com"  # Replace with sender
    specific_subject = "Your OTP Code"     # Replace with subject

    query = f'from:{specific_sender} subject:"{specific_subject}" after:{today_date}'
    msgs = search_messages(service, query)

    if not msgs:
        print("No emails found from today matching the query.")
        return

    msg = get_message(service, msgs[0]['id'])
    headers = pretty_headers(msg.get('payload', {}).get('headers', []))
    body = get_message_body(msg)
    otp = find_otp_in_text(body)

    if otp:
        print(f"Extracted OTP: {otp}")
        return otp
    else:
        print("No OTP found in this email.")
        print(body[:500])
        return None 

Function Summary

Function	Purpose
get_gmail_service()	Authenticates via OAuth2 and returns a Gmail API client. Token is saved for reuse.
search_messages()	Queries inbox using Gmail search operators (from, subject, after, etc.).
get_message()	Fetches a full email by message ID.
get_message_body()	Decodes and extracts plain text or HTML content from an email.
find_otp_in_text()	Uses regex to locate OTP patterns (4–8 digits or “code: xxxx”).
hs_otp()	End-to-end helper that searches recent messages, parses them, and prints or returns the OTP.