Robotframework Python Testing using real email addresses
Python automation email testing Robotframework plugin
MailSlurp is a free Python SDK published on PYPI that lets you create real email addresses on demand. You can use it with Robot Framework to test user sign-up and email verification in real applications. View the example code on GitHub.
Introduction to Robotframework
Robot is a hugely popular integration testing framework with support for Python and Java. You can use Robot to test apps, software, websites and more. This tutorial will show you how to test a website sign-up process using real email addresses with MailSlurp and Robot.
What is Robot?
Robot is a little different to most other Python test frameworks like PyTest: it is keyword-driven or acceptance-test driven. That means you write tests using keywords instead of Python code. It is a bit like Cucumber from Ruby. Test files are written in
.robot
files using a custom syntax. We recommend finding a plugin for your editor that supports the robot file type to make testing easier.
What do tests look like?
Tests in RobotFramework are keyword driven and resemble sentences.
An example might look like so:
*** Test Cases ***
Sign Up With New Email Address
${inbox} Create Email Address
Open Browser To Home Page
Go To SignUp Page
NOTE: in Robot whitespace matters! Separate keywords and arguments with a tab or several whitespaces so that Robot can tell where keywords begin and end.
Every keyword or sentence we write inside a robot file needs to be defined by a plugin or by ourselves using a custom plugin. We will do this to integrate MailSlurp.
Demonstration project
For this tutorial we will test the sign-up process of a real application hosted at playground.mailslurp.com. This is a demonstration React App that uses AWS Firebase for authentication. After entering an email address and password during sign-up the user receives a confirmation code.
The confirmation code must be extracted from the email and entered into the confirmation form to proceed. Once confirmed a user can log in and see a welcome page with a picture of a friendly dog.
Project setup
First, let's setup a project using Python3.
mkdir robotframework-email-testing-python && cd -
Install dependencies
MailSlurp and Robot can be found on PyPI. Create a requirements.txt
file and include the following dependencies:
robotframework >= 4.0.0
robotframework-seleniumlibrary >= 5.1.3
mailslurp-client >= 11.5.10
webdrivermanager >= 0.10.0
Notice we have included robotframework, the robot selenium plugin, MailSlurp and the webdriver manager. Selenium is used for browser testing to open the MailSlurp playground and fill out the login forms. The webdrivermanager is used to download a webdriver to automate browser testing.
Install the dependencies like so:
pip3 install -r requirements.txt
Download webdrivers
Selenium needs a webdriver for firefox (or your browser of choice) to automate it. Let's use webdrivermanager to download geckodriver
like so:
python3 -m webdrivermanager firefox -d . -l SKIP
Then move the driver to the user PATH
sudo mv gecko/v*/gecko*/geckodriver /usr/bin/geckodriver
Setup tests
To setup a Robot test we need to create a few files.
touch test.robot
touch resource.robot
Create a MailSlurp plugin file
To create email addresses during testing we need to include a plugin file called MailSlurp.py
in the root directory.
touch MailSlurp.py
Inside the file include the following (we will explain it later):
import mailslurp_client
import re
# define a library to use mailslurp functions in robot test
class MailSlurp(object):
ROBOT_LIBRARY_VERSION = '1.0.0'
ROBOT_LIBRARY_SCOPE = 'GLOBAL'
# configure the mailslurp client using your API KEY
def __init__(self, api_key):
self.configuration = mailslurp_client.Configuration()
self.configuration.api_key['x-api-key'] = api_key
# function for creating an email address returns an inbox with an id and email_address
def create_inbox(self):
with mailslurp_client.ApiClient(self.configuration) as api_client:
# create an inbox using the inbox controller
api_instance = mailslurp_client.InboxControllerApi(api_client)
inbox = api_instance.create_inbox()
return inbox
# wait for and return an email in an inbox
def wait_for_latest_email(self, inbox_id):
with mailslurp_client.ApiClient(self.configuration) as api_client:
# create an inbox using the inbox controller
api_instance = mailslurp_client.WaitForControllerApi(api_client)
email = api_instance.wait_for_latest_email(inbox_id=inbox_id, timeout=60000, unread_only=True)
return email
# parse the confirmation code from an email body (adjust for your own use)
def extract_email_content(self, email_body):
regex = 'Your Demo verification code is ([0-9]{6})'
pattern = re.compile(regex)
matches = pattern.match(email_body)
content = matches.group(1)
return content
This is how you define a plugin in Robot so that you can create custom test keywords. Now we need to include it.
Create a resource file and include MailSlurp
For Robot tests we need to define our keywords. Typically that involves a resource.robot
file.
Add one that contains the following code:
# set up the tests
*** Settings ***
Documentation Define keywords and variables and import MailSlurp functions
Library SeleniumLibrary
Library ./MailSlurp.py ${MAILSLURP_API_KEY}
# define variables and default values
*** Variables ***
${MAILSLURP_API_KEY} PUT_YOUR_KEY_HERE
${SERVER} playground.mailslurp.com
${BROWSER} Firefox
${DELAY} 0
${PLAYGROUND URL} https://${SERVER}/
${TEST_PASSWORD} test-password
Above we defined library settings for Selenium and our custom functions. We also defined variables for use throughout our tests. Take note of the server url: https://playground.mailslurp.com hosts the functioning React web app that allows sign-up and user confirmation via email verification codes.
Include your API KEY
In the *** Settings ***
section we included two libraries: the Selenium Plugin and our custom MailSlurp file.
This line Library ./MailSlurp.py ${MAILSLURP_API_KEY}
instantiates the MailSlurp.py
file we created and passed the MAILSLURP_API_KEY
variable to the class constructor:
# configure the mailslurp client using your API KEY
def __init__(self, api_key):
self.configuration = mailslurp_client.Configuration()
self.configuration.api_key['x-api-key'] = api_key
MailSlurp is free to use but you need to sign-up to create an API KEY. When running your robot tests set the variable on the command line with the --variable
flag.
python3 -m robot --variable MAILSLURP_API_KEY:$(API_KEY) --outputdir results test.robot
Defining test methods
In Robot you define keywords and then use them to describe acceptance tests. Use the resource.robot
file to define the actions you want to test. For our demonstration app we can define these actions below:
# see https://robotframework.org/SeleniumLibrary/SeleniumLibrary.html for selenium keywords
*** Keywords ***
Create Email Address
${inbox} Create Inbox
[Return] ${inbox}
Wait For Confirmation Code
[Arguments] ${inbox_id}
${email} Wait For Latest Email ${inbox_id}
${code} Extract Email Content ${email.body}
[Return] ${code}
Open Browser To Home Page
Open Browser ${PLAYGROUND URL} ${BROWSER}
Maximize Browser Window
Set Selenium Speed ${DELAY}
Home Page Should Be Open
Home Page Should Be Open
Title Should Be React App
SignUp Page Should Be Open
Wait Until Element Contains //*[@data-test="sign-up-header-section"]//span Sign Up
Go To SignUp Page
Go To ${PLAYGROUND URL}
Home Page Should Be Open
Click Element //a[@data-test="sign-in-create-account-link"]
SignUp Page Should Be Open
Input Email
[Arguments] ${username}
Input Text //*[@name="email"] ${username}
Input Username
[Arguments] ${username}
Input Text //*[@name="username"] ${username}
Input Password
[Arguments] ${password}
Input Text //*[@name="password"] ${password}
Input Confirmation
[Arguments] ${code}
Input Text //*[@name="code"] ${code}
Submit Confirmation
Click Button //button[@data-test="confirm-sign-up-confirm-button"]
Submit Credentials
Click Button //button[@data-test="sign-up-create-account-button"]
Submit Login
Click Button //button[@data-test="sign-in-sign-in-button"]
Confirm Page Should Be Open
Wait Until Element Contains //*[@data-test="confirm-sign-up-header-section"]//span Confirm
SignIn Page Should Be Open
Wait Until Element Contains //*[@data-test="sign-in-header-section"]//span Sign in to your account
User Page Should Be Open
Wait Until Page Contains Welcome
Let's break that down a bit.
Querying pages with Selenium
The selenium plugin adds many keywords to our test suite. We can call them from the resource file to query the playground webapp. We can use css style selectors (xpath) in selenium like so:
Submit Login
Click Button //button[@data-test="sign-in-sign-in-button"]
This would find and click an HTML element that matches:
<button data-test="sign-in-sign-in-button">Submit</button>
The MailSlurp playground app uses [data-test]
attributes to allow for easy acceptance testing that doesn't rely on constant class or ID DOM attributes.
Calling custom Python functions
You may be thinking "where is the actual code in all this?". Let's get to that. You can call custom functions by including the library like we did in the resource.robot
file. Note: the file and class name must be the same. Robot translates method names to keywords by replacing _
underscores with spaces. So the def create_inbox(self):
function defining in MailSlurp.py
becomes the Create Inbox
keywords.
The MailSlurp create inbox function creates a real email address and returns an inbox entity with an ID and an email address.
def create_inbox(self):
with mailslurp_client.ApiClient(self.configuration) as api_client:
# create an inbox using the inbox controller
api_instance = mailslurp_client.InboxControllerApi(api_client)
inbox = api_instance.create_inbox()
return inbox
The code inside resource.robot
that makes this function and its return object available in tests is this:
*** Keywords ***
Create Email Address
${inbox} Create Inbox
[Return] ${inbox}
This is the custom robot syntax and it assigns the result of the create_inbox
function to a local variable before returning it. Inside a test we can call it like this:
*** Test Cases ***
Sign Up With New Email Address
${inbox} Create Email Address
Now that we can call MailSlurp and Selenium keywords let us write a test for the full sign up process.
Writing Robot tests
Create a file called test.robot
. Add the settings header to include our resource.robot
file so we can access all the keywords.
*** Settings ***
Documentation Test user sign-up with MailSlurp and Robot
Resource resource.robot
Our application flow
Next we need to test the user sign up and verification process. To recap, the process we wish to test is:
- create a unique email address for the test
- sign up for the playground app using the email address
- receive and extract a confirmation code via email
- submit the verification code and confirm user
- sign in to the app and view the happy dog
Our tests
Based on the custom functions in MailSlurp.py
and the keywords defined in resource.robot
we can write a test for the above functionality with the following statements:
*** Test Cases ***
Sign Up With New Email Address
${inbox} Create Email Address
Open Browser To Home Page
Go To SignUp Page
Input Email ${inbox.email_address}
Input Password ${TEST_PASSWORD}
Submit Credentials
Confirm Page Should Be Open
${code} Wait For Confirmation Code ${inbox.id}
Log To Console ${code} extacted
Input Confirmation ${code}
Submit Confirmation
SignIn Page Should Be Open
Input Username ${inbox.email_address}
Input Password ${TEST_PASSWORD}
Submit Login
User Page Should Be Open
[Teardown] Close Browser
Two important functions in this test are:
# wait for and return an email in an inbox
def wait_for_latest_email(self, inbox_id):
with mailslurp_client.ApiClient(self.configuration) as api_client:
# create an inbox using the inbox controller
api_instance = mailslurp_client.WaitForControllerApi(api_client)
email = api_instance.wait_for_latest_email(inbox_id=inbox_id, timeout=60000, unread_only=True)
return email
# parse the confirmation code from an email body (adjust for your own use)
def extract_email_content(self, email_body):
regex = 'Your Demo verification code is ([0-9]{6})'
pattern = re.compile(regex)
matches = pattern.match(email_body)
content = matches.group(1)
return content
The wait_for_latest_email
function is a unique MailSlurp feature that lets you wait for incoming emails and return the content. We then combine that with a regular expression pattern match to extract the confirmation code from an email of the pattern Your Demo verification code is 123456
. In this case the ([0-9]{6})
regular expression is a capture group for the 6 digits that are used as the confirmation code. The extract_email_content
returns this value for use in the next steps of the test.
Testing authentication end-to-end
Running the full test suite with python3 -m robot --variable MAILSLURP_API_KEY:your_api_key_here --outputdir results test.robot
will create a new email address ending in mailslurp.com
, sign up for the playground app, receive the email verification code, confirm the account and log into the app. For full usage see the Python documentation or the example code on GitHub.
Conclusion
Robot Framework and MailSlurp enabled acceptance testing of complex applications by generating real test email accounts. You can test user authentication and sign-up flows, lost password or email verification. See the MailSlurp docs for more information and good luck.
Related content
Java email client: send and receive emails and attachments i...
MailSlurp Java SDK for sending and receive email and attachments on the JVM.
Javascript email libraries for MailSlurp
Nodemailer alternatives to send and receieve email in code and tests
Python email API - send and receive email in code and tests
Send and receive email in Python, create test email accounts and more with the official PyPI MailSlurp package.
Ruby Mailer SDK - send and receive emails in Rails, Rspec an...
Receive email and attachments with Rails, Rspec, and Ruby without SMTP using MailSlurp Ruby Gem.
Using Email APIs to send and receive emails
Generate dynamic email addresses on demand with MailSlurp - a modern email API.
How to install Conda on Linux/OSX for python
Conda is a package management system that is popular in the machine learning and python space.
How to test an email address
Test email accounts for testing email addresses in code or online. Create fake email accounts for testing.
How to start selenium in a background process and wait for i...
Spawn Selenium server process before tests start for easier acceptance testing.
Stripe get all customers list (example)
Stripe get all customers list (example)
Send and receive email in Cypress JS tests with MailSlurp
Test email sign-up. password verification and more with Cypress JS and MailSlurp.
Cypress Email Plugin - Test email accounts in Javascript (an...
Use real email accounts in CypressJS to test user sign-up, email verification, and more.
Email APIs for Java and Kotlin projects
Test email sending and receive emails without a mail server.
Java TestNG Selenium user sign up testing with real email ad...
Testing user sign up in Java using TestNG and MailSlurp test email accounts
Codeception PHP acceptance testing using real email address ...
Write acceptance tests in PHP with real email addresses using Codeception and MailSlurp
PyTest email testing
Send and receive email in Pytest Python tests.
Selenium test email plugins: send and receive email in tests...
Receive emails in Java test suites using MailSlurp, Junit, and Selenium.
Robotframework Python Testing using real email addresses
Python automation email testing Robotframework plugin
Testing authentication using real email addresses in Ruby wi...
Cucumber example project using Capybara to test user authentication using real email addresses.
Test applications with real emails using Serenity BDD, JBeha...
Email acceptance testing with Serenity and MailSlurp. Test applications with real email addresses.
.NET SpecFlow testing using MailSlurp email accounts and Sel...
How to test .NET authentication and sign-up using real email accounts with MailSlurp and SpecFlow.
Test Emails in Selenium with DotNet, CSharp and MailSlurp
Send and receive email in DotNET Nunit tests using Selenium and MailSlurp.
Test emails with Cucumber and Ruby
Generate test email accounts with Ruby and Cucumber. Test email sign-up, password verification and more.
Test user sign-up with NodeJS and WebDriver
Test email related processes like sign-up and verification using WDIO WebDriver and MailSlurp.
TestCafe end-to-end MFA testing for user sign-up and email v...
End-to-end testing with MailSlurp, NodeJS, and TestCafe.
Test email address
Free test email address for testing emails online with web dashboard or REST API.
Testing email-related functionality with MailSlurp
Use MailSlurp to test email related functionality using real email addresses.
Testing email with Cypress test email accounts
Test email accounts for CypressJS. End-to-end testing with real email addresses using MailSlurp Cypress plugin.
Send SMTP email with Java
How to use Java SMTP client to send email with MailSlurp mail server on the JDK
Send SMTP email with Python (using SMTPLib)
How to send emails with Python's built-in SMTP client and MailSlurp mail servers.
Testing Email with Cypress JS and MailSlurp
Email testing with Cypress JS