Robotframework Python Testing using real email addresses
Python automation email testing Robotframework plugin
MailSlurp is a free Python SDK published on PYPI that lets you create real email addresses on demand. You can use it with Robot Framework to test user sign-up and email verification in real applications. View the example code on GitHub.
Introduction to Robotframework
Robot is a hugely popular integration testing framework with support for Python and Java. You can use Robot to test apps, software, websites and more. This tutorial will show you how to test a website sign-up process using real email addresses with MailSlurp and Robot.
What is Robot?
Robot is a little different to most other Python test frameworks like PyTest: it is keyword-driven or acceptance-test driven. That means you write tests using keywords instead of Python code. It is a bit like Cucumber from Ruby. Test files are written in .robot files using a custom syntax. We recommend finding a plugin for your editor that supports the robot file type to make testing easier.
What do tests look like?
Tests in RobotFramework are keyword driven and resemble sentences.
An example might look like so:
*** Test Cases ***
Sign Up With New Email Address
${inbox} Create Email Address
Open Browser To Home Page
Go To SignUp Page
NOTE: in Robot whitespace matters! Separate keywords and arguments with a tab or several whitespaces so that Robot can tell where keywords begin and end.
Every keyword or sentence we write inside a robot file needs to be defined by a plugin or by ourselves using a custom plugin. We will do this to integrate MailSlurp.
Demonstration project
For this tutorial we will test the sign-up process of a real application hosted at playground.mailslurp.com. This is a demonstration React App that uses AWS Firebase for authentication. After entering an email address and password during sign-up the user receives a confirmation code.
The confirmation code must be extracted from the email and entered into the confirmation form to proceed. Once confirmed a user can log in and see a welcome page with a picture of a friendly dog.
Project setup
First, let's setup a project using Python3.
mkdir robotframework-email-testing-python && cd -
Install dependencies
MailSlurp and Robot can be found on PyPI. Create a requirements.txt file and include the following dependencies:
robotframework >= 4.0.0
robotframework-seleniumlibrary >= 5.1.3
mailslurp-client >= 11.5.10
webdrivermanager >= 0.10.0
Notice we have included robotframework, the robot selenium plugin, MailSlurp and the webdriver manager. Selenium is used for browser testing to open the MailSlurp playground and fill out the login forms. The webdrivermanager is used to download a webdriver to automate browser testing.
Install the dependencies like so:
pip3 install -r requirements.txt
Download webdrivers
Selenium needs a webdriver for firefox (or your browser of choice) to automate it. Let's use webdrivermanager to download geckodriver like so:
python3 -m webdrivermanager firefox -d . -l SKIP
Then move the driver to the user PATH
sudo mv gecko/v*/gecko*/geckodriver /usr/bin/geckodriver
Setup tests
To setup a Robot test we need to create a few files.
touch test.robot
touch resource.robot
Create a MailSlurp plugin file
To create email addresses during testing we need to include a plugin file called MailSlurp.py in the root directory.
touch MailSlurp.py
Inside the file include the following (we will explain it later):
import mailslurp_client
import re
# define a library to use mailslurp functions in robot test
class MailSlurp(object):
ROBOT_LIBRARY_VERSION = '1.0.0'
ROBOT_LIBRARY_SCOPE = 'GLOBAL'
# configure the mailslurp client using your API KEY
def __init__(self, api_key):
self.configuration = mailslurp_client.Configuration()
self.configuration.api_key['x-api-key'] = api_key
# function for creating an email address returns an inbox with an id and email_address
def create_inbox(self):
with mailslurp_client.ApiClient(self.configuration) as api_client:
# create an inbox using the inbox controller
api_instance = mailslurp_client.InboxControllerApi(api_client)
inbox = api_instance.create_inbox()
return inbox
# wait for and return an email in an inbox
def wait_for_latest_email(self, inbox_id):
with mailslurp_client.ApiClient(self.configuration) as api_client:
# create an inbox using the inbox controller
api_instance = mailslurp_client.WaitForControllerApi(api_client)
email = api_instance.wait_for_latest_email(inbox_id=inbox_id, timeout=60000, unread_only=True)
return email
# parse the confirmation code from an email body (adjust for your own use)
def extract_email_content(self, email_body):
regex = 'Your Demo verification code is ([0-9]{6})'
pattern = re.compile(regex)
matches = pattern.match(email_body)
content = matches.group(1)
return content
This is how you define a plugin in Robot so that you can create custom test keywords. Now we need to include it.
Create a resource file and include MailSlurp
For Robot tests we need to define our keywords. Typically that involves a resource.robot file.
Add one that contains the following code:
# set up the tests
*** Settings ***
Documentation Define keywords and variables and import MailSlurp functions
Library SeleniumLibrary
Library ./MailSlurp.py ${MAILSLURP_API_KEY}
# define variables and default values
*** Variables ***
${MAILSLURP_API_KEY} PUT_YOUR_KEY_HERE
${SERVER} playground.mailslurp.com
${BROWSER} Firefox
${DELAY} 0
${PLAYGROUND URL} https://${SERVER}/
${TEST_PASSWORD} test-password
Above we defined library settings for Selenium and our custom functions. We also defined variables for use throughout our tests. Take note of the server url: https://playground.mailslurp.com hosts the functioning React web app that allows sign-up and user confirmation via email verification codes.
Include your API KEY
In the *** Settings *** section we included two libraries: the Selenium Plugin and our custom MailSlurp file.
This line Library ./MailSlurp.py ${MAILSLURP_API_KEY} instantiates the MailSlurp.py file we created and passed the MAILSLURP_API_KEY variable to the class constructor:
# configure the mailslurp client using your API KEY
def __init__(self, api_key):
self.configuration = mailslurp_client.Configuration()
self.configuration.api_key['x-api-key'] = api_key
MailSlurp is free to use but you need to sign-up to create an API KEY. When running your robot tests set the variable on the command line with the --variable flag.
python3 -m robot --variable MAILSLURP_API_KEY:$(API_KEY) --outputdir results test.robot
Defining test methods
In Robot you define keywords and then use them to describe acceptance tests. Use the resource.robot file to define the actions you want to test. For our demonstration app we can define these actions below:
# see https://robotframework.org/SeleniumLibrary/SeleniumLibrary.html for selenium keywords
*** Keywords ***
Create Email Address
${inbox} Create Inbox
[Return] ${inbox}
Wait For Confirmation Code
[Arguments] ${inbox_id}
${email} Wait For Latest Email ${inbox_id}
${code} Extract Email Content ${email.body}
[Return] ${code}
Open Browser To Home Page
Open Browser ${PLAYGROUND URL} ${BROWSER}
Maximize Browser Window
Set Selenium Speed ${DELAY}
Home Page Should Be Open
Home Page Should Be Open
Title Should Be React App
SignUp Page Should Be Open
Wait Until Element Contains //*[@data-test="sign-up-header-section"]//span Sign Up
Go To SignUp Page
Go To ${PLAYGROUND URL}
Home Page Should Be Open
Click Element //a[@data-test="sign-in-create-account-link"]
SignUp Page Should Be Open
Input Email
[Arguments] ${username}
Input Text //*[@name="email"] ${username}
Input Username
[Arguments] ${username}
Input Text //*[@name="username"] ${username}
Input Password
[Arguments] ${password}
Input Text //*[@name="password"] ${password}
Input Confirmation
[Arguments] ${code}
Input Text //*[@name="code"] ${code}
Submit Confirmation
Click Button //button[@data-test="confirm-sign-up-confirm-button"]
Submit Credentials
Click Button //button[@data-test="sign-up-create-account-button"]
Submit Login
Click Button //button[@data-test="sign-in-sign-in-button"]
Confirm Page Should Be Open
Wait Until Element Contains //*[@data-test="confirm-sign-up-header-section"]//span Confirm
SignIn Page Should Be Open
Wait Until Element Contains //*[@data-test="sign-in-header-section"]//span Sign in to your account
User Page Should Be Open
Wait Until Page Contains Welcome
Let's break that down a bit.
Querying pages with Selenium
The selenium plugin adds many keywords to our test suite. We can call them from the resource file to query the playground webapp. We can use css style selectors (xpath) in selenium like so:
Submit Login
Click Button //button[@data-test="sign-in-sign-in-button"]
This would find and click an HTML element that matches:
<button data-test="sign-in-sign-in-button">Submit</button>
The MailSlurp playground app uses [data-test] attributes to allow for easy acceptance testing that doesn't rely on constant class or ID DOM attributes.
Calling custom Python functions
You may be thinking "where is the actual code in all this?". Let's get to that. You can call custom functions by including the library like we did in the resource.robot file. Note: the file and class name must be the same. Robot translates method names to keywords by replacing _ underscores with spaces. So the def create_inbox(self): function defining in MailSlurp.py becomes the Create Inbox keywords.
The MailSlurp create inbox function creates a real email address and returns an inbox entity with an ID and an email address.
def create_inbox(self):
with mailslurp_client.ApiClient(self.configuration) as api_client:
# create an inbox using the inbox controller
api_instance = mailslurp_client.InboxControllerApi(api_client)
inbox = api_instance.create_inbox()
return inbox
The code inside resource.robot that makes this function and its return object available in tests is this:
*** Keywords ***
Create Email Address
${inbox} Create Inbox
[Return] ${inbox}
This is the custom robot syntax and it assigns the result of the create_inbox function to a local variable before returning it. Inside a test we can call it like this:
*** Test Cases ***
Sign Up With New Email Address
${inbox} Create Email Address
Now that we can call MailSlurp and Selenium keywords let us write a test for the full sign up process.
Writing Robot tests
Create a file called test.robot. Add the settings header to include our resource.robot file so we can access all the keywords.
*** Settings ***
Documentation Test user sign-up with MailSlurp and Robot
Resource resource.robot
Our application flow
Next we need to test the user sign up and verification process. To recap, the process we wish to test is:
- create a unique email address for the test
- sign up for the playground app using the email address
- receive and extract a confirmation code via email
- submit the verification code and confirm user
- sign in to the app and view the happy dog
Our tests
Based on the custom functions in MailSlurp.py and the keywords defined in resource.robot we can write a test for the above functionality with the following statements:
*** Test Cases ***
Sign Up With New Email Address
${inbox} Create Email Address
Open Browser To Home Page
Go To SignUp Page
Input Email ${inbox.email_address}
Input Password ${TEST_PASSWORD}
Submit Credentials
Confirm Page Should Be Open
${code} Wait For Confirmation Code ${inbox.id}
Log To Console ${code} extacted
Input Confirmation ${code}
Submit Confirmation
SignIn Page Should Be Open
Input Username ${inbox.email_address}
Input Password ${TEST_PASSWORD}
Submit Login
User Page Should Be Open
[Teardown] Close Browser
Two important functions in this test are:
# wait for and return an email in an inbox
def wait_for_latest_email(self, inbox_id):
with mailslurp_client.ApiClient(self.configuration) as api_client:
# create an inbox using the inbox controller
api_instance = mailslurp_client.WaitForControllerApi(api_client)
email = api_instance.wait_for_latest_email(inbox_id=inbox_id, timeout=60000, unread_only=True)
return email
# parse the confirmation code from an email body (adjust for your own use)
def extract_email_content(self, email_body):
regex = 'Your Demo verification code is ([0-9]{6})'
pattern = re.compile(regex)
matches = pattern.match(email_body)
content = matches.group(1)
return content
The wait_for_latest_email function is a unique MailSlurp feature that lets you wait for incoming emails and return the content. We then combine that with a regular expression pattern match to extract the confirmation code from an email of the pattern Your Demo verification code is 123456. In this case the ([0-9]{6}) regular expression is a capture group for the 6 digits that are used as the confirmation code. The extract_email_content returns this value for use in the next steps of the test.
Testing authentication end-to-end
Running the full test suite with python3 -m robot --variable MAILSLURP_API_KEY:your_api_key_here --outputdir results test.robot will create a new email address ending in mailslurp.com, sign up for the playground app, receive the email verification code, confirm the account and log into the app. For full usage see the Python documentation or the example code on GitHub.
Conclusion
Robot Framework and MailSlurp enabled acceptance testing of complex applications by generating real test email accounts. You can test user authentication and sign-up flows, lost password or email verification. See the MailSlurp docs for more information and good luck.