Robotframework Python Testing using real email addresses

MailSlurp is a free Python SDK published on PYPI that lets you create real email addresses on demand. You can use it with Robot Framework to test user sign-up and email verification in real applications. View the example code on GitHub.

Introduction to Robotframework

Robot is a hugely popular integration testing framework with support for Python and Java. You can use Robot to test apps, software, websites and more. This tutorial will show you how to test a website sign-up process using real email addresses with MailSlurp and Robot.

What is Robot?

Robot is a little different to most other Python test frameworks like PyTest: it is keyword-driven or acceptance-test driven. That means you write tests using keywords instead of Python code. It is a bit like Cucumber from Ruby. Test files are written in .robot files using a custom syntax. We recommend finding a plugin for your editor that supports the robot file type to make testing easier.

What do tests look like?

Tests in RobotFramework are keyword driven and resemble sentences.

An example might look like so:

*** Test Cases ***
Sign Up With New Email Address
    ${inbox}    Create Email Address
    Open Browser To Home Page
    Go To SignUp Page

NOTE: in Robot whitespace matters! Separate keywords and arguments with a tab or several whitespaces so that Robot can tell where keywords begin and end.

Every keyword or sentence we write inside a robot file needs to be defined by a plugin or by ourselves using a custom plugin. We will do this to integrate MailSlurp.

Demonstration project

For this tutorial we will test the sign-up process of a real application hosted at playground.mailslurp.com. This is a demonstration React App that uses AWS Firebase for authentication. After entering an email address and password during sign-up the user receives a confirmation code.

The confirmation code must be extracted from the email and entered into the confirmation form to proceed. Once confirmed a user can log in and see a welcome page with a picture of a friendly dog.

Project setup

First, let's setup a project using Python3.

mkdir robotframework-email-testing-python && cd -

Install dependencies

MailSlurp and Robot can be found on PyPI. Create a requirements.txt file and include the following dependencies:

robotframework >= 4.0.0
robotframework-seleniumlibrary >= 5.1.3
mailslurp-client >= 11.5.10
webdrivermanager >= 0.10.0

Notice we have included robotframework, the robot selenium plugin, MailSlurp and the webdriver manager. Selenium is used for browser testing to open the MailSlurp playground and fill out the login forms. The webdrivermanager is used to download a webdriver to automate browser testing.

Install the dependencies like so:

pip3 install -r requirements.txt

Download webdrivers

Selenium needs a webdriver for firefox (or your browser of choice) to automate it. Let's use webdrivermanager to download geckodriver like so:

python3 -m webdrivermanager firefox -d . -l SKIP

Then move the driver to the user PATH

sudo mv gecko/v*/gecko*/geckodriver /usr/bin/geckodriver

Setup tests

To setup a Robot test we need to create a few files.

touch test.robot
touch resource.robot

Create a MailSlurp plugin file

To create email addresses during testing we need to include a plugin file called MailSlurp.py in the root directory.

touch MailSlurp.py

Inside the file include the following (we will explain it later):

import mailslurp_client
import re

# define a library to use mailslurp functions in robot test
class MailSlurp(object):
    ROBOT_LIBRARY_VERSION = '1.0.0'
    ROBOT_LIBRARY_SCOPE = 'GLOBAL'

    # configure the mailslurp client using your API KEY
    def __init__(self, api_key):
        self.configuration = mailslurp_client.Configuration()
        self.configuration.api_key['x-api-key'] = api_key

    # function for creating an email address returns an inbox with an id and email_address
    def create_inbox(self):
        with mailslurp_client.ApiClient(self.configuration) as api_client:
            # create an inbox using the inbox controller
            api_instance = mailslurp_client.InboxControllerApi(api_client)
            inbox = api_instance.create_inbox()
            return inbox

    # wait for and return an email in an inbox
    def wait_for_latest_email(self, inbox_id):
        with mailslurp_client.ApiClient(self.configuration) as api_client:
            # create an inbox using the inbox controller
            api_instance = mailslurp_client.WaitForControllerApi(api_client)
            email = api_instance.wait_for_latest_email(inbox_id=inbox_id, timeout=60000, unread_only=True)
            return email

    # parse the confirmation code from an email body (adjust for your own use)
    def extract_email_content(self, email_body):
        regex = 'Your Demo verification code is ([0-9]{6})'
        pattern = re.compile(regex)
        matches = pattern.match(email_body)
        content = matches.group(1)
        return content

This is how you define a plugin in Robot so that you can create custom test keywords. Now we need to include it.

Create a resource file and include MailSlurp

For Robot tests we need to define our keywords. Typically that involves a resource.robot file.

Add one that contains the following code:

# set up the tests
*** Settings ***
Documentation     Define keywords and variables and import MailSlurp functions
Library           SeleniumLibrary
Library           ./MailSlurp.py    ${MAILSLURP_API_KEY}

# define variables and default values
*** Variables ***
${MAILSLURP_API_KEY}    PUT_YOUR_KEY_HERE
${SERVER}               playground.mailslurp.com
${BROWSER}              Firefox
${DELAY}                0
${PLAYGROUND URL}       https://${SERVER}/
${TEST_PASSWORD}        test-password

Above we defined library settings for Selenium and our custom functions. We also defined variables for use throughout our tests. Take note of the server url: https://playground.mailslurp.com hosts the functioning React web app that allows sign-up and user confirmation via email verification codes.

Include your API KEY

In the *** Settings *** section we included two libraries: the Selenium Plugin and our custom MailSlurp file.

This line Library ./MailSlurp.py ${MAILSLURP_API_KEY} instantiates the MailSlurp.py file we created and passed the MAILSLURP_API_KEY variable to the class constructor:

# configure the mailslurp client using your API KEY
    def __init__(self, api_key):
        self.configuration = mailslurp_client.Configuration()
        self.configuration.api_key['x-api-key'] = api_key

MailSlurp is free to use but you need to sign-up to create an API KEY. When running your robot tests set the variable on the command line with the --variable flag.

python3 -m robot --variable MAILSLURP_API_KEY:$(API_KEY) --outputdir results test.robot

Defining test methods

In Robot you define keywords and then use them to describe acceptance tests. Use the resource.robot file to define the actions you want to test. For our demonstration app we can define these actions below:

# see https://robotframework.org/SeleniumLibrary/SeleniumLibrary.html for selenium keywords
*** Keywords ***
Create Email Address
    ${inbox}    Create Inbox
    [Return]    ${inbox}

Wait For Confirmation Code
    [Arguments]     ${inbox_id}
    ${email}    Wait For Latest Email   ${inbox_id}
    ${code}     Extract Email Content   ${email.body}
    [Return]    ${code}

Open Browser To Home Page
    Open Browser    ${PLAYGROUND URL}    ${BROWSER}
    Maximize Browser Window
    Set Selenium Speed    ${DELAY}
    Home Page Should Be Open

Home Page Should Be Open
    Title Should Be    React App

SignUp Page Should Be Open
    Wait Until Element Contains     //*[@data-test="sign-up-header-section"]//span   Sign Up

Go To SignUp Page
    Go To    ${PLAYGROUND URL}
    Home Page Should Be Open
    Click Element   //a[@data-test="sign-in-create-account-link"]
    SignUp Page Should Be Open

Input Email
    [Arguments]    ${username}
    Input Text     //*[@name="email"]    ${username}

Input Username
    [Arguments]    ${username}
    Input Text     //*[@name="username"]    ${username}

Input Password
    [Arguments]    ${password}
    Input Text    //*[@name="password"]    ${password}

Input Confirmation
    [Arguments]    ${code}
    Input Text    //*[@name="code"]    ${code}

Submit Confirmation
    Click Button    //button[@data-test="confirm-sign-up-confirm-button"]

Submit Credentials
    Click Button    //button[@data-test="sign-up-create-account-button"]

Submit Login
    Click Button    //button[@data-test="sign-in-sign-in-button"]

Confirm Page Should Be Open
    Wait Until Element Contains     //*[@data-test="confirm-sign-up-header-section"]//span   Confirm

SignIn Page Should Be Open
    Wait Until Element Contains     //*[@data-test="sign-in-header-section"]//span   Sign in to your account

User Page Should Be Open
    Wait Until Page Contains    Welcome

Let's break that down a bit.

Querying pages with Selenium

The selenium plugin adds many keywords to our test suite. We can call them from the resource file to query the playground webapp. We can use css style selectors (xpath) in selenium like so:

Submit Login
    Click Button    //button[@data-test="sign-in-sign-in-button"]

This would find and click an HTML element that matches:

<button data-test="sign-in-sign-in-button">Submit</button>

The MailSlurp playground app uses [data-test] attributes to allow for easy acceptance testing that doesn't rely on constant class or ID DOM attributes.

Calling custom Python functions

You may be thinking "where is the actual code in all this?". Let's get to that. You can call custom functions by including the library like we did in the resource.robot file. Note: the file and class name must be the same. Robot translates method names to keywords by replacing _ underscores with spaces. So the def create_inbox(self): function defining in MailSlurp.py becomes the Create Inbox keywords.

The MailSlurp create inbox function creates a real email address and returns an inbox entity with an ID and an email address.

def create_inbox(self):
    with mailslurp_client.ApiClient(self.configuration) as api_client:
        # create an inbox using the inbox controller
        api_instance = mailslurp_client.InboxControllerApi(api_client)
        inbox = api_instance.create_inbox()
        return inbox

The code inside resource.robot that makes this function and its return object available in tests is this:

*** Keywords ***
Create Email Address
    ${inbox}    Create Inbox
    [Return]    ${inbox}

This is the custom robot syntax and it assigns the result of the create_inbox function to a local variable before returning it. Inside a test we can call it like this:

*** Test Cases ***
Sign Up With New Email Address
    ${inbox}    Create Email Address

Now that we can call MailSlurp and Selenium keywords let us write a test for the full sign up process.

Writing Robot tests

Create a file called test.robot. Add the settings header to include our resource.robot file so we can access all the keywords.

*** Settings ***
Documentation     Test user sign-up with MailSlurp and Robot
Resource          resource.robot

Our application flow

Next we need to test the user sign up and verification process. To recap, the process we wish to test is:

• create a unique email address for the test
• sign up for the playground app using the email address
• receive and extract a confirmation code via email
• submit the verification code and confirm user
• sign in to the app and view the happy dog

Our tests

Based on the custom functions in MailSlurp.py and the keywords defined in resource.robot we can write a test for the above functionality with the following statements:

*** Test Cases ***
Sign Up With New Email Address
    ${inbox}    Create Email Address
    Open Browser To Home Page
    Go To SignUp Page
    Input Email       ${inbox.email_address}
    Input Password    ${TEST_PASSWORD}
    Submit Credentials
    Confirm Page Should Be Open
    ${code}    Wait For Confirmation Code   ${inbox.id}
    Log To Console      ${code} extacted
    Input Confirmation      ${code}
    Submit Confirmation
    SignIn Page Should Be Open
    Input Username    ${inbox.email_address}
    Input Password    ${TEST_PASSWORD}
    Submit Login
    User Page Should Be Open
    [Teardown]    Close Browser

Two important functions in this test are:

# wait for and return an email in an inbox
def wait_for_latest_email(self, inbox_id):
    with mailslurp_client.ApiClient(self.configuration) as api_client:
        # create an inbox using the inbox controller
        api_instance = mailslurp_client.WaitForControllerApi(api_client)
        email = api_instance.wait_for_latest_email(inbox_id=inbox_id, timeout=60000, unread_only=True)
        return email

# parse the confirmation code from an email body (adjust for your own use)
def extract_email_content(self, email_body):
    regex = 'Your Demo verification code is ([0-9]{6})'
    pattern = re.compile(regex)
    matches = pattern.match(email_body)
    content = matches.group(1)
    return content

The wait_for_latest_email function is a unique MailSlurp feature that lets you wait for incoming emails and return the content. We then combine that with a regular expression pattern match to extract the confirmation code from an email of the pattern Your Demo verification code is 123456. In this case the ([0-9]{6}) regular expression is a capture group for the 6 digits that are used as the confirmation code. The extract_email_content returns this value for use in the next steps of the test.

Testing authentication end-to-end

Running the full test suite with python3 -m robot --variable MAILSLURP_API_KEY:your_api_key_here --outputdir results test.robot will create a new email address ending in mailslurp.com, sign up for the playground app, receive the email verification code, confirm the account and log into the app. For full usage see the Python documentation or the example code on GitHub.

Conclusion

Robot Framework and MailSlurp enabled acceptance testing of complex applications by generating real test email accounts. You can test user authentication and sign-up flows, lost password or email verification. See the MailSlurp docs for more information and good luck.