Skip to content
MailSlurp logo

JMeter email load testing

Send and receive email using isolated test email accounts within JMeter stress tests

JMeter is a powerful load testing framework from Apache that lets you run large scale stress tests against networks and APIs. Sometimes these systems rely on email messaging, for instance user sign-up, bulk marketing, email blasts, and notification systems. When email is required we can use disposable test email accounts from MailSlurp to send and receive email during JMeter tests.

A test scenario

Say we have a sign-up process in our application that accepts a user email address and then sends a confirmation code. What happens when our system is under heavy load, say during a live event or promotion? Well to test this we can use JMeter. We define steps using either a GUI, XML, or Java. These steps should do the following:

  1. Create a disposable email account on MailSlurp API
  2. Call the sign-up endpoint and submit the email address
  3. Wait for a confirmation code to arrive confirming functionality
  4. Repeat many times in parallel for load test

Setup

If you don't have JMeter already download it and place in your project root. Unzip the files and execute the GUI using:

jmeter

This should show a user interface like so:

JMeter test plan interface for an email load test

Controlling inboxes

Use the MailSlurp API to create the inboxes your test controls. Size concurrency against the current inbox, message and API allowances for your account. The key endpoints are:

Base URL: https://api.mailslurp.com

Request Description
POST /inboxes Creates a new disposable inbox and returns its id + emailAddress.
GET /waitForLatestEmail?inboxId={INBOX_ID}&timeout=60000&unreadOnly=true Polls MailSlurp until the latest unread email arrives in that inbox.

Note: for each request we must append a MailSlurp API KEY as an x-api-key header:

MailSlurp API key header configured in JMeter

Creating the email load test

Let's go over the steps now needed to write the load test for our application sign up process.

1) Create email accounts in JMeter

To generate a new dummy mailbox make a POST request to POST /inboxes with your API KEY set to the x-api-key header (Base URL: https://api.mailslurp.com). This returns an ID and emailAddress we can store for the next steps:

read email jmeter

Make sure we set the headers to pass a valid MailSlurp API KEY (Get a free key here).

jmeter-inbox-post-headers.png

Next we need to store the returned JSON result so we can sign up with the email address and then read the emails:

jmeter-create-email-store.png

2) Sign up using email

Now that we have an email in our load test, use it to sign up and stress test our application. Here we will test a dummy app:

jmeter-test-signup.png

If our system is working correctly it will send an email to that email address that we can wait for using MailSlurp.

3) Wait for and read email in JMeter

Lastly, we can now use the inbox ID stored earlier to receive the sent email and verify our system is operational during load tests. For this we call GET /waitForLatestEmail and pass a timeout (ms) plus inboxId parameter. This holds the request until an email arrives in that inbox or the timeout expires. A latest-email wait does not by itself identify the expected subject or prove that the application code works; add assertions before counting the iteration as successful.

jmeter-wait-for-email.png

Then test the email is present:

jmeter-wait-for-assert.png

Make each parallel iteration own its result

Keep INBOX_ID and EMAIL_ADDRESS in JMeter's thread-local variables (vars), not global properties (props). Create a fresh inbox for each independent signup iteration and clear both variables before its create request. Give the JSON extractor a missing-value sentinel, then add an assertion that rejects it before the signup sampler. A failed create must not reuse the previous iteration's address.

For an expiring inbox, use POST /inboxes/withOptions with the JSON body {"expiresIn":600000}. After the iteration, issue DELETE /inboxes/${INBOX_ID} with the API key header. Expiry provides a fallback if a stopped thread never reaches its cleanup sampler; choose a lifetime longer than the expected test iteration.

Add a Groovy JSR223 Assertion as a child of the wait-for-email HTTP sampler. This example checks the HTTP result, subject and recipient. Replace the subject with your application's actual message:

import groovy.json.JsonSlurper

if (prev.getResponseCode() != '200') {
    AssertionResult.setFailure(true)
    AssertionResult.setFailureMessage('Email wait did not complete successfully')
} else {
    try {
        def email = new JsonSlurper().parseText(prev.getResponseDataAsString())
        def address = vars.get('EMAIL_ADDRESS')
        if (!address || email.subject != 'Verify your account' || !(email.to instanceof List) || !email.to.contains(address)) {
            AssertionResult.setFailure(true)
            AssertionResult.setFailureMessage('Expected signup email was not received')
        }
    } catch (Exception ignored) {
        AssertionResult.setFailure(true)
        AssertionResult.setFailureMessage('Email response was not valid JSON')
    }
}

Do not stop at receipt for an authentication load test. Extract the verification code or link, submit it to your application's confirmation endpoint, and assert the resulting account state. If one iteration receives several message types, replace the latest-email request with POST /waitForMatchingFirstEmail, providing the inbox ID, since, a bounded timeout and a subject match; the wait API reference describes that request.

Full JMX test example

The JMX below is a one-thread starting example against MailSlurp's demonstration application. Save it as EmailLoadTest.jmx, then adapt the application sampler, assertions, inbox expiry and cleanup described above before measuring your own service. Its demonstration endpoint and default thread count are not a production load-test configuration:

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.6.3">
  <hashTree>
    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="MailSlurp Magic-Link Flow">
      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments">
        <collectionProp name="Arguments.arguments"/>
      </elementProp>
    </TestPlan>
    <hashTree>
      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Users">
        <intProp name="ThreadGroup.num_threads">1</intProp>
        <intProp name="ThreadGroup.ramp_time">1</intProp>
        <boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
        <stringProp name="ThreadGroup.on_sample_error">stopthread</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller">
          <stringProp name="LoopController.loops">1</stringProp>
          <boolProp name="LoopController.continue_forever">false</boolProp>
        </elementProp>
      </ThreadGroup>
      <hashTree>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="POST /inboxes" enabled="true">
          <stringProp name="HTTPSampler.domain">api.mailslurp.com</stringProp>
          <stringProp name="HTTPSampler.protocol">https</stringProp>
          <stringProp name="HTTPSampler.path">/inboxes</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <stringProp name="HTTPSampler.method">POST</stringProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments">
            <collectionProp name="Arguments.arguments">
              <elementProp name="" elementType="HTTPArgument">
                <boolProp name="HTTPArgument.always_encode">false</boolProp>
                <stringProp name="Argument.value">{}</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
            </collectionProp>
          </elementProp>
        </HTTPSamplerProxy>
        <hashTree>
          <HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="API Key + JSON header" enabled="true">
            <collectionProp name="HeaderManager.headers">
              <elementProp name="x-api-key" elementType="Header">
                <stringProp name="Header.name">x-api-key</stringProp>
                <stringProp name="Header.value">${__P(API_KEY)}</stringProp>
              </elementProp>
              <elementProp name="content-type" elementType="Header">
                <stringProp name="Header.name">Content-Type</stringProp>
                <stringProp name="Header.value">application/json</stringProp>
              </elementProp>
            </collectionProp>
          </HeaderManager>
          <hashTree/>
          <JSONPostProcessor guiclass="JSONPostProcessorGui" testclass="JSONPostProcessor" testname="Extract INBOX_ID &amp; EMAIL_ADDRESS" enabled="true">
            <stringProp name="JSONPostProcessor.referenceNames">INBOX_ID;EMAIL_ADDRESS</stringProp>
            <stringProp name="JSONPostProcessor.jsonPathExprs">$.id;$.emailAddress</stringProp>
            <stringProp name="JSONPostProcessor.match_numbers">0;0</stringProp>
            <stringProp name="JSONPostProcessor.defaultValues">NOT_FOUND;NOT_FOUND</stringProp>
            <stringProp name="JSONPostProcessor.return_default">false</stringProp>
          </JSONPostProcessor>
          <hashTree/>
        </hashTree>
        <JSR223Sampler guiclass="TestBeanGUI" testclass="JSR223Sampler" testname="Assert vars present" enabled="true">
          <stringProp name="cacheKey">true</stringProp>
          <stringProp name="filename"></stringProp>
          <stringProp name="parameters"></stringProp>
          <stringProp name="scriptLanguage">groovy</stringProp>
          <stringProp name="script">
['INBOX_ID', 'EMAIL_ADDRESS'].each { name ->
    def value = vars.get(name)
    if (!value || value == 'NOT_FOUND') {
        throw new IllegalStateException(name + ' was not extracted')
    }
}
          </stringProp>
        </JSR223Sampler>
        <hashTree/>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="POST /test-application/magic-link" enabled="true">
          <stringProp name="HTTPSampler.domain">api.mailslurp.com</stringProp>
          <stringProp name="HTTPSampler.protocol">https</stringProp>
          <stringProp name="HTTPSampler.path">/test-application/magic-link</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <stringProp name="HTTPSampler.method">POST</stringProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments">
            <collectionProp name="Arguments.arguments">
              <elementProp name="" elementType="HTTPArgument">
                <boolProp name="HTTPArgument.always_encode">false</boolProp>
                <stringProp name="Argument.value">emailAddress=${EMAIL_ADDRESS}</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
            </collectionProp>
          </elementProp>
        </HTTPSamplerProxy>
        <hashTree>
          <HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="Content-Type Header" enabled="true">
            <collectionProp name="HeaderManager.headers">
              <elementProp name="" elementType="Header">
                <stringProp name="Header.name">Content-Type</stringProp>
                <stringProp name="Header.value">application/x-www-form-urlencoded</stringProp>
              </elementProp>
            </collectionProp>
          </HeaderManager>
          <hashTree/>
        </hashTree>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="GET /waitForLatestEmail" enabled="true">
          <stringProp name="HTTPSampler.domain">api.mailslurp.com</stringProp>
          <stringProp name="HTTPSampler.protocol">https</stringProp>
          <stringProp name="HTTPSampler.path">/waitForLatestEmail</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <stringProp name="HTTPSampler.method">GET</stringProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <boolProp name="HTTPSampler.postBodyRaw">false</boolProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables">
            <collectionProp name="Arguments.arguments">
              <elementProp name="inboxId" elementType="HTTPArgument">
                <boolProp name="HTTPArgument.always_encode">false</boolProp>
                <stringProp name="Argument.name">inboxId</stringProp>
                <stringProp name="Argument.value">${INBOX_ID}</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="timeout" elementType="HTTPArgument">
                <boolProp name="HTTPArgument.always_encode">false</boolProp>
                <stringProp name="Argument.name">timeout</stringProp>
                <stringProp name="Argument.value">60000</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
              <elementProp name="unreadOnly" elementType="HTTPArgument">
                <boolProp name="HTTPArgument.always_encode">false</boolProp>
                <stringProp name="Argument.name">unreadOnly</stringProp>
                <stringProp name="Argument.value">true</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
            </collectionProp>
          </elementProp>
        </HTTPSamplerProxy>
        <hashTree>
          <HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="API Key header" enabled="true">
            <collectionProp name="HeaderManager.headers">
              <elementProp name="x-api-key" elementType="Header">
                <stringProp name="Header.name">x-api-key</stringProp>
                <stringProp name="Header.value">${__P(API_KEY)}</stringProp>
              </elementProp>
            </collectionProp>
          </HeaderManager>
          <hashTree/>
          <JSONPathAssertion guiclass="JSONPathAssertionGui" testclass="JSONPathAssertion" testname="Assert subject present" enabled="true">
            <stringProp name="JSON_PATH">$.subject</stringProp>
            <stringProp name="EXPECTED_VALUE"></stringProp>
            <boolProp name="JSONVALIDATION">false</boolProp>
            <boolProp name="EXPECT_NULL">false</boolProp>
            <boolProp name="INVERT">false</boolProp>
            <boolProp name="ISREGEX">false</boolProp>
            <stringProp name="TOLERANCE"></stringProp>
            <stringProp name="DEFAULT_VALUE"></stringProp>
          </JSONPathAssertion>
          <hashTree/>
        </hashTree>
      </hashTree>
    </hashTree>
  </hashTree>
</jmeterTestPlan>

Set API_KEY through a protected JMeter properties file outside your repository and pass its path with -q; the JMX reads ${__P(API_KEY)}. Do not include the key itself in command arguments or save it inside the test plan. Use the GUI to edit and debug one iteration. Run the load measurement in CLI mode, as Apache JMeter recommends:

jmeter -n -q /secure/path/mailslurp.properties -t EmailLoadTest.jmx -l results.jtl -e -o report

Running JMeter with mvn

Another way to assemble JMeter tests is with the Java DSL and JUnit. The repository example below uses the MailSlurp Java library. It demonstrates the request sequence; provide API_KEY through the test runner's secret environment. Add the same recipient, subject, account-completion and cleanup checks before using it as a release gate. The final assertion rejects failed samples as well as a run with no recorded latency.

package dev.mailslurp;

import com.mailslurp.apis.*;
import com.mailslurp.clients.*;
import com.mailslurp.models.Email;
import com.mailslurp.models.InboxDto;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.Timeout;
import java.util.concurrent.TimeUnit;
import us.abstracta.jmeter.javadsl.core.TestPlanStats;

import java.util.UUID;

import static us.abstracta.jmeter.javadsl.JmeterDsl.*;

/**
 * Unit test for simple App.
 */
public class EmailJMeterTest {
    @Test
    @Timeout(value = 2, unit = TimeUnit.MINUTES)
    void magicLinkFlow() throws Exception {

        String apiKey = System.getenv("API_KEY");
        ApiClient client = Configuration.getDefaultApiClient();
        client.setApiKey(apiKey);
        // Increase HTTP timeouts for MailSlurp API calls (milliseconds)
        client.setConnectTimeout(60000);
        client.setReadTimeout(60000);
        client.setWriteTimeout(60000);

        InboxControllerApi inboxApi = new InboxControllerApi(client);
        WaitForControllerApi waitApi = new WaitForControllerApi(client);


        TestPlanStats stats = testPlan(
                threadGroup(1, 1,
                        jsr223Sampler(ctx -> {
                            InboxDto inbox = inboxApi.createInboxWithDefaults().execute();
                            ctx.vars.put("INBOX_ID", inbox.getId().toString());
                            ctx.vars.put("EMAIL_ADDRESS", inbox.getEmailAddress());
                        }),
                        jsr223Sampler(ctx -> {
                            String emailAddr = ctx.vars.get("EMAIL_ADDRESS");
                            if (emailAddr == null || emailAddr.isEmpty()) {
                                throw new AssertionError("EMAIL_ADDRESS was not set");
                            }
                        }),
                        httpSampler("https://api.mailslurp.com/test-application/magic-link")
                                .method("POST")
                                .header("Content-Type", "application/x-www-form-urlencoded")
                                .body("emailAddress=${EMAIL_ADDRESS}")
                                ,
                        jsr223Sampler(ctx -> {
                            String inboxIdStr = ctx.vars.get("INBOX_ID");
                            if (inboxIdStr == null) {
                                throw new AssertionError("INBOX_ID was not set");
                            }
                            UUID inboxId = UUID.fromString(inboxIdStr);
                            Email email = waitApi.waitForLatestEmail()
                                    .inboxId(inboxId)
                                    .timeout(60_000L)
                                    .unreadOnly(true).execute();
                            if (email.getSubject() == null) throw new AssertionError("No subject!");
                        })
                )
        ).run();

        // Fail JUnit if any sampler failed
        if (stats.overall().errorsCount() > 0 || stats.overall().sampleTimePercentile99() == null) {
            throw new AssertionError("A sampler failed or the test did not complete");
        }
    }
}

Interpret the load-test results

Label application signup, inbox creation, email wait and application confirmation as separate samplers. Report application response latency and completed verification latency separately; a single total hides whether the queue, delivery or account-confirmation step slowed down. Count non-success responses, timed-out waits and failed content assertions instead of reporting latency only for successful attempts.

Ramp up from one validated iteration. Account for the time each thread spends waiting for mail when setting the offered signup rate: increasing wait time can reduce the rate of new requests in a closed thread group. Treat HTTP 429 and service quota errors separately from wrong-message failures. Avoid logging API headers, full email bodies or tokenized links in shared JTL files and reports.

For a combined workflow, follow email and SMS deliverability load testing. The JMeter example repository contains the project dependencies and source for the demonstration above.