MailSlurp logo

blog

SMTP Relay Testing Guide: Open Relay Checks, Auth Validation, and Inbox Proof

Test SMTP relay behavior safely with open relay checks, authenticated sends, inbox receipt, header review, webhook evidence, and CI assertions.

SMTP relay testing checks whether your mail server accepts the right messages, rejects the wrong messages, and produces proof that the accepted message reached the workflow that depends on it.

For application teams, a relay test should not stop at "the server accepted DATA." It should also confirm that the message reached a controlled inbox, kept the expected headers, triggered the right event path, and can be asserted in CI.

Quick answer

A reliable SMTP relay test covers four paths:

  1. unauthenticated external-to-external relay attempts, which must be rejected
  2. authenticated sender-to-approved-recipient sends, which should be accepted when policy allows them
  3. authenticated sender-to-disallowed-scope sends, which should be rejected
  4. accepted test messages captured in a MailSlurp inbox, with headers, body, links, and webhook evidence reviewed

If an unauthenticated external-to-external relay attempt is accepted, the server behaves like an open relay and should not be exposed until policy is fixed.

Only test relay behavior on systems you own or are authorized to validate. Do not probe arbitrary third-party mail servers.

What is SMTP relay?

SMTP relay is the forwarding step where one mail server passes a message toward another domain's destination.

Relay is required for normal cross-domain email delivery, but it must be policy-controlled.

SMTP relay testing checklist

Before you start, define the evidence you need from the test run:

  • SMTP response code and response text
  • host, port, and TLS mode used by the client
  • sender address, recipient address, and environment
  • authentication mode and sender policy used by the relay
  • message ID from the sending side
  • received message ID from the inbox side
  • Received, Return-Path, Authentication-Results, SPF, DKIM, and DMARC header evidence
  • webhook, bounce, or event payload when the relay exposes one

That evidence lets the team distinguish a transport failure from a sender policy issue, DNS authentication issue, content issue, or inbox placement issue.

Open relay test concept

An open relay is an SMTP server that accepts and forwards mail from unauthorized senders.

This is typically validated by attempting external-to-external relay without authentication.

The test should fail. A secure relay normally responds with an authentication or policy error such as 530, 550, or 554, depending on the server.

Manual relay test flow with SMTP commands

High-level session:

EHLO test.example
MAIL FROM:<sender@external-a.com>
RCPT TO:<recipient@external-b.com>
DATA
...

Expected secure behavior:

  • relay attempt rejected with policy/auth response (530, 550, 554 depending on server)
  • no queue acceptance for unauthorized path
  • no downstream inbox receipt for the unauthorized message

For manual troubleshooting, pair this with the SMTP commands and responses guide so every response code is captured with context.

1) SMTP CLI tools (swaks / smtp-cli)

These are better than raw telnet for repeatable tests with auth and TLS combinations.

2) OpenSSL for TLS relay-path validation

Use OpenSSL to ensure encrypted relay session behavior is correct where STARTTLS is required.

3) MailSlurp inboxes for receipt proof

Use an isolated MailSlurp inbox as the recipient for authorized relay tests. That gives you a safe destination for real messages, raw headers, body checks, attachment review, and link or OTP assertions.

4) Staged CI assertions

Use automated tests to enforce relay policy across environments and prevent regressions.

Practical relay test scenarios

Scenario A: unauthenticated external relay

  • should fail
  • confirms open relay protections

Scenario B: authenticated sender to approved domain scope

  • should pass if policy allows
  • confirms valid submission path

Scenario C: authenticated sender to disallowed scope

  • should fail with policy rejection
  • confirms least-privilege relay boundaries

Scenario D: authenticated send to a MailSlurp inbox

  • should pass when policy allows the sender and recipient
  • confirms the relay accepts the message and a real inbox receives it
  • gives the team headers, body, links, attachments, timing, and webhook evidence

Example relay test command

Use the exact host, port, auth method, and sender domain for the environment under test:

swaks \
  --server smtp.example.com \
  --port 587 \
  --tls \
  --auth LOGIN \
  --auth-user "$SMTP_USER" \
  --auth-password "$SMTP_PASS" \
  --from "qa@example.com" \
  --to "$MAILSLURP_INBOX_ADDRESS" \
  --header "X-Relay-Test: staging-cutover" \
  --data "Subject: SMTP relay test

This message verifies authenticated relay, inbox receipt, and header evidence."

Save the command output with the timestamp, message ID, response code, and environment name. Then open the received message in MailSlurp and compare the message headers with the sending-side output.

What to inspect after the message arrives

SMTP acceptance alone is not enough. After the relay accepts the message, inspect the received copy:

  • subject and body match the expected template
  • From, Reply-To, and Return-Path match your sender policy
  • Authentication-Results shows the expected SPF, DKIM, and DMARC outcomes
  • Received headers show the expected relay path
  • links, OTP codes, and attachments are present and usable
  • webhook or event payload includes the received message ID

Use the Email header analyzer when sender identity, authentication alignment, or relay hops need a closer review.

CI workflow for release-safe relay testing

For release gates, turn the manual checks into a repeatable test:

  1. Create or reuse a MailSlurp inbox for the environment.
  2. Trigger the application path that sends through the relay.
  3. Wait for the latest email in that inbox.
  4. Assert the subject, body, links, OTP, and headers.
  5. Fail the build if the message does not arrive within the expected window.
  6. Store the SMTP response, received message ID, and webhook event with the test run.

Example shape:

import { MailSlurp } from "mailslurp-client";

const mailslurp = new MailSlurp({ apiKey: process.env.MAILSLURP_API_KEY! });
const inbox = await mailslurp.createInbox();

await triggerRelayBackedEmail({ to: inbox.emailAddress });

const email = await mailslurp.waitForLatestEmail(inbox.id!, 60000, true);

expect(email.subject).toContain("SMTP relay test");
expect(email.body).toContain("This message verifies authenticated relay");
expect(JSON.stringify(email.headers)).toContain("Authentication-Results");

That test checks the path users actually depend on: application send, SMTP relay, recipient delivery, content, and message metadata.

Common relay test failures and fixes

Symptom Likely cause Fix
Unauthenticated external relay accepted Open relay misconfiguration Restrict relay to authenticated/authorized senders
Authenticated relay denied unexpectedly Wrong auth/TLS mode or policy mismatch Verify 587 + STARTTLS (or 465 implicit TLS), credentials, and relay ACL
Production-only relay failures Environment-specific policy or egress drift Standardize config and add CI relay assertions
Message accepted but not received downstream rejection, DNS auth drift, or content issue Inspect headers, relay events, bounces, and inbox placement evidence
Header identity changed unexpectedly relay rewrite, sender-domain policy, or Return-Path change Compare raw headers with sender-domain and DMARC policy

Troubleshooting evidence to collect

When a relay test fails, capture enough detail for the owner of the next system in the path:

  • failing command or application request
  • SMTP response code and response text
  • whether STARTTLS or implicit TLS was used
  • sender, recipient, and envelope values
  • relay queue ID or provider event ID
  • MailSlurp inbox ID and received message ID when a message arrived
  • raw headers from the received copy
  • webhook delivery attempts and retry payloads

This avoids vague "email did not work" tickets and gives engineering, security, and deliverability teams a shared record.

Secure relay checklist

  1. Reject unauthenticated cross-domain relay attempts.
  2. Enforce SMTP AUTH and TLS before relay.
  3. Restrict relay by sender identity and policy scope.
  4. Log and alert on relay-denied spikes.
  5. Validate domain authentication posture (SPF, DKIM, DMARC).
  6. Keep staging, CI, and production relay credentials separated.
  7. Test accepted messages against controlled inboxes before rollout.

Release workflow for relay changes

When to use MailSlurp for SMTP relay testing

Use MailSlurp when a relay test needs proof beyond a protocol transcript. MailSlurp helps you create private inboxes, capture real messages, inspect raw headers, assert links and codes, trigger webhooks, and keep relay checks inside Playwright, Cypress, Jest, or CI workflows.

That makes relay testing useful for signup email, password reset, OTP, billing, alert, campaign QA, and sender-authentication changes.

FAQ

Is relay always bad?

No. Relay is necessary for cross-domain delivery. Open relay is the problem.

Can AUTH alone prevent open relay?

It helps, but policy/ACL configuration still determines allowed relay scope.

Which error means relay was blocked?

Commonly 550/554, and sometimes 530 when auth is required first.

Does SMTP acceptance prove delivery?

No. SMTP acceptance proves the relay accepted the message. You still need inbox receipt, header review, and event evidence to prove the workflow completed.

Can I test SMTP relay safely in CI?

Yes. Use isolated credentials, a controlled sender domain, and MailSlurp inboxes for receipt assertions. Keep open relay checks limited to authorized infrastructure.

Next steps