Testing SMTP relay is mainly a security and policy validation task.

You are verifying that your server relays only for authorized senders and approved scopes.

Quick answer: what to test

You should test two paths:

  1. unauthenticated relay attempts (must be rejected)
  2. authenticated relay attempts (must follow allowed policy)

If unauthenticated cross-domain relay is accepted, your server behaves like an open relay.

What is SMTP relay?

SMTP relay is the forwarding step where one mail server passes a message toward another domain's destination.

Relay is required for normal cross-domain email delivery, but it must be policy-controlled.

Open relay test concept

An open relay is an SMTP server that accepts and forwards mail from unauthorized senders.

This is typically validated by attempting external-to-external relay without authentication.

Manual relay test flow

High-level session:

Expected secure behavior:

  • relay attempt rejected with policy/auth response (, , depending on server)
  • no queue acceptance for unauthorized path

1) SMTP CLI tools (swaks / smtp-cli)

These are better than raw telnet for repeatable tests with auth and TLS combinations.

2) OpenSSL for TLS relay-path validation

Use OpenSSL to ensure encrypted relay session behavior is correct where STARTTLS is required.

3) Staged CI assertions

Use automated tests to enforce relay policy across environments and prevent regressions.

Practical relay test scenarios

Scenario A: unauthenticated external relay

  • should fail
  • confirms open relay protections

Scenario B: authenticated sender to approved domain scope

  • should pass if policy allows
  • confirms valid submission path

Scenario C: authenticated sender to disallowed scope

  • should fail with policy rejection
  • confirms least-privilege relay boundaries

Common relay test failures and fixes

SymptomLikely causeFix
Unauthenticated external relay acceptedOpen relay misconfigurationRestrict relay to authenticated/authorized senders
Authenticated relay denied unexpectedlyWrong auth/TLS mode or policy mismatchVerify + STARTTLS (or implicit TLS), credentials, and relay ACL
Production-only relay failuresEnvironment-specific policy or egress driftStandardize config and add CI relay assertions

Secure relay checklist

  1. Reject unauthenticated cross-domain relay attempts.
  2. Enforce SMTP AUTH and TLS before relay.
  3. Restrict relay by sender identity and policy scope.
  4. Log and alert on relay-denied spikes.
  5. Validate domain authentication posture (SPF, DKIM, DMARC).

Release workflow for relay changes

FAQ

Is relay always bad?

No. Relay is necessary for cross-domain delivery. Open relay is the problem.

Can AUTH alone prevent open relay?

It helps, but policy/ACL configuration still determines allowed relay scope.

Which error means relay was blocked?

Commonly /, and sometimes when auth is required first.

Next steps