Email Testing in GitHub Actions

What mocking does not catch

The standard approach to email in CI is to swap the transport for an in-memory stub and assert that it was called. That is a useful unit test and it catches a real class of bug: the code path that forgets to send. It catches nothing else.

Failures that a mocked transport cannot see, all of which have taken down real signup flows:

• Provider rejection. An expired API key, a suspended account, or a sending domain that fell out of good standing. Your code called send() and got a 401 it never checked.
• Authentication failures. An SPF record edited during an unrelated DNS change. The message sends, and lands in spam or is dropped outright.
• Template breakage. A variable renamed in the backend but not in the template, so the verification link renders as https://app.example.com/verify?token=undefined.
• Encoding damage. Quoted-printable wrapping that inserts a soft line break in the middle of a long token, so the link works in one client and 404s in another.
• Queue regressions. The job is enqueued and never drained, because a worker deploy silently failed.

Every one of those passes a mocked test and fails a real one. The distinction is whether your assertion reads "we tried to send" or "a message arrived and its contents work".

The shape of a real test

An end-to-end email test has four steps, and only the second is specific to your application:

1. Create a disposable inbox and get its address.
2. Drive your application so it sends to that address — a signup form, an API call, a queued job.
3. Poll the inbox until the message arrives or a deadline passes.
4. Assert on the contents: the OTP, the verification link, the sender, the subject.

Steps 1, 3 and 4 are what an API like TempMailGrab's exists to make short. Step 3 in particular is where hand-rolled solutions go wrong: a fixed sleep(10) is both slower than it needs to be on a good day and flaky on a bad one.

Step 1: store an API key as a repository secret

Generate a key from your developer dashboard, then add it to the repository under Settings → Secrets and variables → Actions as TMG_API_KEY. Never inline a key in the workflow file — workflow files are visible to anyone who can read the repository, and on a public repo that is everyone.

Step 2: a helper with real polling

Create tests/helpers/mailbox.ts. The important part is waitForMessage: it polls on an interval with a hard deadline, so a fast delivery finishes fast and a broken one fails with a useful message instead of hanging until the job times out.

// tests/helpers/mailbox.ts
const BASE = 'https://tempmailgrab.com/api/v1';
const KEY = process.env.TMG_API_KEY;

if (!KEY) throw new Error('TMG_API_KEY is not set');

const headers = {
  Authorization: 'Bearer ' + KEY,
  'Content-Type': 'application/json',
};

export interface Inbox {
  id: string;
  address: string;
  expires_at: string;
}

export interface Message {
  id: string;
  sender: string;
  subject: string;
  snippet: string;
  extracted_otp: string | null;
  created_at: number;
}

/** Create a disposable inbox. The prefix only aids debugging; it must be unique. */
export async function createInbox(prefix?: string): Promise<Inbox> {
  const res = await fetch(BASE + '/inbox/generate', {
    method: 'POST',
    headers,
    body: JSON.stringify({ prefix, ttl_seconds: 3600 }),
  });
  if (!res.ok) throw new Error('createInbox failed: ' + res.status + ' ' + (await res.text()));
  return res.json();
}

/**
 * Poll until a message matching the predicate arrives.
 * Throws on timeout with the subjects that DID arrive, which is the difference
 * between a two-minute debug and a two-hour one.
 */
export async function waitForMessage(
  inboxId: string,
  match: (m: Message) => boolean = () => true,
  { timeoutMs = 45_000, intervalMs = 2_000 } = {},
): Promise<Message> {
  const deadline = Date.now() + timeoutMs;
  let seen: string[] = [];

  while (Date.now() < deadline) {
    const res = await fetch(BASE + '/inbox/' + inboxId + '/messages', { headers });
    if (res.ok) {
      const { messages } = (await res.json()) as { messages: Message[] };
      seen = messages.map((m) => m.subject);
      const hit = messages.find(match);
      if (hit) return hit;
    }
    await new Promise((r) => setTimeout(r, intervalMs));
  }

  throw new Error(
    'No matching message within ' + timeoutMs + 'ms. Subjects seen: ' +
      (seen.length ? seen.join(', ') : '(inbox empty)'),
  );
}

/** Full message body, including the extracted links. */
export async function getMessage(inboxId: string, messageId: string) {
  const res = await fetch(BASE + '/inbox/' + inboxId + '/messages/' + messageId, { headers });
  if (!res.ok) throw new Error('getMessage failed: ' + res.status);
  return res.json();
}

/** Delete the inbox. Call this in teardown so CI runs do not leak addresses. */
export async function destroyInbox(inboxId: string): Promise<void> {
  await fetch(BASE + '/inbox/' + inboxId, { method: 'DELETE', headers });
}

Step 3: the test

This example asserts a signup flow end to end. Note the teardown — without it, a long-lived CI pipeline accumulates inboxes it never cleans up.

// tests/signup.spec.ts
import { test, expect } from '@playwright/test';
import { createInbox, waitForMessage, getMessage, destroyInbox } from './helpers/mailbox';

test('a new signup receives a working verification link', async ({ page }) => {
  const inbox = await createInbox('ci-signup');

  try {
    await page.goto('https://staging.example.com/signup');
    await page.fill('[name=email]', inbox.address);
    await page.fill('[name=password]', 'correct-horse-battery-staple');
    await page.click('button[type=submit]');

    const msg = await waitForMessage(inbox.id, (m) =>
      m.subject.toLowerCase().includes('verify'),
    );

    // Assert on the envelope, not just the body — a template change that
    // breaks the From address is invisible if you only check the HTML.
    expect(msg.sender).toContain('@example.com');

    const full = await getMessage(inbox.id, msg.id);
    const verifyUrl = full.extracted_links.find((l: string) => l.includes('/verify'));
    expect(verifyUrl, 'no /verify link in the email').toBeTruthy();

    await page.goto(verifyUrl);
    await expect(page.getByText('Your account is verified')).toBeVisible();
  } finally {
    await destroyInbox(inbox.id);
  }
});

Step 4: the workflow

name: E2E

on:
  push:
    branches: [main]
  pull_request:
  schedule:
    # Delivery breaks without a deploy — an expired key, a DNS edit, a provider
    # suspension. A nightly run catches those; a push-triggered run does not.
    - cron: '0 6 * * *'

jobs:
  email-flows:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm

      - run: npm ci
      - run: npx playwright install --with-deps chromium

      - name: Run email end-to-end tests
        env:
          TMG_API_KEY: ${{ secrets.TMG_API_KEY }}
        run: npx playwright test tests/signup.spec.ts

      - name: Upload traces on failure
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-traces
          path: test-results/

The scheduled run matters more than the push run

This is the part teams skip. Email delivery has dependencies that live entirely outside your repository: a DNS record, a provider account, a domain reputation, a billing status. None of those change when you push, and all of them can break silently.

A pipeline that only runs on push will happily stay green through a week of undelivered signup emails. A nightly scheduled run turns that into an alert the morning after. If you add one thing from this article, add the schedule trigger.

Making it reliable

Give every run a unique address

Parallel jobs that request the same prefix collide. Either omit the prefix entirely and take the generated address, or make it unique per run:

const prefix = 'ci-' + process.env.GITHUB_RUN_ID + '-' + Math.random().toString(36).slice(2, 8);

Do not assert on exact timing

Assert that a message arrives within a generous deadline, not that it arrives in under two seconds. Delivery latency is a property of the internet, not of your code, and a tight timing assertion is a flaky test that will eventually be deleted along with the coverage it provided.

Separate "did it send" from "is it correct"

Two failures with very different causes deserve two different messages. A test that reports "no message arrived" sends you to your provider and your DNS. A test that reports "message arrived, but no /verify link" sends you to your templates. Collapsing both into "signup test failed" wastes the first twenty minutes of every investigation.

Clean up, even on failure

The try/finally in the example is not decoration. A test that throws before its cleanup leaves an inbox alive for its full TTL, and a pipeline running on every pull request will accumulate hundreds.

Testing the pipeline itself

There is a bootstrapping problem here: if your test never receives mail, is delivery broken or is your test harness broken? The API has an endpoint for exactly this — it injects a synthetic message into an inbox without involving your application or any mail server:

curl -X POST https://tempmailgrab.com/api/v1/inbox/$INBOX_ID/test-message \
  -H "Authorization: Bearer $TMG_API_KEY"

If the synthetic message arrives and yours does not, the harness is fine and the problem is upstream in your application. That single check removes the most common ambiguity in a failing email test.

Where to go next

The same helper works unchanged under other runners — see the Playwright tutorial for a deeper treatment of the browser side, or the Cypress guide if that is your stack. Full endpoint documentation is in the API reference.