How to Build a Playwright Smoke Test in GitHub Actions

The build is green, every unit test passes, you deploy, and then the home page turns out blank because one script throws in the browser. No test saw it, because no test ever opened a browser. A smoke test closes that gap: it opens the built app and checks that the few things that must work, do.

This post takes you from nothing to a Playwright smoke test running on every pull request. Every command and option was checked against the official Playwright and GitHub documentation, and the pattern matches the game-smoke.yml workflow that guards the game page on this site.

What you need

Not much. The examples use a Next.js project, but the same pattern works for any web app you can build and start with one command.

• Node.js and a web project that has npm run build and npm run start scripts.
• A GitHub repository, so GitHub Actions can run the tests.
• A terminal and some comfort with npm. You do not need to have used Playwright before.

Step 1: Install Playwright

Run one command in the project folder. Playwright asks a few questions: TypeScript or JavaScript, the tests folder name, whether to add a GitHub Actions workflow, and whether to install browsers right away. Answer no to the workflow, because we write it ourselves in Step 5 so you understand what is in it.

# Run inside your existing project folder
npm init playwright@latest

# The prompts ask for: TypeScript or JavaScript, the tests folder name
# (tests by default), a GitHub Actions workflow, and whether to install browsers.
# Answer no to the workflow question: we write our own in step 5.

# Check what got installed
npx playwright --version

You end up with playwright.config.ts, a tests folder with one example test, and Playwright recorded in package.json and the lockfile. Delete the example test, because it opens someone else's website. We will write tests for your own app.

Step 2: Write the first smoke test

Create tests/smoke.spec.ts. Each test opens a page and checks something a user sees: the title, the main heading, and one click on the most important path. Replace My App and Blog with text that really exists in your app. Playwright waits for elements to be ready before acting, so you never need a sleep.

// tests/smoke.spec.ts
import { test, expect } from '@playwright/test';

test('home page has the right title', async ({ page }) => {
  // baseURL from playwright.config.ts is prepended to '/'
  await page.goto('/');
  await expect(page).toHaveTitle(/My App/);
});

test('home page shows its main heading', async ({ page }) => {
  await page.goto('/');
  await expect(page.getByRole('heading', { level: 1 })).toBeVisible();
});

test('the Blog link opens the blog', async ({ page }) => {
  await page.goto('/');
  await page.getByRole('link', { name: 'Blog' }).first().click();
  await expect(page.getByRole('heading', { name: 'Blog' })).toBeVisible();
});

One habit beginners often get wrong: calling isVisible() and comparing the result yourself. It asks once, at that instant. Use the assertion that waits, await expect(...).toBeVisible(), which the docs say waits until the condition is met.

// Wrong: isVisible() answers once, right now. If the page is still
// rendering, the test fails even though the heading appears a moment later.
expect(await page.getByRole('heading', { level: 1 }).isVisible()).toBe(true);

// Right: a web-first assertion keeps retrying until the heading shows up
// or the timeout runs out.
await expect(page.getByRole('heading', { level: 1 })).toBeVisible();

Step 3: Set up the config and webServer

In playwright.config.ts, the webServer block tells Playwright how to start the app and which address to check. Playwright waits for that address to answer before it runs the tests, so you do not write a wait loop yourself. baseURL makes page.goto('/') go to your app. We also limit the projects to chromium, because one browser is enough for a smoke test.

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: 'tests',
  // Fail the run on CI if test.only was left in the source
  forbidOnly: !!process.env.CI,
  // Retry on CI only; a retry is what records a trace
  retries: process.env.CI ? 2 : 0,
  reporter: 'html',
  use: {
    baseURL: 'http://localhost:3000',
    trace: 'on-first-retry',
  },
  // One browser is enough for a smoke test
  projects: [{ name: 'chromium', use: { ...devices['Desktop Chrome'] } }],
  webServer: {
    // The app must already be built: npm run build comes first
    command: 'npm run start',
    url: 'http://localhost:3000',
    // Reuse your own running server locally, always start fresh on CI
    reuseExistingServer: !process.env.CI,
  },
});

Three things matter in this config. First, the port in command, url and baseURL must match. Second, the webServer timeout defaults to 60000 ms. Third, reuseExistingServer is true locally so a server you already run gets reused, and false on CI so it always starts a fresh one.

Step 4: Run it on your machine first

Do not jump straight to CI. Build first, then run the tests locally, because a failure here is far quicker to read than one in the GitHub log. Tests run headless by default, so add --headed if you want to watch the browser move.

# Build once, then run the smoke test (headless by default)
npm run build
npx playwright test

# Watch the browser do it
npx playwright test --headed

# Only the chromium project, or only one file
npx playwright test --project=chromium
npx playwright test tests/smoke.spec.ts

# Open the HTML report (it opens by itself only when something failed)
npx playwright show-report

When everything passes, the terminal summarises how many tests passed. When something fails, the HTML report opens by itself and shows which step failed. If Playwright complains about webServer, it is almost always because npm run build has not run yet.

Step 5: Run it in GitHub Actions

Create .github/workflows/smoke.yml. The order is: checkout, set up Node, npm ci, build, install Chromium, run the tests, upload the report. The browser is installed after npm ci because each Playwright version needs specific browser binaries, so the install command must read the version that is actually installed.

# .github/workflows/smoke.yml
name: smoke

on:
  push:
    branches: [main]
  pull_request:

# A new push cancels the older run of the same branch
concurrency:
  group: smoke-${{ github.ref }}
  cancel-in-progress: true

permissions:
  contents: read

jobs:
  smoke:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm

      - name: Install dependencies
        run: npm ci

      - name: Build
        run: npm run build

      # After npm ci, so the browser build matches the installed Playwright
      - name: Install Chromium
        run: npx playwright install --with-deps chromium

      # webServer in playwright.config.ts starts the app and waits for it
      - name: Smoke test
        run: npx playwright test

      - name: Upload report
        if: ${{ !cancelled() }}
        uses: actions/upload-artifact@v4
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 14

On this site, game-smoke.yml does the same job another way: it starts npm start in the background and waits with a curl loop instead of using webServer. Both are valid, and the free lesson below covers the loop version. The workflow also matches it in three ways: it installs only Chromium, cancels the older run on the same branch, and grants only read access to the repository contents.

Step 6: Download and read the report when it fails

When a run fails, open the run page on GitHub, find the Artifacts section and download playwright-report. Extract the zip, then serve it through Playwright. The HTML report needs a web server, so opening index.html straight from the folder will not work.

# 1. In the failed run, open Artifacts and download playwright-report (a zip)
# 2. Extract it, ideally inside a folder where Playwright is installed
# 3. Serve it with Playwright, passing the extracted folder name
npx playwright show-report playwright-report

# Wrong: double-clicking index.html. The report needs a web server.

In our config, retries is 2 on CI and the trace is recorded on the first retry. A failing test therefore carries a trace you can open from the report: you see every step and how the page looked, instead of guessing from error text.

Step 7: Keep it small

A smoke test is a tripwire, not a regression suite. Three to five checks on the most important path are enough: the home page renders, the main navigation works, and one core action completes. The Playwright docs advise testing only what you control.

// Wrong: testing a site you do not control. It goes red when they have a bad day.
await page.goto('https://playwright.dev/');

// Right: only your own app, through baseURL.
await page.goto('/');

Every extra test is one more way to fail for reasons unrelated to your change. If the app calls a third-party service, the docs suggest using the Playwright Network API to guarantee the response you need, so an outside site cannot turn the run red. Put deeper testing elsewhere.

Common errors and how to fix them

The six problems that most often break a new smoke test on CI, with their causes and fixes.

Checklist before you push

• npm run build then npx playwright test passes on your machine.
• There are only three to five tests, all on the most important path, and none loads a site you do not own.
• The port in command, url and baseURL match, and projects are limited to the browsers you install.
• The workflow installs the browser after npm ci and builds before the tests.
• The report upload step uses if with !cancelled(), and forbidOnly is on for CI.

Go deeper

The free lesson Playwright smoke test in GitHub Actions covers the other way to start the server, a curl wait loop, and printing the server log when a test fails. If you are curious about AI agents that drive a browser, compare the approaches in Playwright MCP vs Browser Use vs Stagehand.

The rule to carry: a smoke test is a few checks on the most important path, run against the built app in a real browser, with the report always uploaded on failure. Start with three tests, and add one only when a failure that slipped through proves you need it.