Run automation tests in GitHub CI/CD: Playwright & JavaScript

Setting up your automation tests in a GitHub CI/CD pipeline is primarily achieved using GitHub Actions. This is a powerful, built-in feature that allows you to define workflows that are triggered by events in your repository, such as a code push or a pull request.

Here’s a breakdown of the typical process to set up your playwright (JavaScript) automation tests in Github CI/CD:

Create the Workflow File

You define your CI/CD pipeline using a YAML file that must be placed in the .github/workflows/ directory of your repository. File Location Example: .github/workflows/your_workflow_name.yml

You have to use the ‘on:‘ keyword to specify the events that should start the workflow (e.g., push to the main branch, pull_request).

Example Code:

name: Playwright Tests
on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]

Define Jobs and Steps

A workflow consists of one or more jobs, which are executed on a specified runner (a virtual machine hosted by GitHub, or a self-hosted runner). Each job contains a sequence of steps.

jobs:: Defines the main tasks (e.g., build, test, deploy).

runs-on:: Specifies the runner environment (e.g., ubuntu-latest, windows-latest).

steps:: A list of sequential actions to perform:

• Checkout Code: The first step is almost always checking out your repository code using the built-in action: uses: actions/checkout@v4.
• Setup Environment: Set up the necessary language or runtime environment (e.g., Node.js, Python, Java) using an appropriate setup action: uses: actions/setup-node@v4 or uses: actions/setup-python@v5.
• Install Dependencies: Run the necessary commands to install your project and test dependencies (e.g., npm install, pip install -r requirements.txt).
• Run Tests: Execute your test runner command. This is the core step for automation testing. You use the run: keyword to execute shell commands.

Example Step for Running Tests (Node.js/npm example):

    - name: Run Playwright tests
      run: npx playwright test

If your tests need specific environment variables, especially sensitive ones like username password keys, you should store them as GitHub Secrets and reference them in your workflow.

    - name: Run Tests with Secrets
      env:
         # Non-sensitive configuration data
         BASE_URL: https://example.com

         # Need to create these secrets in repository settings:
         ADMIN_USERNAME: ${{ secrets.ADMIN_USERNAME }}
         ADMIN_PASSWORD: ${{ secrets.ADMIN_PASSWORD }}

CI/CD Flow Control

Test Success/Failure: If the command in the “Run Tests” step exits with a non-zero code (indicating a test failure), the job will fail, and subsequent steps (like deployment) can be prevented from running.

Dependency: You can ensure that a deployment job only runs if the testing job succeeds using the needs: keyword.

deploy:
  needs: test # This job only runs if the 'test' job succeeds
  runs-on: ubuntu-latest
  # ... steps for deployment ...

Basic Workflow Example for Playwright JavaScript:

name: Playwright Tests
on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]
  
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    # 🚨 ENVIRONMENT VARIABLES BLOCK
    env:
      # Non-sensitive configuration data
      BASE_URL: ${{ secrets.BASE_URL }}

      # Need to create these secrets in repository settings:
      ADMIN_USERNAME: ${{ secrets.ADMIN_USERNAME }}
      ADMIN_PASSWORD: ${{ secrets.ADMIN_PASSWORD }}

    steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with:
        node-version: lts/*
    - name: Install dependencies
      run: 
        npm ci
        npx playwright install --with-deps chromium

    - name: Install Playwright Browsers
      run: npx playwright install --with-deps

    - name: Run Playwright tests
      run: npx playwright test
    - uses: actions/upload-artifact@v4
      if: ${{ !cancelled() }}
      with:
        name: playwright-report
        path: playwright-report/
        retention-days: 30

To enable manual runs for workflow:

  workflow_dispatch: #manual runs