Skip to main content

#16: Test Automation with Playwright

· 7 min read
Nitya Narasimhan
Debbie O'Brien

Welcome to Week 3, Day 2 of #30DaysOfSWA!!

Yesterday we kicked off "Developer Tools" week by talking about the end-to-end developer experience associated with building modern web applications. Today we focus on a key segment - testing. Let's look at what test automation means for modern web applications, and how Playwright can help you automate end-to-end testing of your Azure Static Web App.

What We'll Cover​

  • Test Automation for Modern Web Apps
  • What is Playwright?
  • Get Started With Playwright
  • Explore Playwright Tools
  • Use Playwright with SWA
  • Exercise: Try the Tutorial with your deployed app.

Test Automation​

Modern mobile web applications have to work correctly and consistently across all modern browsers (like Chromium, Firefox or WebKit) and the device platforms they run on (with varying screen sizes, orientations). Testing your application UI/UX manually becomes difficult when we think about the granularity of tests across these combinations. Testing manually becomes impossible when we factor in the frequency at which testing needs to be done to validate these on every code commit or pull request. This is where test automation shines!

Test Automation is about executing tests automatically to validate software specs, then using the reported insights to improve software quality iteratively. For web apps, this requires tools that can automate test actions in the browser (web automation) and support this consistently across browsers. This is where Playwright shines!

Playwright Overview​

Playwright is an open-source framework for reliable end-to-end testing of modern web apps. It features a built-in Playwright Test Runner for automating test execution, and supports a Playwright Library to simplify integration into third-party solutions.

Visual Guide to Playwright

This visual guide shows the key features of the framework (downloadable hi-res version here) with these useful features:

  • Unified API - you write tests once, they run on all browsers and devices.
  • Mobile Web Testing - rich set of device profiles, usable with native emulators.
  • Multi-Language Support - write tests in Java, JS/TS, Python or C#.
  • Reliable Testing - no flaky tests thanks to auto-waiting and web-first assertions
  • Powerful Tooling - get help authoring, debugging, profiling, and reporting, tests.
  • Full Isolation - browser contexts (in just ms), options to parallelize tests

Check out the Playwright documentation and API for more details.

What kinds of things can we test with Playwright?

  • Test if UI component behaviors work as expected (events)
  • Test if UI workflows are correct (navigation, inputs)
  • Test UX behaviors in different contexts (with fixtures)
  • Modify network traffic (auth, proxy etc.) for testing
  • Validate behaviors under network conditions (modified)
  • Validate behaviors for device contexts (emulated profiles)
  • Automate capture of evidence (videos, screenshots) in tests

Explore the Guides and Demo.Playwright for usage examples. And check out Debbie's playwright-testing series for more examples of Playwright in action.

Playwright Quickstart​

It's time to get some hands-on experience with Playwright. Check out the Getting Started guide on their site for reference.

However, we'd like to walkthrough it with a real-world example in mind and figure out how to go from the boilerplate setup to the start of a basic script and configuration that sets us up to do more for testing our Azure Static App.

#30DaysOfSWA Website Example: Step-by-Step Tutorial:

Check out this repository that I setup today, to use the quickstart guide as a basis for setting up end-to-end testing for the #30DaysOfSWA website! The README documents the following steps in detail:

By the end of this tutorial you should have:

  • A starter test script customized for your application
  • A test configuration file with tracing on to capture detailed profiling data
  • A GitHub Actions workflow that automates test runs with each codebase change.

Test Run Report

At this point, you are now ready to build on this:

Playwright Tooling​

In the tutorial above, we mentioned some Playwright tools (CLI, TraceViewer, HTML Reporter, Codegen) in passing. Playwright has a really powerful set of tools that can streamline your end-to-end testing experience from authoring to reporting.

Start by familiarizing yourself with the Playwright CLI.

It's the main launching pad for all other command-line utilities. Using the CLI you can navigate to pages ("open"), install browser dependencies ("install"), take screenshots ( "screenshot"), launch the test runner ("test"), author tests ("codegen"), view traces ("show-trace") and more.

Launch the CLI as follows:

npx playwright <command> [options]

Use npx playwright --help for details on commands and options supported in your installed CLI version. Want a deeper dive?

Usage Example:

Let's use the screenshot command to capture an image of the kickoff post from the #30DaysOfSWA website for an "iPhone 6" device profile. You can see the named device profiles currently supported for reference.

npx playwright screenshot --device "iPhone 6" https://www.azurestaticwebapps.dev/blog/kickoff 30Days.png

Navigating to https://www.azurestaticwebapps.dev/blog/kickoff
Capturing screenshot into 30Days.png
Result: This is what the saved image looks like.
Screenshot

Then start exploring these tools:

  • Inspector - a GUI-based tool for debugging test execution
  • TraceViewer - a GUI-based tool to explore recorded test traces
  • Codegen - an authoring tool to generate tests based on user interactions
  • Debugging - additional tools to support runtime debugging
  • Reporters - built-in reporters of different types

What's Next​

We covered a lot today.

At this point, you should know what Playwright its, and why it's useful for automating tests for modern web applications. You should have a sense of the powerful developer tools it has, and understand how to setup and configure it for use with your Azure Static Web App. And you should have a clean basis for building up your test suite.

So where can we go next to skill up? We have you covered:

  • Watch this repository I setup today. I hope to continue adding more tests that can walk through various Playwright features with a deployed SWA,, and use that to explore both the Playwright capabilities and SWA-specific features or usage.

  • Join us tomorrow when we explore a "Multi-Stage Deployment with Azure DevOps" demo that shows you a full-stack application with Playwright tests integrated. Learn interesting aspects - like using SWA environment variables to define test parameters - that highlight best practices for test setup in SWA.

  • Last but not least, check out this talk from last year for more insights into End-to-End Testing for Playwright from the Azure Static Web Apps Team.

Exercise​

Pick one of the apps you've built (or use) - preferably one that is currently deployed to Azure Static Web Apps so you can explore other scenarios later.

Walk through the same steps with your deployed app - then try adding one new test script that goes beyond the starter test case. Look at the demo project for inspiration!