Playwright vs Selenium - It's not even close

Disclaimer: This blog post is highly biased towards Playwright over Selenium and Cypress.

In my 15 years of automating (tests and process automation bots), Playwright has given me the most stable runs and the highest ROI. This is me writing down why I think that is.

Why Automate Tests at all

So.. your product is growing. More and more people are flocking to your website everyday. Sales were good, customer reviews were even better. Your hardwork has slowly being paying off, Congrats! But, unless you’ve hit the jackpot (what even is the jackpot?!), your journey is far from over. Its time to conquer new markets and product segments, branch out to more demographics and maybe finally address that pesky, ever-present, ever-rising demand for a dark mode. You add more devs and product people. The team slack channel posts “Here we grow again 🌱” every other day.

It’s all good… till, a pesky bug shows up in prod. The last release is rolled back and crisis is avoided postponed. You call the dev and QA team to a post-mortem meeting to understand what happened. A new feature went in and a seemingly totally unrelated bug rears its head on prod. The feature itself was tested, but everything around that - nope.

You’ll have to live with some of these growing pains, but, you can build an automaton to ease a lot of it. Run E2E tests pre-merge, post-deploy, daily, nightly and ever so rightly. Free up your QAs to do exploratory testing and fix processes.

Now that I’ve sold you on the idea of automating your tests, you can go on to make the crucial decision of which tool stack to use. In mid 2024, there is only tool that I can recommend whole-heartedly - Playwright (using TypeScript). There are other tools in the market though - Cypress, Puppeteer and Selenium being the others that I’ve used.

Technical - Chrome Dev Protocol vs The Cypress Approach

My arguments against Cypress are simple. I prefer using CDP (Chrome Developer Protocol) which performs the “User Actions” as direct API calls to an independant target browser. Cypress is an (Electron) app which opens a browser wrapped within itself and controls and monitors DOM events as the main trigger for “User Actions”. Both approaches have its pros and cons. Also, did I mention that Cypress gates its “premium” features behind a paywall?

This reddit post does a wonderful job of listing out the differences between Playwright and Cypress. Check it out if interested.

Playwright vs Selenium

Playwright vs Selenium is another battle altogether. The differences here are not a matter of “opinion” (imho :D ). Selenium used to be the de facto best choice for browser automation till mid 2020. I’ve spent a good part of 8 years automating tests and worker bots with Selenium (Java and Python).

Then, the folks who created Puppeteer for Google decided to jump ship to Microsoft and built what we now call Playwright. I haven’t looked back ever since, and Playwright is what I recommend without a shadow of a doubt for any new browser automation project being developed. Heck, I’d advice you to look into the ROI for Playwright even if you have an existing Automation suite written on Selenium. Its not a straight swap (Except the selenium hub part), but it might be worth your time because of the speed and stability advantage that Playwright offers.

Here are a few points of differences where Playwright blows Selenium out of the water:

Test Stability and Flakiness

Auto waits in Playwright:

One of Playwright’s standout features is its ability to provide reliable and stable tests. It automatically waits for elements to be actionable before performing actions, which reduces the likelihood of flakiness. For e.g., if you are clicking on a button, but the button itself takes a few seconds to be enabled, you do not need to explicitly wait for the button to be enable to click it. Just call the .click() method and it will be executed when the button becomes enabled.

When using Selenium, you’ll have to manage all that syncing yourself. It was done with implicit and explicit waits and that opens a big can of worms wrt test execution speed.

Forget about StaleElementExceptions with Playwright’s Locator resolutions:

Consider a test that asserts that a Login button is disabled till a user enters username and password. In Playwright, it will look something like this.

  //Locators
  const emailInput = page.getByRole('textbox', { name: 'Email' });
  const passwordInput = page.getByRole('textbox', { name: 'Password' });
  const loginButton = page.getByRole('button', { name: 'Login' });

  //Assertions
  await expect(loginButton).toBeDisabled();
  await emailInput.fill('[email protected]');
  await passwordInput.fill('XXXXX');
  await expect(loginButton).toBeEnabled();

Now imagine doing it in Selenium.

    // Locate the username, password fields, and the login button
    const usernameField = driver.findElement(By.id('username'));
    const passwordField = driver.findElement(By.id('password'));
    const loginButton = driver.findElement(By.id('loginButton'));

    // Assert that the login button is disabled before entering any credentials
    const isLoginButtonInitiallyEnabled = await loginButton.isEnabled();
    expect(isLoginButtonInitiallyEnabled).to.be.false;
    console.log('Login button is initially disabled');

    // Enter username and password
    await usernameField.sendKeys('yourUsername');
    await passwordField.sendKeys('yourPassword');

    // Wait until the login button is enabled
    await driver.wait(async () => {
        return await loginButton.isEnabled();
    }, 5000);  // Wait up to 5 seconds

    // Assert that the login button is enabled after entering credentials
    const isLoginButtonEnabledAfterInput = await loginButton.isEnabled();
    expect(isLoginButtonEnabledAfterInput).to.be.true;
    console.log('Login button is enabled after entering credentials');

The code is longer, the Assertions are not as helpful, and the worst part - this code will work 8 times out of 10. It’ll fail twice (seemingly randomly but always before a prod release) because referring to the login button for the second assertion will throw a StaleElementReference exception. This is because Selenium resolves the selector once in line 4, saves it in a variable of WebElement type and uses that webelement for further actions. Any time there is a change in the element, it will invalidate the WebElement that is saved in loginButton and any further references to that will throw the dreaded exception. You can get around this by not saving the state of an element and overloading methods like .click() to accept locators instead of the built-in methods. These extra methods are a must when creating a robust framework based on Selenium and adds up a lot of cost BEFORE providing any sort of test coverage. And, while good coding practices and additional libraries can improve stability, Selenium may still struggle to match the out-of-the-box reliability that Playwright offers. This can lead to more maintenance efforts and potential frustration for your QA team, particularly if they are new to automation.

It’s just better:

In addition, Playwright’s architecture is designed to handle network events and page navigations gracefully, ensuring that your tests are robust against variations in application speed or load times. This stability is crucial for maintaining a dependable CI/CD pipeline, especially when scaling up your test coverage.

Maintainability

Playwright offers a modern and developer-friendly API, which integrates seamlessly with a language like TypeScript. A combination like this ensures type safety, better code completion, and fewer runtime errors. For a team with JavaScript experience, this means you can leverage your existing knowledge, making the code more maintainable and reducing the onboarding time for new team members. Playwright’s consistent API across different browsers simplifies the maintenance of your test suite, ensuring that your tests remain manageable as your application evolves.

Selenium is a powerful and widely-used tool, but it can feel more complex to maintain. Managing different browser drivers and handling various configurations require additional effort. Without built-in TypeScript support, your tests in Java which is often not part of the modern language for a web development team or Python with its lack of type safety. This could lead to increased maintenance overhead, especially when dealing with a large and growing test suite. However, Selenium’s extensive documentation and community support can help mitigate some of these challenges.

Learning Curve

For a team with experience in JavaScript (as usual web dev teams are), adopting Playwright with TypeScript will feel intuitive. Playwright’s documentation is clear, and its API is straightforward. The TypeScript types provide excellent in-editor documentation and error checking, which significantly shortens the learning curve. Given that your development team is already familiar with JavaScript, they can effectively coach and train the QA team, accelerating the overall adoption process. Bonus points if the team is already using React Testing Library for unit tests (a lot of the concepts are same).

Selenium has a steeper learning curve, especially for a QA team new to automation. Managing browser drivers, handling different configurations, and writing stable tests in Java or Python can be complex. While Selenium’s flexibility and language support are strengths, they also introduce additional layers of complexity. Extensive community resources and documentation can aid learning, but the initial setup and understanding of Selenium’s intricacies can be challenging for beginners.

The Final Word

Pick what you think is the best for YOUR team and app, consider the current skill set of your development team and the need to train your QA team. Playwright with TypeScript is often the more compelling choice. It offers maintainable, stable, and intuitive test automation, leveraging the existing JavaScript expertise within your team. The modern architecture and built-in features of Playwright make it easier to write and maintain reliable tests, reducing the learning curve for your QA team.

While Selenium is a powerful and flexible tool, its complexity and steeper learning curve may pose challenges, especially for a QA team new to automation. Playwright’s ease of use, stability, and maintainability make it a more suitable option for your team’s end-to-end test automation needs.

My vote is always for Playwright (till something better comes along :D). Choosing Playwright with TypeScript has streamlined my testing efforts, enhance collaboration between developers and QAs, and ultimately lead to more reliable and maintainable test automation for the web applications that I’ve maintained.