🎞️ Videos → The future of cross-browser automation testing

Introduction and Speaker Background0:00

Hello everyone.

Let me introduce myself. My name is Ivan. I'm from Jakarta, Indonesia. Who's been to Jakarta or Indonesia? Who's been to Bali?

Same person.

I'm a Web GDE, Google Developer Expert for web, and I'm a senior engineer at Human Made. I am very excited to be here with you, and thank you for coming to my talk today. This topic I'm so excited to share with you

because I've been waiting it for a while.

Testing for Software Reliability0:52

This is about testing. To test your application to ensure its reliability

when you have upgrade, or when you have any additional features, you ensure your software still do what it does

and still reliable.

Common Browser Automation Challenges1:14

Speaking about browser automations in your life,

has any one of you feel like when you code or you make something for your application, you feel like, "It works on my machine." But in client's machine, it doesn't.

Also, browser automations is supposed to be when you test on your machine, it's fast, but when you test on the cloud, or you call it continuous integrations or CI, it's so slow.

Sometimes, it burns your GitHub action time, and you have to pay a lot more money for it.

And it's only run on certain browser, unfortunately. We have the same problem.

Overview of Browser Automation Tools2:09

So, how can we solve it?

Tools that we use to automate our browser usually cut one of this. It's either WebDriver, Puppeteer, or maybe you use Playwright, TestCafe, or maybe you use Cypress.

How Browser Automation Works: High-Level vs. Low-Level2:30

So, how it works, actually? It's on a high level, and it categorize into high level and low level. On the high level, usually where we use it, we use either like Cypress or TestCafe. We write our test in JavaScript. We interact with the web APIs through Node.js like, "browser, navigate to this link, and click this button, or click that element." And you try to run the end-to-end testing. For example, you run a login process, like you go to a login screen, fill in the username and password, and login, and to ensure the login still works. On the low-level part, we have two protocol. It's either WebDriver classic protocol,

and we use Chrome DevTools.

As the name suggests, Chrome DevTools is only works for Chromium-based browser.

WebDriver Classic Protocol: How it Works and its Limitations3:39

So, how the WebDriver works? It's usually like this, like your automation tools, like 𝚠𝚎𝚋𝚍𝚛𝚒𝚟𝚎𝚛.𝚒𝚘, it use the browser driver. It's either ChromeDriver, Gecko for Firefox, Safari, and Edge driver. It's already built-in inside your browser. You can use it, and it interact from your automation tools to the browser. Let's take example of this simple simplified demo. You navigate to a URL,

and you click on the Espresso. So, how it works on the 𝚠𝚎𝚋𝚍𝚛𝚒𝚟𝚎𝚛.𝚒𝚘 code? It's easy. You set up the browser: Firefox, Chrome, Edge, Safari. You ask the browser, the driver, "Go to this URL." And then, click the Espresso link. That's it. This is how it works inside, behind the scene. From the WebDriver, it navigate, it send HTTP request. And then, the browser driver come back with the same HTTP response like this. It's nothing special. It's just HTTP, you request, response. It's like a REST API endpoint. That's how it works behind the scene. But the problem and the challenge comes with this number two. What happened here is when your automation tools ask,

"navigate to this page." But the element that you are asking or you're looking for is not yet available because the browser still render it. What happened is behind the scene, it has a long polling. Like it keep asking, "Is the Espresso loaded? Is the Espresso loaded? Is the Espresso loaded?" Nope. And until it's time out, then you wasted your time and money on a GitHub action or Bitbucket time.

So, it's so slow.

But the good thing with the WebDriver, it support cross browser. It works on Safari browser, it works on Firefox, it works on almost every browser. The not-so-good part, it is slow and only support high-level control.

CDP (Chrome DevTools Protocol): Benefits and Drawbacks6:34

Okay, the second part is CDP, Chrome DevTools Protocol.

As the name suggests, it's only for Chromium browser. And of course, you know, Puppeteer. It's behind the browser automation for Chrome.

How it works is, it doesn't use the browser driver. It directly interacts through the browser, through WebSocket. And since it's used WebSocket, it's bidirectional by default.

So, how we set up on Puppeteer is the same. We set up the browser. We ask the browser to go to the URL, click it. Now, how it works behind the scene is, it sends everything, all the command, all at once, not all of them, but just asks without waiting for response. So, it's not HTTP request and response anymore. It's just send it and forget it and waiting for the browser to reply through the WebSocket. So, there is no long pulling. That's why it is fast, bidirectional.

And it also supports low-level control, such as, it can intercept network requests. It can also simulate device mode, like in Puppeteer, you can define you want to simulate mobile.

You want to simulate geolocations. And you want to subscribe to console messages. And while WebDriver doesn't support this low level. Thus, in summary, CDP or Chrome DevTools Protocol, it's fast.

It can access low-level control, but it's not a standard. It's only available in Chrome-based browser. It's not inclusive.

Introducing WebDriver BiDi: Combining the Best of Both Worlds8:42

So, how we can improve this? And why I'm so excited about this, is, how can we combine both of the best from these two worlds? Like, how can we have a fast bidirectional from CDP and have a cross browser from WebDriver?

WebDriver BiDi: A New Web Standard for Browser Automation9:04

So, if you have two tools and we combine, and tada! It's called WebDriver BiDi. It's called BiDi, not BD. It's called bidirectional. So, it's a WebDriver and it supports WebSocket direct bidirectional. So, with WebDriver BiDi, it uses the same protocol,

I can say it's a new web standard, which is getting the technology from the Chrome DevTools Protocol and make it as a new standard as a web next generation of WebDriver. And it uses WebSocket. WebDriver BiDi is in collaborations between browsers vendor,

automation tools like Selenium and webdriver.io, and it's also testing provider.

Integrating WebDriver BiDi into Existing Tests10:07

So, we know, we have a new tools. We have a new standard. And how can we propose this to the current test? As a good thing with WebDriver BiDi is, you can upgrade your code without touching or refactor your code at all. All you need later on is just change configurations to make it work with WebDriver BiDi. For example, webdriver.io version 9 already has initial support for WebDriver BiDi. All you need is later on, I will give you example, but it's just a simple configurations, just one line of configurations, and you can use WebDriver BiDi in your test.

As same as Puppeteer. Now, Puppeteer already use Chrome and Firefox behind the scene. And all you need is just active, Firefox already support WebDriver BiDi by default, while Chrome still use CDP, but you can change it to use WebDriver BiDi.

Example: Using WebDriver BiDi with WebDriverIO11:21

Okay. Now, let's take example of this page. And now we add one more, which is not possible for classic WebDriver protocol before,

which is catch a console errors log.

Same like WebDriver WebDriverIO, you can just add WebSocket URL.

That's it. Now, your test can use WebDriver BiDi capability.

And then, you can use either Chrome or Firefox. And that's it. Safari? Where is Safari? Not yet. I hope so. Okay? Soon.

Monitoring Console Logs with WebDriver BiDi12:12

And now, to monitor log messages,

all you need is just like how you do it with CDP

or you can subscribe to your session and then you can see the log entry on the console log. This is useful if you want to test to ensure that your app doesn't throw any JavaScript error or if you're expecting your application to throw any error, so you can make an assertion or test about it. You can do this now with WebDriver BiDi using WebDriverIO. It's not possible in WebDriver version 8 below. It's only possible on WebDriver version 9 above. So, you subscribe to it through the WebSocket, then you can listen to it and make an assertion if any JavaScript error occurs, and you can ensure your application doesn't throw any error

during your test. This is an example. You can run and then you can see assertion error here

and it's coming from those log error messages. So, WebDriver BiDi can monitor the console log caught exceptions as early as possible. One thing that I found useful is when I combine it

Combining WebDriver BiDi with Web Vitals13:52

with the Web Vitals Core Web Vitals, there is a web-vitals.js. You can combine it with web-vitals.js and you can log your message where you can track your LCP, Largest Contentful Paint, or your INP, Interaction to Next Paint, and CLS. So, you can log those data during your test.

You can ensure that your application doesn't regress in performance while developing your application.

This is useful because you can output your text inside the console log and capture it inside the test.

WebDriver BiDi Roadmap and Interoperability14:39

And plenty more, you can see the Web BiDi roadmap and there are more controls they are building into this protocol.

And it interoperates as I said before, you don't need to change your code if you're already using Cypress, TestCafe, and all other tests. It has support by default if your application

or test already supports Firefox and Chrome and Safari.

It will switch between itself. If the test is running on Chrome and Firefox, it will use BiDi by default.

Firefox uses BiDi by default. Chrome uses CDP, but you can set it. While if the test runs on Safari, it will fall back to the previous protocol using WebDriver classic. Of course, the monitoring console log doesn't run, but it doesn't break your code. Your test will still run as usual. So, that's the interoperability and backward compatibility with this protocol.

Using WebDriver BiDi with Puppeteer15:54

And same goes with the Puppeteer code. You can set up the browser. All you need is add the protocol: 'webDriverBiDi'.

And it will use the protocol behind the scene automatically. While the same like the monitor log messages, you can do the same. You can already do this if using Puppeteer with Chrome, but now you can use this if using Firefox with WebDriver BiDi.

And that's all, and thank you.