Videos → Unlock the power of visual testing with Vitest

Introduction of Maya Shavin and the topic0:00

For the next session is Maya Shavin. She is very popular speaker for Vue community. I have follow her for many many conferences. Many of her is about testing. If you like about testing or unit testing or

end-to-end testing, Playwright, you have to follow Maya Shavin. I am the one that follow her for many conference. Besides from testing, she is one of the popular contributor of Vue community. Maya Shavin really want to join our event.

She really want to speak in live session, but

she is unavailable. But she try to make the pre-record for us. Especially this session is the new topic, the new session from Maya Shavin. We are the first event that can watch Maya Shavin session. So please welcome Maya Shavin!

About Maya Shavin - Speaker, Author, and More1:15

Hey. Welcome to my talk. I'm really sad, what a pity that I'm not there in persons. But welp… I hope next time we will see each other in person. Meanwhile, hope you enjoy the first edition of Vue.js Conf Thailand and my talk. Let's talk about testing, shall we? First and foremost, my name is Maya Shavin and I'm a senior software engineer at Microsoft and also an accessibility champ in my group. We oversees accessibility compliance and trying to provide best practices and guideline to help developer comply with accessibility during their development before release. I'm also Cloudinary ambassador, Vue.js Israel organizer, and a Google Developer Expert on Vue on web technologies. You can follow me on mayashavin on Twitter, LinkedIn or Facebook or visit my website at mayashavin.com where I like to talk and write about all the experiments and what I learn during my journey in high-tech. In addition, I'm currently the mom of 4 kids, which is actually 3 and a +1.

My latest kid is about 1½ month, so I'm still in maternity leave. But hey, it's fun to be back to talk about things that I'm passionate about. I'm also an illustrator in free time and I am a book author, which yeah, this is my book, Learning Vue. I'm so excited to have this book released this year. If you are interested in learning Vue with TypeScript, check my book out. I hope you enjoy it. That's it about myself.

What is Frontend Component Testing?2:58

Let's go back to our main topic today. First and foremost, let's talk about component testing. In a frontend context, or we can say frontend component testing. So what is frontend component testing? In the sense of normal development and

especially for backend, component testing is like modular testing or we can call it like unit testing. But for frontend, it's a bit different. For frontend component testing, it means that we have to validate the GUI element are functional as expected. GUI element means graphic user interface element, which we validate also the style, the class, the hooks, the component props, the data flow, the interaction within the component with other components and with user. When we talk about graphical user interface element or UI, we are talking about validate appearance, how it looks, how it functions, not just the functional testing. Frontend component testing means we validate

how the component looks and appear to the end user on the browser and how it functions or interact with the user or with other components under the same parent or with its children components. Once you validate things like this, you also need to validate the consistency or regression test between change, between feature added, remove, between bug fixes on different device, on different browser, and so on.

Two Aspects of Frontend Component Testing: Functional and Visual4:44

For this, we divide frontend component testing

into two biggest aspect. The first one is behavior or functional testing including how the component functions. So hooks, how it process data, how it take the interaction of user and reply to the user and so on. The other aspect of frontend component testing is visual UI testing or UI testing. It including how the component appear on the browser to the client, to the end user. How it looks, color, image, styling, layout, and so on. We also need to validate the consistency of the look whenever we add a new feature, we add a new button, so on and so forth. Not just test how it appear. We test how it appears, how it looks on different device, on different browser, on different operation system. And this is our main topic of today, visual testing or UI testing.

Visual Testing in the Testing Pyramid5:55

But hold on a minute.

We know the functional component testing including visual testing or UI testing. Remember this pyramid, which we have unit test, integration test, and end-to-end test. So where this visual testing or functional component test of a frontend component testing fall into? You can say it fall into unit testing or can fall into end-to-end testing. It makes more sense with end-to-end testing because in end-to-end, you actually simulate the user actual flow on the actual browser.

But we also can have visual testing on unit test by snapshot comparison, for example. So, In this talk, we're going to explore what the difference between visual testing in unit test and end-to-end test and how we can combine these two in the same testing system for visual testing of a component.

Demo Component: Pizza Store Search7:06

Before we move on, we're going to take a look on the demo component. In this demo, we're going to use a search component from a demo app that is Pizza Store. It has only three features: receive the search term and synchronize the search according to what the user type and of course, it will emit the search term, the new search term and the filter item that has within, beginning with, by this, the new search term. Okay, simple. You can implement it in any way in Vue application, in a Vue component, but these are the main features that we're going to test on.

Visual Testing in Unit Tests with Vitest7:47

Okay. The first thing we're going to look at is visual testing on unit test with Vitest.

Okay. So, let's recap what is Vitest.

Vitest is a test runner that is super lightweight, super fast, and it's a headless test runner in a Node.js environment. It was created for Vite projects, which is JavaScript projects, and mainly used for unit test, integration test that's written in Node.js. And it's not only for Node.js. We can extend it to work with browser, mimicking the browser and configure it with other package to make sure that Vitest can run on specific framework application, such as Vue. So for Vue, when you install Vitest, you need to install also test-utils, Vue Test Utils or the package for testing library in Vue in order to get the set of APIs

for assertions, a component in Vue and also be able to get a sets of methods that help you to render a component in a Vue environment with all the capability that you needed for running the test successfully.

Okay.

Visual Testing with Vitest: Limitations9:09

So let's talk about visual testing with Vitest. So for Vitest, like I said, it's a very generic test runner, so we won't have any

browser support here because we are using the Node environment. For browser environment, well, because it's in Node environment by default, when you run a test on a component which is for front-end, for browser, you need to have extra implementation for DOM API and HTML standards, such as jsdom or happy-dom. These package will provide the APIs like query selector, like to have attribute and query by role and so on, so that you can assert it according to the DOM API or even window object, for example. It does have visual testing by default using snapshot comparison with the function toMatchSnapshot, but very limited and why, we're going to take a look in the next slide. For visual testing with Vitest, since it's a generic test runner for unit test and integration test, you cannot test per device, you cannot test per browser, and you cannot add behavior, accessibility testing. You can test the ARIA standard, but for

anything that require navigation, user interaction, actual user interaction for accessibility, this doesn't work. Okay. Let's take a look how we set it up and we try our test. So, first of all, you need to set up the environment, jsdom or happy-dom, whatever you feel like. And then you can just use the function, just mount it. mount the component at the wrapper and then expects the mounted component to match the snapshot. That's it. We have this visual testing for in Vitest

implemented. In the first run, it would generate the snapshot in the locations that we define in the <i>vitest.config</i> for snapshot. And then you can open the snapshot and then whenever run the second run, it would try to match the snapshot with the new snapshot and see whether it's the same. Okay. So we can take a look at the snapshot and the reason why I say it's limited because the snapshots is actually virtual node. It's mean node JavaScript.

And you can see here that is not something that you can look and understand straight away what is this component. And whenever you make a change, let's say I changed the label to search label, the snapshot will fail. In order to make sure that the snapshot updated, we have another command called <i>vitest -u</i>. And then when you run this, it will run like the normal <i>vitest run</i>, but it will automatically update the snapshot to the latest code. And this is something pretty dangerous because when you do something like that, it kind of erase on the idea of snapshot testing because you can just update the snapshot and you never find the bugs. You never see what went wrong and what can be wrong. And since it's hard to read, it's not very maintainable or reliable because no one can

understand what they written in the snapshot except for tree and object and so on. But the more the bigger component, the harder to understand it. Okay.

Visual Testing with Playwright: End-to-End12:58

So next, so that is visual testing in unit test in Vitest. What can we do with other end-to-end test runner such as Playwright for visual testing? Let's take a quick look about it. So Playwright as the test runner is the browser-based runner for end-to-end test in web application. So it's pretty freaking fast and isolated and it have component testing features that is specifically for testing component, not testing the whole application or whole page. And it cross language. So if your project have two different language, one for backend, one for frontend, using Playwright can be very beneficial for your application management.

Okay. So let's take a look how we enable visual testing in Playwright. First, we using component testing. So we go to install something called <i>@playwright/experimental-ct-vue</i> package for component testing. And when we install it, it will auto generate for us Playwright config specifically for component testing with all the browser, the basic browser we want to test. And then we can go to the spec like test

file, import the <i>test</i>, <i>expect</i> from this package and write the test as normal because we in Playwright, we can write JSX here. And then we can check use the function <i>screenshot</i> and we pass the path where do we want or what format and what name of the screenshot they want to generate under. And we can also use full page, but for component, full page will not do anything much. In other, we can use the screenshot function of page or method of page. Similar syntax, we can pass the path and we can enable full page to make sure they

capture the whole page as this is in the browser. And we can run it. And this is how we run the test.

Visual Testing in Playwright: Screenshots15:06

Okay.

Now if you look at the screenshot, so for component screenshot is only the component. But for the page it's display the whole page. So really depends on how you want to do it. With this screenshot, you can actually do manual, look at this and see whether it's render correctly, because render as is, so you can really see look and feel, what you see what you get. And you can also upload it to a cloud, integrate with the third party smart tool to do comparison between the two screenshot and see which one doesn't match in the

automation process. So we also have full accessibility testing

with additional package as core Playwright. And it sounds cool, right? But that's not the point of this talk.

Can Playwright's Power Be Used in Vitest?16:10

Here comes the next question. So that stay along. Obviously, Vitest have more limited Playwright have a lot more advantage towards end-to-end test or visual testing for component since it have component testing also. But the question in this talk is, can we have the similar power of Playwright in Vitest to support full browser visual testing?

Vitest Browser Mode: Combining Unit and Visual Testing16:39

Well, the answer is yes, kind of. With Vitest browser mode, you can have this together in the same Vitest, in the same test runner. How we going to do it, especially with Playwright? What we do, we install the package called

<i>@vitest/browser</i>. This package will enable the browser features, more features for Vitest. And then, we write a command

<i>npx vitest init browser</i> and it will walk it through the configuration automatically to set up the environmental configuration ready and even the sample test ready for browser mode. And yes, of course, we can choose Vue, we can choose another browser, Chromium, or we can choose another browser provider, not just Playwright, can WebDriverIO or something. In this talk, we're going to use Playwright. And then, that's not it, we need to do extra setup. Because now in our application, we have unit test and we have browser test. And this doesn't really work together because each test, the test runner need to know different mode, different configuration in order to run the test correctly and not throw error. So we need to provide different test

configuration. Like in here, you can see one is for unit test where I set the environment to be <i>jsdom</i> and pass the include

directory of files that's meant to be for unit test. And for browser test, I have the object browser where I enable the browser and pass the target browser, the provider,

and other options of the provider inside the configuration. And with that, I add another command to

point the command Vitest when to run the Vitest. With Vitest, it will use this workspace file instead of the normal <i>vitest.config</i> file.

Demo: Visual Testing with Vitest and Playwright18:49

And with that, let's go to do a quick demo. Okay So in this demo, this is the search box. And I already have…

The component. And now I already have the end-to-end test right here. So we can copy it and move it to browser.

And now, instead of using the component testing view package, we can just use the normal Vitest package. Now we're going to update the route path to the component. And because we are in Vitest and not in Playwright native environment for writing the test, we cannot use JSX. Instead, we use the normal mount and we do the prop. And of course, this will not be available here because it's not on the server and it's not a Playwright going through Vitest. So instead of mount, we using render function from Vitest browser view. And then locator, some of the native

function of Playwright will not present in the browser mode of Vitest yet. So instead of locator, we will do getByTestID and instead of ID, we put the

test ID which is the search input. And here we getByText where we get the text of the component by passing the text of the label. And we keep everything the same except for component doesn't have a screenshot. Instead, we will use the element where we take the screenshot of that element, the input element. And the page not here. So instead, we will use the page from context of Vitest browser context package.

And we will just pass element because this

one work and since a label is the locator, it's always true and we will return the element and we can do the actual element to be truthy.

And to have value, element is a locator and not an element. So we also need to use the same function element to get that. And similarly, we don't need this test. Just for the purpose here. So now let's run the test.

Once I run the test, you can see here. It will run the test. It run the unit test and then it start running the browser test. And you can see the test and once it's done, it shows us a GUI where we can see all the passing running and each of the test will have the label to tell us exactly whether it's the unit test or the browser test. And you can see the UI, can interact it, can check in different device and we also have a dashboard.

Awesome, right? And that's it. We have both visual testing together with

unit testing for a component in Vitest with the help of Playwright very little and we don't need to install the whole Playwright standalone. It do everything for us. Cool.

The Pros and Cons of Vitest Browser Mode22:26

Before we finish, what we expected for

Vitest browser mode with Playwright? Well, the pros are obviously we have everything in one place, everything. And we can also actually testing our component in the state of the isolated mode with the actual browser, with actual UI and with actual data. And we can have one UI to see all the report on the status of the test. And we don't need to run different command and different mode for different type of testing. And of course, the biggest disadvantage of the mode at the moment is that it's in preview mode. So some of the functionality is not there yet. But I'm sure it will be there very soon and I'm sure that Vitest team is working very hard on that. But meanwhile, check it out. The browser mode, you may not need component utility testing anymore. Instead, you can use unit test plus component testing with Vitest browser mode and Playwright.

Conclusions23:35

And remember, test what a component does, not how it does it. And thank you. I hope you enjoy my talk. If you have any questions, feel free to talk to me.

I will be happy to answer any questions and see if I can help you in any case. And have a great day and enjoy the conference.