🎞️ Videos → Build your own Svelte 5

Introduction and Welcome to JavaScript Bangkok0:00

Okay, hello everyone.

Good morning JavaScript Bangkok.

วู้! สวัสดีครับ Thanks for the organizer for letting me speak today.

I know we have some technical issues in the morning, but I think we all should give another round of applause to all the organizers to make this happen with such a short time. So thank you for everyone.

Speaker's Background0:39

I have a few proposals to come to Bangkok. I submitted a few proposals, but Phantip told me that you guys want to have a coding talk first thing in the morning. So I hope everyone is energized and ready for it. Let me introduce myself. I'm a frontend developer at Shopee. I'm based in Singapore. But I'm born and raised in Malaysia, in a beautiful town called Bukit Mertajam in Penang, which is very close to South Thailand. My parents used to go to Hat Yai over the weekends. This is my first time in Bangkok. So let me know if you have any place that you can recommend me that I should be visiting. Outside of work, I do a bit of open source, and this is my Twitter account. I also have a YouTube channel doing content about frontend

that I wish was there when I first started out. Two years ago, I talked about "Build Your Own Svelte" at a conference. Back then it was about Svelte 3. And two years later, which is now, we now have Svelte 5 at release candidates. So I want to challenge myself to do a similar talk again, but for Svelte 5. This is my first time attempting it. This is the premier in JavaScript Bangkok.

What is Svelte? A Compiler-Based Framework2:08

So what is Svelte? Well, Svelte is a compiler-based framework. This is how you write a Svelte component. The Svelte syntax closely resembles HTML syntax. You can define variables in script tags and use it in HTML using the curly brackets. And you can define CSS in the style tag. So in this case, we've made the text in red color.

To make changes to variables, you have to mark the declaration with state runes. With the $state, you can modify them directly like how you would modify a variable in JavaScript. Here we add a click event listeners with onclick, which calls the decrement function, which then modifies the counter by doing counter minus 1, and then it just works. It's that simple.

Svelte 5: Runes, Reactivity, and Signals3:04

Svelte is a compiler-based framework. The code you see on the left is what you write, but it's not the one that is sent to the browser to interpret. The code you write on the left goes through a compiler, which analyzes and compiles it into JavaScript, the one that you see on the right. This is what is sent to the browser. Because of this compilation step, there's a lot of magical things you can do. But sometimes more magic doesn't mean it's always better. In Svelte 4, any top-level variables is magically

analyzed and tracked during compile time for reactivity, which is very magical. It works like most of the cases, maybe 90% of 95% of the case, but sometimes it doesn't work. And when it doesn't work, it can be very frustrating. In Svelte 5, we introduce runes, which is like a marker for the Svelte compiler. You have to mark the variable when you declare, you use the $state runes to mark it. And then the compiler doesn't have to guess based on heuristics to figure out whether the variable is being used or modified.

We know for sure what variables need to be reactive because it's being marked by the developers using runes. By using runes, it also opens up new opportunity, which wasn't possible in Svelte 4 or the previous versions of Svelte, which is universal reactivity, where you can move the code for reactive states into reusable functions. You can copy the same code, move into a function, and it will still work. You don't have to rewrite the code in a totally different way to abstract it out, like how you would do in Svelte 4 or the previous versions.

Or you can also move it into a separate JavaScript file, and it still works. In Svelte 4, we simply can't track reactive states beyond the Svelte components. And that is where the magic breaks down. In Svelte 5, as reactive states no longer bound to a component, it can be created anywhere. So we stop trying to figure out how state changes affects the component UI or side effects during the compilation time. And now we leave the reactive dependency tracking to runtime, which we use as a concept of signals and achieve a fine-grained reactivity. We'll talk about that later on. In this talk, we're going to talk about how to build your own Svelte, which is to implement Svelte compiler during a talk, but not learning how to use Svelte. If you are interested to learn more about how to use Svelte, you can watch my YouTube tutorial.

Understanding Compilers: Parsing, Analyzing, and Generating5:55

And so we mentioned that Svelte is a compiler. The word "compiler" sounds very scary. But let me try to explain to you in simple terms. In general, this is the breakdown of what a compiler does. It takes our code, it parses it, do some analysis, and finally generates an output. Let's take a closer look. When we say compiler parses your code, it takes the code, generates a representation of code, usually in a form of a tree structure, which we call it an abstract syntax tree.

It's called abstract because it contains the abstract structure of your code. It does not contain all the details such as semicolon, parenthesis, spaces, or tabs. It just contains the structure and meaning of the code. And then the compiler analyzes your code. What this means is that it recursively goes through every node of the AST to gather information such as the JavaScript scopes, variables, looking for areas for optimizations.

And then the compiler, with this information...

The screen is not working. Hold on.

With this information... Please.

Okay.

With this information, the compiler is able to generate a more optimized code. Today, with the time permits, we are going to implement

Implementing a Simplified Svelte Compiler7:42

a very simplified version of Svelte. And this will be our components that we are going to compile. Any Svelte features that you did not find in this component will not be implemented. And this example, this is the counter example that we just saw just now. Let's take a look at my project setup.

So this is my project setup.

Let me hold on the mic for a while.

So this is my project setup. Let me zoom in and we can take a quick look over here. The first thing is that, as you can see, this is a basic JavaScript project where we have our package.json. And I have gone ahead of time to install all the necessary dependencies that we need over here. And this is basically where we're going to write our compiler. And the source folder, this is the components that you just saw earlier on. And we're going to build this into the dist folder. So, first thing first, I think we are in a hurry so I have to type very fast. First thing is I'm going to read the file content from the app. And then we're going to do three things: parse, analyze,

and generate. And after it generates the JavaScript code, we need to write it into this folder like this.

So we have the parse, analyze, and generate.

I think we are very delayed so I have to write very fast.

Examining Svelte 5's Compiled Output9:44

We keep mentioning about Svelte compiler will compile in compile time into JavaScript. But how does that even look like? This is where it's going to be a bit different from my previous talk where we talk about Svelte 4 and the versions before and Svelte 5. If you've seen my previous talk before, or played with it, you notice that the compiled output for Svelte 5 is going to be different and much smaller. For this talk, our simplified version of Svelte 5 will be

based on the output from the Svelte 5 output.

Let's take a look at this button over here. What's the best way of creating a button like this? It's going to be creating an element, using the DOM API 𝚌𝚛𝚎𝚊𝚝𝚎𝙴𝚕𝚎𝚖𝚎𝚗𝚝 and 𝚊𝚙𝚙𝚎𝚗𝚍𝙲𝚑𝚒𝚕𝚍.

These two methods, we're going to use many times. So, let's extract it out into element function and also extract it out to the attribute function. And we want to be able to create this component, this button, multiple times. So, let's wrap it around with a function. Then when we want to mount or create this component, we can call the add function with an element that we want to create the component in, which in this case is document.body.

We're going to use this element and attribute function multiple times as well. So, we're going to move them over into a separate file that looks like this. And so, we're going to import them from runtime.

I actually went ahead and created this file. If you take a look over here, I have created these functions over here as well.

So this is basically what the runtime is about.

Let's go back to the slides.

Generating Code Element by Element11:54

We're going to move on to element by element, like the buttons, we have this callback. And then for this text, we're going to create the text element with the text function. And we'll move on one by one. That step's basically we go through every node. And maybe you will ask me one question. So, what happens for the counter, which is dynamic expressions? Is there any special handling that we need to do?

Well, we're going to keep that thought right now. We're going to come back to it later. And maybe another question would be, this visualization

looks simple from the screen, but how are you going to

do it in your code? Because ultimately, the code that you see when you pass it into a compiler is just a string. How do you understand the string, what are the elements there? That's why we need to parse our code into AST, a tree structure. It will be much easier if you visualize it with a tree structure because going through element by element is basically traversing through the node one by one, using depth first search or breadth first search.

Writing a Parser and Defining Svelte Syntax13:03

Now this basically begs the question of, how do you write the parser? How do you parse the string into an AST? Like all things in programming, there's always a technique to do it. First of all, and also like all things in programming,

you need to first have a PRD. A design document so that you implement exactly like the documents unless you be questioned like why you do something different. So, the PRD or the design document for

A parser will be something like, use a syntax notation to describe it, right? So, a syntax notation can be, there's multiple ways of describing a syntax. You have the railroad diagram. You can have the Backus-Naur form. And in this example that I show you over here, describes how a JSON string will look like. Okay? So we know how a JSON string looks like, right? That's usually what a JSON will be passed through your API calls. And a JSON object string, right, the JSON object itself is described with, it can be either open and close curly brackets, so that describes an empty object, or the open curly brackets with a list of

properties and a close curly brackets. And you ask me like, what is inside a property list? Well, the syntax for that will be either one property or multiple properties, or a property list with a comma and then a property. And you can see that there's some recursiveness in the syntax as well. And like, how do you describe the syntax for property? Can be a JSON string with a colon and then the value.

Right, but so, you roughly get the idea of a syntax document, syntax notation of a JSON string. Right? And today we're going to talk about Svelte, right? So how's the syntax of Svelte? Well, for Svelte, I'm going to define it as a list of fragments. And the definition of the syntax for fragments can be one fragment or a fragment with more, one fragment, So it can be, it's recursively many numbers of fragments. And a fragment, the syntax of fragment can be the syntax of scripts and then the syntax of elements if that's available, or expressions or text. Right? And then the syntax for the script is basically the angle bracket script and a close angle brackets with some JavaScript inside and an angle bracket slash script, right? And we can basically keep going on and on, right? Like, for example, expressions could be a curly brackets with some JavaScript and a curly brackets. So this describes what's the syntax of a JavaScript,

sorry, describe the syntax of Svelte, right? So, let's, I initially planned to implement this whole thing in a talk. Even if I try to type very fast, I still couldn't finish the whole talk within 40 minutes. So, if you want to see me code it live, like going through the syntax documents, you can watch my previous talk. You can search for "Build your own Svelte", but for this talk, I'm going to still do some copy pasting.

Implementing the Parser16:26

That's faster than typing very fast.

So, where's my cursor again? Oh, it's here. Okay, so I prepared this. So this is what I'm going to copy. Right? And I'm going to paste it over here.

Right? And let's see.

Right? Okay, and I also need to import Acorn as a JavaScript parser to parse the code. And basically, let me quickly go through, right? So how it looks like is that you have the functions for each of the parsing, right? You parse the fragments, fragments, scripts, elements, and stuff, right? And if you look at fragments, right, in our design previously, we say that it's either scripts, elements, expression, or text. So that's how I'm going to write it. And for expressions, it's like a curly bracket.

So if it matches a curly bracket, we're going to consume a curly bracket, take in some JavaScript, and then consume another curly bracket. And that will be our expression node.

Right? So, let's see.

So let's try to run this code first, right? So, let me comment some of the things out, like this.

And so this is what we have, and let's run.

So this is the AST, the structure that we have. Let me just try to maybe save it into a file.

This is me slow typing of AST or JSON.

Let's save this. And basically, you can see that this is the syntax, this is the tree structure for the script as well as the HTML where we have the buttons and attributes, the text, and expressions and stuff, right? So now we have our parser.

Generating Code: Walking the AST18:31

Let's try to go back and, let's see, close this.

Let's reset. Let's come back and move on to the next thing, which is, why is it my, sorry.

Oh, okay. Got it. So we're going to move on to generate the code, right? So to write the generate, so we're going to skip the analysis part because I think today, for the lack of time, we probably don't need to analyze too much of the code, and we jump straight to write the generate function. So I'm going to have the code that we just saw that we ran through earlier on over here on the screen, So this is the code that we need to generate, right?

So let's move on to writing this.

So first thing is that let me collapse this. Okay, so first thing is we're going to go through the writing the generate function. So we're going to have the string where we're going to create an array, and then we're going to join all the, we're going to push to the string array, and then we're going to join all of them into with new lines. So the first line, sorry, let me try whether I can

see. Oh, okay. So I can't. Okay, anyway, so over here, I have the imports and

the Okay, so here we have the elements, right?

We have the import statements and a function. So that's what we're going to write first. And then we're going to close the brackets. Then we're going to walk through our AST one by one, go through every nodes. And for this, I'm going to need a library called estree-walker. And with a 𝚠𝚊𝚕𝚔 method to walk through, we're going to basically walk the HTML, right? And it takes in two like the node that we're going to traverse through and then it takes in a state which I'm going to explain later on. And then this will be the visitor which I'm going to explain later on as well. So for the state, it's something that you can pass in as like a state of that traverser tree traversing. And we can use it when we traverse to each of the nodes.

And for the states, what we're going to store here is we're going to keep basically know what is the parents that I need to insert the elements into, right? So the initial node is basically this parent that

we're going to get from the function. And then for visitor, basically we can look over here what are the types of elements, right? So the first one is elements. So I'm going to visit this node. So over here, I'm going to write like elements, right? And basically we're going to create some variables to create elements and you've seen this where we can iterate through the attributes and then creates the nodes. Basically this will look exactly the one that we saw earlier on in our slides where we create an elements and attribute for the elements.

Then so for the attributes, right?

If you take a look over here in the AST, it's expressions. Basically it's JavaScript. So I'm going to turn JavaScript in AST into string. So that's what we're going to do. We're going to import a library called print from esrap. Basically it's a reverse of parse, that turns parse is to turn string into AST, but this is to AST into back into the string. So here we are going to call the print to print it

into the string code. And then we're going to look through all the children's. And we're going to go for every node as well, the text and expressions, okay? So now with this, I think we are good to try,

Testing the Compiler and Initial Output22:53

give it a try to run our compiler. And if you can take a look at the compiled code over here, basically it looks like this, right? So we recursively going through every nodes and generate the strings needed. And here, I think the button itself needs to deconflict,

So we're going to do it real quick over here.

Sorry, where's my... Hold on, where

Yep, so I'm going to do this. And now, basically, the generated code, we have adding the index at the back, and this probably will create the elements that we need for our code, right?

So, at the same time, I think I'm going to start the dev server, so you can look at what's the generated code so far. Let me zoom this in and also inspect elements. So, this part basically we haven't added,

we've created the code, but we haven't include the definition of decrement and increment from the scripts, right? So that's what we're going to do. We're going to come over here and do that real quick. So we're going to print the script into the function as well. And basically that will give us this.

I mean, I think switch this like this. Okay, so basically this gives us all the code over here. And I think it's ready to run.

Oh, okay, state is not defined. Hold on, let me try to quickly fix this.

I think we can add over here.

And if you refresh, you can see basically we have the elements created through our compiled code in our script. Right, so although like the function is not working, but basically you can trust me that it's everything is created nicely.

Making Reactivity Work: Introducing Signals25:03

And the next thing we need to do is to basically figure out how to make the reactive works, right?

So here, I think I need to quickly fix a bit where the part where we,

I see where the part where we, oh yeah, we need to change all, we're going to walk through all the scripts and change all the things that we use the dollar states into the dollar.states, right? That's something that we just now was fixed in our output. So now we need to go back to our slides and figure out how we make the reactive if works. Right, so we are still here, trying to figure out like how do we make sure that this updates. Right, over here, we have an expressions in a template. We need to know the expressions or any variables that's being used in this expression, when does then change, right? Or whether for the variables, we need to know like how is it being used or when is it being used, so that we know when what we need to do when that variable has changed. But how? Well, let's take a step back. Right, so I know we didn't notice this,

but every time when we have variables,

what we would do is that when we pass variables around, we're actually doing two things at once. We're trying to get the reference of the variable and also read the value from the variable itself, right? So, for example, when you pass a variable to console.log like this, you're actually referencing to the variable itself as well as reading the value, right, and printing out to the logs.

We kind of know that it will read the value from the variable object because we see the value printed out in the logs. But for the variable itself, it has no way to know whether its value is being read, right? For example, I pass this variable into a function called doSomething, I wouldn't know that whether my value is being read.

And why is that important? Well, it is important for our case because for our state variables, we want to know which elements or like when or how it's being read, right? If no one is reading this value, when we update this variable, we know we don't need to do anything. That's good. But if there's one element reading its value, we need to know which element or how is it being read, so that we can precisely update only that element. And this is what we call fine-grained reactivity. So how do we implement this? Well, we're going to use the concept called signals, which separates like getting the reference and the actual reading of the value. There's many ways of implementing signals, and one of it is to have the signal implemented like an object with a get method, right? You can pass the reference, like the written object, we call it a signal. You can pass this object, reference of this object anywhere, right? You can pass it into a…

Do something function and you know you're just passing in a reference and you also know when the value is being read because it has to call the 𝚐𝚎𝚝 methods, right? So you know exactly when it's being called to read the value inside the signals and you can actually do something about it like making an alert.

So this is a very simplified version of signals that

we're going to create and we're going to pass it into we're going to make some changes to our generated code. So here in our generated code instead of passing counter, now we got the counter is a signal. So we're going to take in a function and we're going to return we're going to read the value only inside the text method over here. So only call the 𝚐𝚎𝚝 when we want to create the elements, when we want to set the text content for that text node. So this statement itself, we want this statement to run

whenever the counter signal is changed. The value of the counter signal has changed. And in general, basically any signal values that is being read inside the text function, whenever those signal values has changed, what we want is that we want to rerun this statement again to update the text value of our text node.

Introducing the Concept of Effects29:25

And to do that, we're going to introduce another concept called effect.

We're going to wrap it inside effect. So we haven't talked about what is effect, but let's just imagine that this is our goal. Our goal is that the callback function within effect, this function will be called again whenever any signals value that is being read within the effect callback function is updated. So I'm not sure whether it sounds confused, but let me try to illustrate how it works. So hopefully this makes it clearer. So this is the function. The effect function and we will first run its inner callback function which is this one which itself has one statement which is this. So when we call this statement which eventually calls the text function and eventually will try to get

the counter signal, the value of the counter signal. And basically this signal knows that its value is being read. It's like, "Oh, okay. I'm being read. So may I ask like which effect callback is calling causing my value to be read?" Oh, it's this one. So I'm going to keep a reference of this function inside myself so that I will know to call this again the next time when my value has changed. So after I keep the reference and also return the value of the signals which is zero, I'm going to set the text element content to be zero. So we'll run this line and then the text you see is zero. And then this is done. Until sometime later, maybe somewhere else, someone modified the signal value to one.

Now the signal will think like, "Oh, okay. My value has changed. I need to call the effect callback again." Which is this one. We're going to call this which will call the text function which will call the counter.𝚐𝚎𝚝 which will returns me the value of one and I'm going to update the text of the elements to be one. And so you get it. So the text element will update whenever the signal value has changed. Of course, this oversimplifies the complexity of signals of all the different frameworks out there. And there's also a few things that we didn't explain or handle well in this illustration. For example, we need to make sure that if we have multiple signals updating at the same time, we only want this function to be called only once or maybe there's multiple signals being read in or the next time when the effect has called and we have more or less signals, we need to update that as well. But those are not handled, but basically you get the idea. So now you know how it works. Let's quickly implement that.

Implementing Signals and Effects in Runtime32:07

I think we left with 5 minutes. So let's see where's my code. This is the runtime.

Okay, so we'll run this. So here is our states. So we're going to change this to like the 𝚐𝚎𝚝 function that we saw just now. And next thing is that we're going to create the effect function and we need to remember what's the callback. So we need to keep the reference. So we create a variable to store it and then we call the callback function. And then so over here, we go back to our states over here. We know that okay. So we check whether what's the current effect that's been running. I'm going to store that reference within the signal itself.

So with now the signal knows what are the effects that's been run whenever we call the effect. We will add another method called the 𝚜𝚎𝚝 which will basically update the value and then also call all the effects that is being stored. So that's basically the idea.

And then lastly, I think we need to update the text function to do effect like this. And I think that's it. That's the basic very basic implementation for our runtime.

And I think I quickly go over here to make some changes to see how it actually works. So first is that for the text function over here, now we are taking a function instead. So let me break it down. So this takes in a function and then the counter we need

to call 𝚐𝚎𝚝 instead of just returning the value. And over here, we need to call 𝚜𝚎𝚝.

With the counter get.

So basically these are some small changes that we will implement in a compiler later on. Let's save this and let's quickly go to our browser and then refresh. I think I make a typo just now. Okay, S. Refresh.

Updating the Compiler for Reactivity34:26

So now we figure out the runtime, which is the tracking, and then we also updated our tweak a bit of our compile code. And now if you click on this increment, right? It will just works, right? So this reactivity and all handles on the browser side. Yeah, so.

I think we will last one left with one last bit, which is we need to go to the compiler and then make all these changes required. I think I will really run like breeze through as fast as I can. Basically, we're going to analyze the scope of the script that we have to make sure that we modify the right variables to add the set or like get method, So basically, if you here the assignment expression is the one that is like A equals something, we're going to add the set method to it. And also in the identifier, we're going to add the gets method as well. So we're going to read the value out from the states.

And basically, we're going to add this transform.js for both expressions as well as attributes as

basically anywhere we are using referencing the JavaScript.

And I think that's it.

And so, basically now if we run our code. One, I think some typo just now.

Okay. So here, if we run our compiler again, this basically will generates the same code that you just saw I make some changes to, right? It's exactly the same, but we just make that into the compiler, right?

Universal Reactivity Demo: Moving State into Functions36:16

So one last demo I want to show is that in our component, right, over here, this was what not possible previously in Svelte 4 and below, which is to create a function, right? And move all this code inside this like function, right? And to do something like universal reactivity.

Right? And we're going to update this quickly.

So we move all our state reactivity into a function.

I'm going to use that to update to call decrement, increment, and getting the value, right? So this we're going to compile it, and we are going to run this, right? So the compile code, once again, this basically is moving all this reactivity inside a function, and it will still works as we expect, right?

Summary of Svelte Compiler Concepts37:30

So that's about the compiler that we create, right?

So a quick summary, right, what we have gone through in this talk. First one is that we learn about how to write a compiler, which basically have the parts, but we didn't go through the details of every step of implementation. And then we do some analysis, and then we generate into a JavaScript code, right? And then we also learn about signals and fine-grained reactivity. I think one thing to remind you is that if you take a look at the compile code, you realize that the app function we just execute once to create the entire elements, and that's it, right? And when we create elements, we actually go to set up the necessary subscriptions to register like the reactivity, right? And those subscriptions are made on the element level or the attribute level or like the text content level, right? So that's why it's fine-grained. Only those statements being rerun whenever the variables or the signals has changed.

And last but not the least, there's actually a lot of things that I did not manage to cover in this talk, like all this and above, like many rules and

other optimization techniques, right? So that's it for my talk. Thank you so much for listening through my talk, and thank you very much for having me here. Thank you.