Videos → Optimization design patterns - from games to web

Introduction and Speaker's Journey into Optimization0:00

So now it's time for next session.

Optimization Design Patterns: From Games to Web0:39

Optimization Design Patterns from Games to Web. Please welcome Mr. Yonatan Kra, Staff Engineer from WalkMe.

Good morning, Bangkok. How are you today? Anyone?

It's time to wake up. I know it's morning. But let's wake up a little bit. Okay, thank you for having me here. I've had a pleasant time so far at Bangkok. You have a very beautiful city. A lot of what to see and enjoy.

The Importance of Performance Optimization1:29

I'd like to start with a personal story of mine.

About six years ago, I started doing JavaScript professionally. Until then, I was doing MATLAB and PHP and stuff like that. And I worked for an enterprise company, and I created a real-time big data application. And I created it, it was very beautiful. AngularJS, Kendo UI, and a lot of sprinkles and design.

And then I had to connect it to the actual server. And then I got the out-snap screen that some of you might know. And I had two choices. I could quit and find a new job, or I could fight this browser thing that keeps on crashing and found out what's happening. So I'm standing here today because I kept on fighting.

Gaming's Influence on Software Performance2:25

And this talk is part of the journey that I experienced. I found out ways to optimize my application for it to withstand a big load of data that's coming into both the Node.js server and the actual client in the browser. So I hope you enjoy it, and I sprinkled some gaming. Anyone actually likes to play computer games here? Raise your hands so I'll know. Yay! Okay, so we can talk about it later, maybe do some game party or something.

About the Speaker: Yonatan Kra3:04

Okay, so a little bit about me. I'm Yonatan. I'm a Staff Engineer at WalkMe. I used to be CTO at webiks. I used to be a neuroscientist. I can talk about it later. I'm a speaker, blogger, I'm an Egghead instructor. And I'm also a runner. This is me winning a half marathon, which is kind of one of my best achievements in life. And this is my son, which is one of the three best achievements in life. I have three kids. He's running with me and continues the legacy.

Presentation Overview and the Triangle of Considerations3:42

So what are we going to talk about today? I'll speak a bit about motivation of when and when not to use design patterns for optimization. We'll look at design patterns and how they affect our actual machine. Because JavaScript developers usually develop, and they don't tend to think about how their code is affecting the actual machine, memory, CPU, and stuff like that. We develop for the browser. And eventually, we'll summarize because I needed a third part.

Design Patterns: Everyday Use and Gaming's Role4:19

Okay, so design patterns. It's kind of a big word or big term.

And but you use it all the time. I mean, if you're doing a for loop, you're using the iterator pattern. And you're using them all the time because design patterns are things that were kind of made up or made official because they solve a common problem. Which means it's a problem you stumble upon every day at your day job.

And why gaming? We'll see a short video now. Let's see if it's working. I hope it is. Try to focus on what the user says, if you can hear it. You can't hear sound?

Okay.

It's not that important.

Okay, we don't need the sound. It's okay, I can skip it.

Why Gaming Leads in Performance Optimization5:38

So what you were supposed to see from this video is how the player reacts to the game. The player, if you could hear it, is swearing and giving comments as if he's inside the game. And this is part of the reason why the gaming industry leads the software world when it comes to performance. Because they need to make sure that the user gets the best experience it can have. If the user does not get the best experience, let's say you have junk, like the animation is jumpy, so he won't swear at the creatures of the game, he would swear at the developer of the game. And you don't want someone swearing at you. So this is a little bit about motivation, about design patterns, why you shouldn't be afraid of it because you're actually using it all the time. And gaming, why learn from gaming?

I'll have references to the gaming industry as we go along.

Balancing Architecture, Performance, and Deadlines6:42

A few points to consider, there's a kind of triangle of points to consider

when you want to use some design pattern or any solution. It's true for any solution. You should consider architecture. And architecture stands for reusability.

My former talk was about web components, reusable components, or clean code and stuff like that that architects preach about. And sometimes performance collides with that because sometimes hard coding something

is the most performant thing to do. So at performant sensitive places, you'll ditch the architecture aside and do something that looks kind of nasty, but it will be good for performance. And of course, you have deadlines. Most of us or all of us have deadlines because we work at a company that needs to make money and we need to deliver eventually. So you have to consider these or put them in the back of your head when you're considering a solution.

Memory Management and the Object Pool Pattern7:52

Okay, so let's jump into the actual design patterns.

The first stop will be memory and I'll explain a bit about the problems we have. So this is a picture from a game called Star Control. And you can see all the explosions around this ship and this ship firing the laser bursts or whatever they are.

And almost every pixel can be a class

that is instantiated. And if it would have been done naively, this could have been a problem and we'll soon see why. In order to understand the problem, I'll remind you some computer science terms. Allocation is the first one. So we have some data in memory and we want to create an array with five members. So it finds a place in memory and then we want to push to the array another element. So it needs to allocate a place in memory for this element. So this is allocation. And fragmentation happens when we delete something. So we have a cool thing and we instantiate it, a lousy thing that we instantiate, another cool thing, and then we delete the lousy thing.

And we have a hole in memory. And this is a fragmented memory.

Why Memory Fragmentation and Garbage Collection are Bad9:16

If anyone remembers the defrag software from Windows 7 or XP,

so this is it just in RAM. And why is it bad? Because allocation takes time. And fragmentation makes it worse because now it needs to find a place in memory and sometimes the fragment doesn't really fit to what we need and we need to sometimes even defrag. And it makes the allocation worse. And what's very relevant to us is garbage collection. When we delete something in JavaScript, an object or whatever, it is being garbage collected. And garbage collection is the process that runs periodically and cleans up our memory from unneeded objects.

And it also takes time.

Code Example: Object Pool Performance Comparison10:11

So let's see some code. Hope it'll work. Let's see.

So what's the problem with garbage collection? So we'll go to the performance tab. And let's see what the code does.

So our code has this demo class.

And what it does, it instantiates the class a thousand times, four thousand times, and it does that ten times. So it's ten million instantiations and deletions.

Let's see what happens when I do these all instantiations and deletions. So I start recording. I do create and destroy. And we see that it took around two seconds to run, pretty fast. But I see below here, can you see it on screen? Yeah. All these GC, garbage collection instances. And we see that the whole thing took us two seconds. It might be too much. But can we solve it?

Let's see.

So the solution would be to use something that's called

a design pattern called an object pool. An object pool is, in essence, to create an array of the objects that you intend to instantiate. And instead of instantiating them dynamically during the for loop, you do it before the for loop, or even when your application loads. And then you don't need to do all these memory allocation and garbage collection stuff.

So let's look at the code again.

This create and destroy was the naive way. And below, we have the create with a pool. And you can see that before I start the whole operation, I actually instantiate a pool with a thousand demos. And what I do here is I ask for a free demo.

I do what I want with it. And then I release everything. I tell it just release the object that I don't need it anymore.

So let's see what's the difference would be now when I'm using this method. So I'll just delete that, let's garbage collect everything, and start recording. So I create with a pool, and you can see that it took like four times faster.

Might be even more.

You can see that with the pool, it took like half a second. And without the pool, it took us in the last time two seconds, and now three seconds. And it's clearly faster with the pool. But let's look at the difference in the performance tool. Without the pool, we still have the minor GC. And with the pool, we actually made the garbage collection disappear. And this is a very important thing to notice that we can actually improve the performance in our application and save on CPU time and memory

and make animations even better for our users.

When and Why to Use the Object Pool Pattern14:02

So to summarize, when to use this? When you have a lot of objects of the same type or class that you instantiate and delete a lot. For instance, if you have a socket connection that brings in a lot of updates to your application, some of them are create commands, some of them are deletes, some of them are updates to existing objects.

But this can save you a lot. And you should actually see even reduction in cloud costs because as I showed you, you actually save on CPU time. And let's say you're using Lambda, you actually pay for the time you're using the CPU or the machine. And this can actually save you a lot of time, especially when you scale.

So we saw the pros in this. The cons are that as you saw in the code that we're now not using arrays. So a new developer needs to learn the API for the pool you created, for instance. So in terms of architecture, you need to actually build some mechanism and then teach your developers how to use it. And this can be cumbersome. So always monitor and see if you really need this solution. Now it helps you.

And you can read more in this link. It's a link to my blog. I've written a few articles about it with some more use cases that you can see and see if it fits to your needs.

The System Bus and Cache Misses15:35

The second station in our machine would be the bus.

What is the bus? Before that, let's see a problem and why we need this next solution. So according to Moore's law, our machines getting much faster every year,

every month even, some would say. But the CPU gets a lot faster than our memory. And that means that if the CPU relies on things in memory, it will slow us down.

So this is the system. So we have the CPU, we have the memory, and we have the bus that brings data from the memory to the CPU.

And we said that the memory is slow. So what happens in modern machines, we have the cache memory inside the CPU. And what the cache memory is actually is that the CPU requires memory from, tells the bus to bring something from the memory. The bus goes to the memory and brings the memory the CPU required plus a little bit from the sides. Because the memory is kind of a big array. So it goes to the address that the CPU required and takes a little bit from the side. You can think about it as an array. And that's all good and well.

But sometimes the cache misses. And that's called the cache miss. And it happens in this case. So this is the memory that we need in the CPU, and it starts to process the data. Then it finds out that it needs this part of the memory as well. So it looks at the cache, but the cache limit is here. So the cache missed the data that we need. And now the CPU needs to wait until the bus goes to the slow memory and brings the missing data, and then it continues. So we actually, the CPU set idle for a few even microseconds or I don't know.

But it has a big effect, especially when you're developing a game that needs to run really fast or an application when you scale up and you really need things to move very fast.

Cache Misses in JavaScript: A jsPerf Example17:57

But you're a JavaScript developer. What, why should you care? You're running the browser or in Node.js or in Electron or whatever. But let's see a website that might tell you something. Anyone knows this website? Raise your hand if you know this website. Yeah. jsPerf. It's a website to measure performance, to compare scripts and their performance. So if you look at the bottom one, this is actually running over an array

in the simple way.

And this one is running on the same array, but every thousandth index.

So you run at 0, 1000, 2000, and so on. And then 1001, 2001, and so on.

And you can see that this one is slower by 90%.

And how does this happen? Because the CPU runs on your array, and then it runs on, it needs to get to the thousands index after being in the zero, but it's not in its cache. So it need to ask the bus to bring it the thousand index, and it's waiting. Then it needs the 2000s, and it's not in its cache now. So it needs to send the bus almost every iteration to bring the data and just sits idle. And this is how cache miss can affect your JavaScript code.

So go local with your code,

Code Example: Data Locality and Performance19:31

and we'll see an example for this.

So let's look at the code.

So this is the exact same code I showed before. Let's see it in time. And there is another use case here that I'd like you to see. So let's run the locality. So when we're running it with local, we see it takes around 600 milliseconds.

And then when we run it not local, it takes 3½ seconds.

And this is the sorted example. It shows you how important it is to access data that is closer in memory in order to improve performance in this case. The non-local sorted is actually the same array.

Only I sorted it. So let's say that the 0, the thousand, and the one and all these objects were closer to each other in memory.

So this is how you can improve the speed. So if you make sure to sort your arrays all the time and make sure the objects that are closer in memory, that objects that you need to process together are closer in memory, that would actually save you a lot of time and CPU cycles. So your code would run faster.

When and Why to Use Data Locality Optimization21:14

Okay, let's summarize when to use it. You'll use it when you have a lot of data. Actually, cache I think is around 64 kilobytes. So you don't really need a lot of data. You just need to run a lot of times over the data. For instance, if you have a change detector, like in Angular.

And you want to save on CPU time, and you want to make your application much faster.

The pros, I just showed you. You'll have a better performance, but the cons is that you'll need to create a mechanism to make sure your objects are closer in memory.

And this can be costly because sorting is also costly. So you have to monitor and see if it works for you. You can enter the live example I just showed you and test for yourself. And you can read more again. I've written about data locality and some more use cases

for this if you'd like to see.

The Flyweight Pattern: Reducing Memory and Bandwidth22:25

Okay, the last one we'll talk about today is in gaming. It's very much related to GPU.

In JavaScript, we also use it in GPU when you use WebGL. But for most use cases, it will be relevant for bandwidth and memory, and we'll soon see why. So the problem here is in gaming is that we have a repeating scenery with a lot of similar properties. For instance, if you look at the fans in this game, they'll kind of look similar with, I don't know, just the shirt is a bit different, the position is a bit different, but the mesh and the texture are kind of the same. And you can see here the trees in the background. They're all kind of the same with, you know, a little bit different branching, maybe a little bit tint in the color and stuff like that.

And let's focus on the fans again. So if I take a fan and I call it a person, I can naively save it like this with all the properties

inside. And then I have something like this. So I have mesh, shirt, head, scarf, and some params that are the actual unique parameters for the person. But maybe a better way to do it would be to take the person model outside, which is the common stuff. It's called the intrinsic properties and just inject it

to the person and keep the extrinsic properties, which are the unique, inside the person. And then I have something like this. And this I get by reference and I save on memory.

Flyweight Pattern in Production: A WalkMe Use Case24:12

So this is called the flyweight design pattern. And we use it in several ways in production, actually, at WalkMe and even in other places. For instance, WalkMe allows users to get search results from various search providers. And search providers get us a lot of results, and a lot of the objects that we get have similar properties. For instance, Google sends us stuff and all of the stuff have common properties.

So what we do is we create a Google model, Bing model, and Yahoo model for instance, and all the search results refer to the common properties. And then we save memory and bandwidth.

Code Example: Flyweight Pattern's Impact on Network and Memory25:01

So let's see some code again. This will be the last example.

So here I have the request data and request the flyweight data. So let's look at the network first. Let's do it anew. So I request the data and I request the lightweight data, and I see that I save around 70 kilobytes.

It might sound not too much, but this example is very simple, and probably in your application, you have more complex data. In addition, imagine that you have customers or users with slow internet connection. And these 70 kilobytes can be the difference between page loads in two seconds or page loads in a second.

Another thing is, let's go to the memory. I'll just request the data. And then I'll request the flyweight data. And then I'll compare between them. And I see that the flyweight took 160 kilobytes less

than the regular data.

So I save both on bandwidth and memory. And again, this is a very simple example.

When and Why to Use the Flyweight Pattern26:33

Okay, so when to use it? When we have a lot of objects with common properties, and we can save memory and bandwidth. And I didn't mention how you use it with WebGL, but it can save you a lot with WebGL. You can look at libraries like CesiumJS, for instance. Pros and cons. The pros I just showed you. The cons is that you again need to build a mechanism to match the common properties to the unique.

And you can read more about it again in my blog. I actually explain how to build one such mechanism if you'd like to see, and some use cases we use it.

Summary and Resources27:14

So to summarize, we went over the motivation, why to use, or when and when not to use design patterns. I didn't stress it enough that you need to monitor. Always monitor to see if the solution is good for you and if it's working and if it's worth the effort. I highly recommend this book, "Game Programming Patterns". It's actually a free book. You can read it online. It's one of the best I've seen. And you can also read at my blog. I write a lot about performance and how you can improve it. Thank you. Thank you so much.

Thank you very much, Mr. Kra. So, ladies and gentlemen, if you like to talk to Mr. Kra personally, he will be, after this session, he will be at the Q&A room. The Q&A room is, once you get out of the theater, on your right-hand side is the Q&A room.