Using The FASTEST Cache For Your ASP NET Core Application

You've probably never even heard of this cache solution that's actually faster than Reddus in terms of throughput and has lower latency. Hi, my name is Nick Cantino and I'm a principal software engineering manager at Microsoft. In this video, we're going to be looking at using Garnet alongside Dapper, Fusion Cache, and our ASP.NET Core solution that we've been looking at in the previous videos. In this video, I'm not going to do a super deep dive on Garnet. We are going to look at it a little bit in the intro. So, if you're just looking to get it hooked up, you can skip ahead further in this video. I'll try to include timestamps below so you can jump ahead if you just want to see how to hook it up in the code. But otherwise, we'll flip over to the Garnet web page and read a little bit about what it has to offer. If that sounds interesting, just a reminder, subscribe to the channel and check out that pin comment for my courses on domain. Let's go see what Garnet is. All right. So, Garnet is Microsoft research technology that we have that essentially has the similar surface area to Reddus. And what I mean by that is it actually implements the same protocol. So, you can see right here, Garnet uses Reddus's as its primary wire protocol. Thus, one can use Garnet with unmodified Reddus clients available in most programming languages. For example, the stack exchange Reddus in C. So compared to other open- source cache stores, you get much better performance, latency, extensibility, and durability features. Now, in just a moment, we will walk through some of the benchmarks. Like I said, I'm not going to try to go analyze them all with you. It's all available on Microsoft's page here. So, let's go over this list of features that they have here. And then you'll see a little bit later that we have some features that they're still trying to work on. So, Garnet offers the following key advantages. So they say orders of magnitude better server throughput in terms of operations per second with small batches and many client sessions. And they have the screenshots of those benchmarks as well. They say that's relative to comparable open-source cache stores. So you'll see Reddus and a couple of others. Extremely low single operation latency. So often less than 300 micros at the 99.9th percentile. So that's pretty awesome on commodity cloud. So Azure in this case machines with accelerated TCP enabled for Windows and Linux better scalability as we increase the number of clients with or without client side batching. So again like this is probably something that you end up having your clients doing so that they can batch on the client side and that way you're not totally hammering the cache like the server side of the cache. But apparently even without client side batching they can still get really awesome performance. They say the ability to use all the CPU memory resources of a server machine with a single shared memory server instance. No intra node cluster is needed. Support for larger than memory data set. So spilling to local and cloud storage devices, right? So as you need bigger and bigger caches, it does look like Garnet supports not only inmemory but the ability to go to local storage and cloud storage, which is awesome. But before we continue on, this is just a quick reminder that I do have courses available on domain. If you're just getting started in your programming journey and you want to learn C, you can head over to dome train. I have a getting started in C course, it's approximately 5 hours of content taking you from absolutely no experience to being able to program in C. And after that, I have my deep dive course which will take you to the next level with another 6 hours of content so that you can start building basic applications. Head over to do train and check it out. Let's head back to the video. Database features such as fast checkpointing and recovery, publish and subscribe. So some event- based systems as well support for multi-node sharded hash partitioning. So this is like reddis cluster mode state migration and replication. They talk about the test that they have and then it's actually in C. So it's you can go check out the code for it which is super cool. Cool. So like I mentioned they still have some features that they are working on. So they have API coverage and they say this is the uh the list of sort of API surface area but they do say you know reach out if you'd like to have more APIs added. So the list is growing over time, but we would love to hear from you what APIs you want the most. Right. So I think this is super interesting. I personally have not used it in any solution or project that I've been working on. I have heard about this. I've never put it into practice, but someone did request it. In terms of this playlist that I have, which by the way, if you've not watched the beginning videos in this playlist, you can check it out up here. I will remind you once again once you start looking through the code because I'm not going to explain it all from scratch. There's so many of the videos that came before this one and we're just going to build on top of it. Let's go check out the benchmarks because I think that this is a really interesting part. Like I said, we're not going to go through every single one of them, but I just want to be able to kind of highlight to you that they have benchmarks for all of these different scenarios. And it looks like Garnett basically in every single situation does a much better job. Which the first time I looked through these, which would have been many months ago now, I remember thinking, okay, like you know, you start off these benchmarks or looking at these pictures and you're like, wow, that's impressive. Okay, obviously they put the ones that are like really good at the beginning, but it's like every single benchmark, which uh I find hard to believe, but at the same time, they have the data for it, so I can't say that that's not real. But you can see this one this is throughput. So log sale varying number of client sessions for a database size for 1024 keys and 256 million keys. So you get two different variations, right? And you can see that even when you have smaller number of keys and then millions and millions of keys, you still have Garnet sort of going way higher in terms of throughput uh which is super cool. And this is the number of client sessions as well by the way on the bottom. So on the uh the x-axis. So it does seem to start to level out up at the top. But pretty interesting, right? Like if you look at Reddus for example, it's flat regardless of the client sessions. So you're not really getting additional throughput that way. But when you have more client sessions, it does look like Garnet is kind of going up. Like the same thing happens with Dragonfly. I've never used Dragonfly, by the way. And same with uh KDB. I've never used that either. So uh for me, I found this most interesting comparing it to Reddus because that is something that I have used. If we scroll a little bit lower, so this one was get as well and this one down here is get. This is for batch sizes, right? So obviously you expect that as you have larger batches that the throughput goes up, which is the case. It seems like it's just going up more significantly than you have with the other comparables. But you know, latency is a super interesting one. This is kind of hard to see in this picture. So if we open it up here, that's uh it's bright for me, but at least it's a little bit more readable. But you can see like Reddice when you have more client sessions the the latency really goes up. We're still talking on the order of like this is micros secondsonds on the Y-axis, right? But pretty wild that across the board when you're going through these you can see that Garnett is in red and basically only I guess in like the the 99 and 99.9th percentile do we start to creep over like that 400 microcond latency which I think is extremely impressive. But like I said, I'm just going to scroll through these without clicking on all of them. But you can see that if you want to go check these out, there is tons of data that they've collected. They explain how the benchmarks work, how to collect them. It's all available if you want to go browse through that. But you'll notice as I'm scrolling through, like it seems pretty clear that Garnet across all of these is just doing a significantly better job. So very interesting, but I think this is probably why someone requested that they want to see if we can make a video on this, right? So what I'm going to do is jump over to the releases page. Garnett is, like I said, it's available. This is on GitHub, right? So the GitHub.io page for them. What we were able to do in the previous videos is we went over to Docker Desktop and basically I could go search for Reddus when we were doing Reddus and you have it like right at the top. You know, it is extremely downloaded. It's a super popular one. Over a billion downloads, right? But I did this and I didn't see anything that seemed obvious to me. Like this is Stanford at the top. I think it's the like Stanford University, but I'm not even sure. I didn't see anything specifically from Microsoft. So I said that's a little weird. I don't know if I just want to go downloading these or trying to research this way. But I said they must have a Docker image. So when I did a quick search, I was able to find that we can get this command, right? So if we go run this, this will actually, let's go back to docker desktop. If you pop open the terminal, if you don't know where that button is in docker desktop, it's in the bottom right. There's a little terminal text. You can click it. It pops up this. Or if you're already familiar on the command line, then you don't need me to explain this to you. But if you go paste this in here, this will go run a Docker image for you of Garnett. Clearly, we were able to get a Docker image. So, I already did this and I just did it like I showed you right in the UI here. And that means if I go to images, you can see that I have this Garnet image right here. So, if I want to go run it, if I go over to containers, I don't have any running now, but if we want to go run it, I would press run. If you recall, especially from previous videos, or if you're familiar with Docker, you know to do this. If you're not familiar, this part here is where we need to map the port of the host to the port that we want to have on the inside of the container. Right? So, because it's just like Reddus, we're talking about Garnet, but because it's just like Reddus, it will be by default on 6379 for the port. For me, because I don't like doing additional mapping and having overhead, especially when I'm trying to make a video, I like just doing 6379 so that it matches. Now when I press run, basically on my own machine, if I try to connect to port 6379, that will actually get connected to port 6379 inside of this container. So let's go ahead and run it. And you can see that we have Garnet running. This looks pretty sad, but when I saw it running in the terminal before, it looked like a little red gem, which is pretty cool. But I think the asyard is kind of busting this view. But trust me on that one, it looked kind of nice. Now we have that running. What I'm going to do is switch over to Visual Studio. And at this point, this is where we're going to start to look at the code that we had before. Okay. So, in one of the previous videos, we had things set up to use Fusion Cache and we have Reddus enabled here. But what I'm going to do is run this without changing any code. Right? So, this is the really cool part. But if you recall in the documentation or if you jumped ahead to this part in the video, you might have missed it. But Garnet basically uses the same wire protocol as Reddus. We can just use fusion cache. We can use the Reddus client that we had before. All hooked up the exact same way. And now that Garnet is running, we can take advantage of that. I'm not touching any of the code, but just to do a quick recap because if you have watched the previous videos, friendly reminder, I linked it earlier, but you can check out this video up here for the very last one in the series before this one. What we have is we're setting up fusion cache on our dependency injection. We set it up with a serializer and then we have the distributed cache that we want. And you can see that specifically on line 13, we have a reddis cache. It does not say Garnet. There is nothing in this code that says Garnet anywhere. In fact, just to prove it to you, if I search Garnet in here, no results. Okay, literally nothing to do with Garnet. This is hooked up for Reddus. But we have Garnet running. I am pulling in the connection string for my app settings. This is just going to connect to the Reddus default port. And then I had this set of options here. It's not really relevant. And then we have it set up as a hybrid cache. This was sort of where we left off before using all the fancy options to have fusion cache running as hybrid cache. So we get inmemory, we get reddis or in this case we get garnet which is pretty cool. I'm going to go ahead and press play. If you've seen the previous videos, this part's a little bit boring, but I'm just going to show you and prove to you that we're actually caching using Garnet. Let me go ahead and run this. Okay, so you can see that I have this Fusion cache project running here, right? So, this is all good to go. We're going to visit this web page, but I just wanted to call out that I have a breakpoint inside of my Dapper repository. Let me make that a little bit easier to read. When we go to try and get a record hitting the web page, we should hit this break point. And that's because we just started the cache. You watched me start it up and we just started up the application. So, that means we have to go pull that data in. So, let me go ahead and do that. Okay. So, I already had this running from before, but if I go ahead and press enter, see that we hit our break point. This means that we're going to go query this data from the database. Okay, when I press F5, we get this result back. This is the exact same as it was on the screen. So, if I go to fetch it again, right, we're hitting the cache. It's not hitting the break point anymore. But like in the other videos, because we're using a hybrid cache, you might say, "Nick, that's all smoke and mirrors. It's just using the memory cache. How do you prove to us that it's actually using Garnett? How do we know this is even working at all?" Great question. I'm going to go ahead and stop this. Okay, what I'm not doing is I go over to Docker. I'm going to leave this running. Okay, so this is our Garnet instance still running. If I go run this application again, what we won't see is that we're not going to be hitting that's too many negatives. We're basically not going to see this breakpoint get hit. So, let me go ahead and run it. And the reason that we won't see this breakpoint get hit is because we don't have to go to the database. We are using Garnet using the Reddis protocol. We're using Garnet as our hybrid cache. So, if I check, we are running. This is a little bit misleading because it didn't actually clear. If you can check right here, you can see the carrot is flashing here. So, this is just leftover logs. There we go. Okay, so much better. Now, let me go ahead and fetch this again. So, when I press enter here, we're not going to hit the break point because the back plane that we have in fusion cache is going to get this from Garnett. Okay, there we go. So, it fetched the data. It's still fetching. You can see fusion cache with the very verbose logging is just getting it from the cache. I still have this breakpoint here. There's no smoke and mirrors. That's Garnett working as Reddus. And just a quick recap in the program.cs file, we changed literally nothing from the prior videos when we had Reddus hooked up with Fusion cache as a hybrid cache. All that we did was went over to Docker, we got Garnet and we ran it. That's that super simple. I was honestly expecting there to be some gotchas with this because when I saw that they support the Reddis Vire protocol, I went that's nice. But I also saw the list of features that there's still like that they're supporting and then they mentioned that they're planning to add more and I went okay what are the odds that this is just going to work out of the box? It just did which is super cool. Now there might be many more advanced features in Reddus that they don't yet have in Garnet. I am not familiar with all of those. So when it comes to a production system, this is something you're going to want to research. So I'm not necessarily sitting here suggesting that go swap all of your Reddus instances for Garnett. Please don't just blindly do something like that. I just thought that this would be a fun video to make because someone requested it and it truly just did work with Fusion Cache out of the box. So thanks so much for watching and I will see you next time. Take care.