How To Cache In C#: Caching For Beginners

caching is a very important but also complicated part of software development and I wanted to put together a video for beginners to get introduced to some caching Concepts in C and.net hi my name is Nick centino and I'm a principal software engineering manager at Microsoft this video is going to be a little bit different compared to some of my other ones given that we're going to be looking at caching Concepts but we're going to cover a lot of ground we're not going to go super deep super technical but just look at a high level just so we can cover some of the basic concepts if you're a c developer and you haven't really been exposed to caching it's still sort of A New Concept for you then I'm hopeful that this video will give you some of that understanding if you've had more experience with caching and you've worked with distributed caches this is probably going to be a little bit too basic for you but we're going to be exploring some different libraries and some of them will support distributed caching so if that sounds interesting just a reminder to subscribe to the channel and check out that pin comment for my courses on dome train now let's jump over to visual studio and get started with a very simple example on my screen I have the sample weather forecast application that we get when we're creating a new project in Visual Studio but I've modified it a little bit just to reorganize some of the content and from here we're going to be talking about a very simple starting point now why do we want cashing in the first place this is probably something that's worth answering because again if you're very new to the concept of caching you might be wondering why do we even care well there's many benefits to caching and like I said in the intro to this there's also a lot of complexity that comes along with caching as well some of the benefits that we get with caching include reduce latencies as well as reduce load on other services when we're talking about big complex distributed systems having low latency can be incredibly important as well as reducing load on other services these two reasons together can be a huge motivation to introduce caching but given that caching has so much complexity that comes along with it it's not necessarily something that you want to rush into especially if you're a beginner and that's why I wanted to take this starting point to go look at this very simple application put in a bit of a delay and then see how cashing can start to transform some of the performance characteristics now for the weather forecast application what I've done is I've introduced a day of the week it's not really going to matter we're not actually using that to do something based on Monday Tuesday Wednesday whatever but I just wanted to change the API so that we can talk about caching a little bit more as we go through these examples but one thing that I am doing is I'm introducing this delay here and it's going to be 1 second and that's going to be so that we can simulate if we had to go to a database or another service to get the information for the weather forecast this is going to simulate on the request path that we're going to have some additional delay this right here of course is just the delay but that's where I want you to think about that we might be having to interact with something else again that could be a database that could be some other micros service or whatever it has to be that's not necessarily just in process like we're dealing with here we have to be a little bit creative but that's going to help us illustrate some of these characteristics so other than that there's not really a whole lot to look at right now but I want to run this and just kind of show you what we're going to be exploring step by step as we start to tune things with caching so let me press play okay so in order for us to hit this API you can see that I have slwe forecast and then we have to pass in the parameter for the day of the week like I said though this day of the week isn't really going to be important it's just going to be used in the examples as we go forward with caching now because I have that 1 second sleep if I press enter here we can see it's going to have a bit of a delay before we get new numbers I'll do it again and I'll do it again and you can see that as a user if you were trying to use even something like this to load a web page that's going to be kind of crappy to have to wait a full second and if you think about more complicated services that have to do requests like this of course having a delay of a full second is not really that great and this gets even worse when we're layering in more complexity but something that I wanted to show you is what happens when we start to do things in bulk so let me go ahead and go back to visual studio I have this bit of code here that I'm going to uncomment it's going to be a little bit nasty it really wants me to have an await out the front here but I'm not going to actually leverage that when we go to run this just to make the warnings go away as I talk through this is just a bit of a hack to kind of simulate a whole bunch of traffic so I'm going to wait for 5 seconds and that means the app should be up and running then what I'm going to do is use the HTTP client Factory and this is going to allow us to make a whole bunch of HTTP clients and all that I'm going to do is in a loop for a thousand different tasks I'm going to start hitting our service and then we're going to wait for all of them to finish the result of this is we're going to see in the console that there's a whole bunch of web requests and we'll start to see the responses coming back and we're going to see the delay that it took to actually get that request to finish something that we'll notice and I'll show you in just a moment here but something that we'll notice is that we're going to see all of these tasks piling up so that means that the latency in terms of finishing that request is going to keep growing and that's just because there's a whole bunch of requests to serve if we think about it each one of these is supposed to delay really only 1 second and then the work that's actually done on the rest of the request path is pretty lightweight so it wouldn't be so unusual if we saw all of these tasks that we're going to go fire off take roughly 1 second but that's not what we're going to see it's going to be significantly worse so I'm going to go ahead and press play we're going to watch in the console instead of the browser and we should see a bunch of these fire off then I'm going to sort of pause so we can look in the console and start to see some of the tasks coming back I'm going to switch over here to our application and we should start seeing these all Pile in and in just a moment when I'm going to start to do is start seeing if we can get some results coming back so if I start to highlight here just to pause it a little bit you can see end processing HTTP request after 7 and 1 12 seconds right this one's about 4 seconds this is 6 and 1/2 this is 7 and 1/2 seconds over 8 seconds here so that sleep that I was talking about that Tas delay of only 1 second that's absolutely not what we're seeing here we're seeing so many requests come in that it's actually introducing a lot of latency for our system and really this is a very simple API request right so it's not like it's actually going out and doing anything complex we're not actually relying on any services or anything external but just to sort of simulate this very basic Behavior you can see that just having a lot of requests like this is going to be pretty detrimental to this service right the latency is really just through the roof so we can do better than this what I'd like to do now is start talking about caching and I should mention that as I go through all of these examples this is going to be a longer video I mentioned that in the beginning I'm going to put a playlist together and this playlist is going to contain more succinct videos on some of these topics so I'm going to go very high level through some of these just for a beginner view at a lot of them but I will have other videos that you can check out in that playlist where we can do a little bit more of a deep dive they'll be a lot more pointed in terms of what we're trying to tackle but this one is just to start comparing and contrasting and we'll get a little bit more complicated looking at different packages and different ways to cash as we go through through this entire video to start things off the very basic type of cash that we can look at is just a dictionary now if you have a little bit more knowledge about cashing you might say Nick well I don't think a dictionary is going to be a good idea here and you're right but I think it's going to help us get started and looking at some performance gains truly a dictionary is just a really simple way that we can get a little bit of cash in going on so what I'd like to do is introduce this cache and then we can go back to the browser and see some of the behavior that we get okay so on line 11 I've introduced a dictionary here and it's just going to be keyed by the day of the week and it's going to have the collection of weather forecast okay so I realize that this whole setup here is kind of awkward right it's not actually this forecast down here is actually doing five days at a time not really the point the point is that we're going to have a cash that's by days of the week just bear with me for this example and what we're going to do is see now that we have this caching logic here how that's going to improve some of our performance so to explain what's going on here instead of just creating that weather forecast what we're going to do first is check if our cash for the day of the week has a forecast for it and that cash like I said is just going to be this dictionary up here if you're not familiar with what a dictionary data structure is I do also have some beginner YouTube videos available you can check out that look at different types of collections in C and I Do cover a dictionary so you can check that out if you want to see more details I won't really dive into that in this video we're going to check if the cash has a key for the day of the week and if it does this triy get value will give us that value for that key but if it doesn't which is why we have the exclamation out the front we're going to say okay go create that weather forecast for us then we're going to cash it that means ideally the next time that we go run this code we should be able to go get the value back out of the cash and that means the second time through we shouldn't have to go into this if statement at all all and that's because it's going to be inside the cash and we can return it directly let's go run this and see what the performance characteristics are like in the browser okay so if you recall the first time I press enter to go hit our API we should see a delay of about 1 second and there we go so new results but if I press enter now it might be difficult to tell but it's finishing basically instantaneously right um to me as a user hitting this page I'm not really seeing any delay at all it's just going through so this is an example of cashing right that's the huge benefit that we get because now our latency is reduced dramatically and if this was a service call that we had to go make to some other resource we're not spending more Cycles trying to go do some other execution and you know cause other resources to have to spin up and do other work there is a trade-off though and that's now that we're using some memory to be able to shortcut that right so in this case it's a pretty good tradeoff and that's because to store this in memory for us is very lightweight if we think about the size like the footprint size of memory in terms of what this cash is going to hold if we're just talking about the days of the week and then some of these forecast entries this is a pretty small footprint right we're talking about seven days of the week and I know like I said this is kind of confusing because the way this weather forecast app to start with was probably supposed to do 5 days at a time but if you just imagine that this is the forecast within the day for for an example here then we could imagine that we might see something like I don't know seven days of the week times like uh maybe there's four or five different forecasts within the day so you might see something like 28 to 35 entries it's a pretty small price to pay in memory to be able to have this return very very fast but if we were caching other data something that might have tons and tons of records for example at work when I'm deal with cash is we have things that have to cash things like IP addresses but like how many IP addresses are we talking about it's not like 10 or 100 it could be Millions it could be millions and millions of different records that we have to cash and that means that all of a sudden the footprint isn't like a couple of Records in memory we actually really have to start thinking about resources and that's going to be an interesting tradeoff when we're building different systems this is something to think about when you're working with with caches it's going to be a tradeoff and this is the case for basically all things in software engineering now when we're using a dictionary like this there is a few different problems and I'm not going to be able to highlight all of them but I want to show you one thing that can happen when we start introducing scale to the number of requests so I'm going to go back and put in this little bit of code that spams our server with a bunch of requests and if we get lucky or unlucky we'll see what kind a problem shows up when I go run this okay so we're going to go run our server and we'll start hitting that API and we'll see if we can trigger this bad behavior and there it is so we have this problem and it says an object reference is not set to an instance of an object and it's going to be a very misleading exception when you're dealing with this kind of thing the reason I'm saying it's misleading is even when I was getting this video set up with these different examples this isn't even the exception I was seeing I was getting a key or index out of range and the reason that we're seeing this exception or the ones that I was seeing before I started recording is because this is actually what's called a raise condition a dictionary in C is not thread safe and what's happening is we're now running this web server and we have many different threads that are trying to hit our web server and they're saying give me the data give me the data and our cach is sitting there going hey no problem I'm like let me get that for you but what's actually happening here is that like I said there's a raise condition so as we're reading the cache we're also updating the cache as well and that means that we're going to have this little bit of an issue that is what we're seeing here so if I were to go look at the details of this exception um the stack Trace is going to be inside right so it's trying to insert like why why is this a problem right and it's just a problem because like I said it's racing we have multiple threads trying to do something and this collection is not designed to be sort of used across different threads like this so there's a solution and I'm going to stop this for a moment and we're going to see that we can basically fix this particular problem pretty easily there's something called a concurrent dictionary and you'll see that if I add concurrent here and then I use system collections concurrent notice how none of the other code broke right like the cache the apis that we call here to get the value to add it back in they still work so I'm going to go ahead and run this again and it's not going to guarantee the fix when we go run this cuz it is a race condition we were seeing before maybe we don't hit the same raise condition but hopefully you believe me that when we go to use this it should solve this particular problem so I'll press play again we will pull this up and we'll start watching these requests come through so it's really nice as we start firing all these off and this is still running you can see in the console in the bottom it might be hard to see that the scroll bar is actually moving but this is working one of the tricky things and I realized my example might be a little bit misleading from earlier is that when I scroll back up I did see some of these requests that were taking a little bit uh longer than they probably should right I think this is probably just piling up because I'm making the requests from the same process so I apologize that this example might have seemed a little bit misleading but if I scroll down you can see down here that we have these sort of times coming back right so you know 50 milliseconds uh 50 milliseconds 78 like that's why when I was in the browser I could press enter very fast so if we go to the bottom here you can see that it's all done processing it's gone and made a thousand different requests and theyve finished but something else that explains if I scroll back up we're going to see this actually in a little bit more detail when we get to some of the more advanced caching libraries is that these times this is actually something that's happening because when we have multiple threads trying to check the cache if you think about each thread trying to enter that execution you have let's just pretend we have two coming in at the same time when they're at the exact same time what's happening is one's checking the cash and saying hey do you have that no okay like let me go generate it but at the exact same time another one's doing that and because we have a whole bunch of them in parallel going to do that they're basically doing what's called like a stampede so they're all trying to go into the cash sort of logic and nothing stopping them all from entering so what would happen is if there was nothing in the cache if this was a downstream service and we had just to do a quick check right there's one here that's taking a while to 3 four five there's way more than 10 way more than 20 if I keep scrolling you can see there's more and more of these these would technically all be Downstream calls to some other service so instead of our cash protecting those things what's actually happening is that our cash is going I don't have the entry hey all of you you can go check Downstream we'll go fetch the data do whatever processing and then we'll put it into the cache so we can do a little bit better and some of the things that we'll see coming up one of the challenges that we might have with a dictionary or a concurrent dictionary maybe not really for the weather one that we're looking at but in terms of how much we're storing in a cash we may want to limit that and the other thing is that we may want to have more control over how long those things live in the cash when we're using our dictionary data structures we don't really have control over either of those things unless we start to build some of that logic around the cach itself so out of the box neither of those two collections really do that for us but we could do things like trying to store like the timestamp information we could try to check the count of the dictionary but there's other things that we can use that already have this logic available to us so one of the libraries that I've really enjoyed using in different projects that I built with is called bit faster so bit faster has this caching library and just to show you when we go over to the project let me click over here you can see that I'm using bit faster caching 2.5.2 it's a bit of a spoiler alert for some of the other things that we're going to be looking at but you can see at the top here that I'm using bit faster caching lru and that's because the cache that I want to look at here is this fast concurrent tlru so if you hover over it bit faster has a really good set of documentation for how their different collections work for me this is really helpful to get an understanding so when I start to pull information into memory I'm storing it in this collection like what's actually happening here like how do I control the stuff that I'm putting in right so it has a bit of an explanation about the algorithm right and then it explains how it's based on some of these other comparable things so if you want to go read more about these understand how the actual caching algorithm works you can go ahead and do that with respect to some of the internals but at a very high level the things that we're going to be looking at are this capacity so just to kind of write it out there's a capacity and then this other part is how long we're going to keep things in the cache so we can actually control both of these things that means that if we tried to keep caching more and more items if we went over seven and in these examples the day of the week I'm always setting it at one so there should only ever be one thing in the cache but again just to kind of walk through if we were to go over seven items what would happen is that it would evict the one that it needs to based on the algorithm we're using so it has a capacity limit there will never be more than seven and then there is this duration as well and that means that the entries in the cash will expire so I have that set to be three seconds what we can do now is actually go to the browser and see that we've have more control over the behavior of this API it's not just going to be every time I press enter in the browser that we're going to have this immediate response in fact we'll see that the data refreshes periodically and that period is 3 seconds so let's go ahead and run it okay so when I go to press enter here we should see this one take around 1 second there we go new set of data but if I press enter again enter again enter again see on that last one it took a little bit longer and it gave us new data so doing it again and again and again and then now another set of data so the way that this cache is working for us is that it's actually expiring the time after 3 seconds what I'm not doing is exercising sort of this code path so if I do three I should have used Postman or something else to go simulate this but you can see that as I was doing these other numbers I don't know if I can do this fast enough but if I tried to go back to one one was still cached right it's a little bit too hard to show you in a video but the point here is that I'm able to sort of cash these other entries if I were to somehow go over seven in that period of time we could actually see that will evict the ones that were at the beginning that's how bit faster caching could be used if you wanted a little bit more control but like I said I'm just going to keep these at a very high level and I'll have other videos where we can dive into some of the more specific details for each of these different libraries that we look at before we continue on this is just a quick reminder that I do have courses available on D train if you're just getting started in your programming journey and you want to learn C you can head over to dome train I have a getting started in C course it's approximately 5 hours of content taking you from absolutely no experience to being able to program in C and after that I have my deep dive course which will take you to the next level with another 6 hours of content so that you can start building basic applications head over to D PR and check it out let's head back to the video we've talked about being able to be more thread safe so that's where concurrent dictionary came in the fast concurrent tlru is also thread safe given that it's concurrent it's great but the fast concurrent tlru also gives us a a capacity check and it also gives us the ability to expire so we're sort of building up on more capabilities that we could use with our cash but so far everything that we've been looking at are just these helpful different collections that we can use not really specific to asp.net core not really specific to anything to do with web apis we're just able to leverage these sort of anywhere that we want and personally I've used these uh certainly outside of web apis just different systems that I built but something that we can start to look at now going forward are a little bit more specific for working with asp.net core and the next one that we're going to look at is actually builtin the built-in one that we're going to be looking at is the very simple memory cache you can see online six here I'm using services. add memory cache if you're not familiar with what this is doing like what's going on here this is just using dependency injection with this extension method to add a memory cache and if you're not familiar with dependency injection I have plenty of videos on my YouTube channel it's one of the things that I love to talk about because dependency injection super powerful especially if you're building plug-in systems which I love to do but we add that into the services and that way our application has this service available to us called a memory cache that we can leverage in our minimal API so if you look on line 11 here I have IM memory cache and this works because we've added it on line six that means that the runtime will allow us to resolve that dependency for this API here if we look a little bit lower we can see that the caching structure looks a little bit different now it's not just doing a check to get the value and setting it and actually has this get or create a sync so this this type of pattern we see here is going to be more and more common as we go forward and this is supposed to be one of the structures that helps us with what's called Stampede protection right so the fact that we can ask for an item and if it's not there it will use this call back so let me walk through some of the syntax here cuz it might look a little bit weird but the idea is sort of like hey if you don't have it then this is the thing that you're going to use to go create it so it is very much like what we saw before in the more simple examples it's just kind of built into how you Leverage The API itself so the first part is going to be the key that we're working with here right so just the day of the week that's going to consistently be the key that we have but if that entry is not found what we're going to do is call this what's called a factory method to say go create the entry that we want to put into the cash so this will get called only when the item is not in the cache based on this key the next part that we have here though is is going to be sort of like what we had with bit faster caching when we can actually expire the items that are going in this is pretty cool that we have access to this built right in we don't need any fancy libraries or anything and it's just IM memory cache to go use it so if I go ahead and run this so let's stop it we'll go run it again we will head over to the browser and I'm just going to press enter we should see that the first time like every time it's going to take one second to get a result back there we go and now every time after until it expires like it just did right keep doing it and this one will expire right so this is basically like we've seen so far right I just want to be able to show you like I said having these different comparisons so you can have different options and understand the different tools that you have to work with so let's go ahead and stop that and talk about sort of uh conceptually something that's a little bit more advanced than this the memory cache and everything that been looking at so far is in memory right like this one is literally called IM memory cache there are more advanced things built into net that we can leverage and what we're not going to be looking at in more detail is the distributed caching part we're going to start looking at some libraries and including another one that's built in that is for distributed caching but I'm not going to show you how to go set up redus in this video that's going to be way more complicated than I want to get across here we can do deep Dives on that in other videos but I want to show you and sort of talk about this concept so that as we go forward you'll understand why caching starts to get a lot more complicated at a high level like we've been saying in this video so far everything that we've been doing for caching is in memory so we don't have to coordinate our cach with anyone else we have this service that's running right here on my desktop when we're calling it all that's happening is we have this process with items in memory and the some of the advanced features we've seen is that we can expire those items or limit how many items we keep in the cache and it's thread safe so this is super helpful when you have one service like this but what about when you're dealing with multiple Services what happens not only when you have multiple services so say we wanted to have five instances of our weather app does every single one of them need to have its own cash is there a benefit or is there something else we could do to try and have something like let's say a shared cach what's interesting when we start talking about some of these more advanced caching Concepts is that we still have to look at the tradeoffs that are happening you'll hear in different types of systems you might have different layers of caching so you might have like a what you refer to as an l0 or L1 cache you can kind of index however you want here but you can start with an inmemory cache that's local sort of like in process kind of like our dictionary right or the the memory cach that we have here these should be extremely fast the reality is that the cost of the cash is greater because you have it rate in process it consumes memory if the process goes down you lose the cash so there's more cost to it but it is the fastest type of cash that we have to work with then you have these other layers of cash and what's interesting about these is that these can start to be sort of out of process or they could even be their own services that are running the idea with these is that they're way faster to use than still going to another Downstream service like say having to go run a complicated database query or go process data that might take you a second or more or whatever it happens to be or some latency in between here we can actually take this information and cach it into a dedicated caching service now this caching service can now be shared across different instances of other service and we can layer these things together which is really cool so we can have something that's very fast in memory and then we could also have this second layer of cash that's shared across different processes or different Services Al together but as you might start to suspect here that complexity starts to go through the roof like we haven't even talked at all in this video about cash invalidation right we've talked about sort of expiring entries but what happens if someone said hey I need to go update this item the thing that's in your cash is no longer relevant it might not be so bad if you just control it like a diary in memory but as soon as it's across all these other processes and shared into service good luck trying to coordinate that right it's very complicated and there's really Advanced Services to try and make this more streamlined for you but the reality is it's still not easy it's one of the most difficult things that we have in software engineering the next cach that I want to look look at it's just going to be the interface that we're going to talk through but it is going to be I distributed cache and we're going to build on top of that so let's go see that on line 7 what I'm doing is just changing the dependency injection I'm adding distributed memory cache this is a bit of a cheat here because I'm not looking at implementations of distributed caches the most popular one that people will know is called redis there's other ones available as well but you could have something like a redice implementation of the distributed cache and then you can see on line 12 I'm now using I distributed cache here instead of IM memory cache here's where things get a little bit weird you'll see that this code here isn't compiling anymore and that's because there is no get or create a sync and the way that we have to work with the distributed cache is with a different API so if I get rid of that and type cach if we look at what's available right I can get things and I can set things and what's even more kind of awkward if I want to get something out of here you can see that it takes a string key and then what it's returning is a bite array the things that I just want to pause to talk about here in terms of the API usage is number one we're forced to use a string key that's not so bad right we could go take our day of the week and we could go make that a string really not that much to complain about there but the two things that are a little bit crappy compared to what we've seen so far and especially if we've contrasting the sort of evolution of these different caches that we've seen is that well one this only returns a bite array we're dealing with so far this um weather forecast type inside of a list but now that has to be a bite array so already what's happening is we need to consider that we need to serialize and deserialize this data well why does that have to happen it's because we're talking about a distributed cash we need to take these things from memory and take them and convert them into bytes so we can go send them to somewhere else this is like I said another layer of complexity serialization and deserialization is performance overhead so another reason why something like an offbox cash so something that's not running locally might take more time to go do because you have to do these processes but the other weird thing that we have here is that we have to either get the data and if it doesn't exist then we have to set the data right so here's the other API we can use set async we can put in the key right and then we could pass in the serialized data but if you remember what I said earlier about having to do separate get and set is it brings us back to this problem of what's called a cash Stampede just to kind of repeat it and I realize that for folks that are more familiar with uh cash stampeding this is probably not going to do the explanation Justice because this is a pretty complicated thing that happens in distributed systems so what I do suggest you do is when you hear me talking about cash stampedes please go read more about this online look for visual examples uh there's other videos online that will explain these specific things more in detail the whole point of this video is to give you some highle Concepts that you can go drill into but when we're talking about a cash Stampede we kind of saw this in some of the earlier examples where we sent in all these requests and none of them had had access or none of them had that entry in the cach right so all of them said not in the cash cool let me go to the downstream service when we have this type of API it forces us to separately get and set things some of the other apis we saw like the memory one or some of the other ones we'll see cing up they actually build this in for you so that the sort of API the infrastructure can try to control how many things in parallel can sort of bypass that cash so just a super quick example if you had a hundred requests coming in so you have a really heavy Service uh or heavy traffic for this service have 100 requests that come in what might happen is a hundred of them at the same time go to check the cash and with this cash this distributed one all of them if it's not in the cash all of them will check the cash and then go Downstream to call that service so basically all 100 sneak through and have to go Downstream to go get data in our example that's just a 100 of them that have to wait here and kind of uh sit here on this delay but in a real system that could be a database call or some other service right now with these other caches that have what's called Stampede protection what they are able to do is basically control how many things go through so maybe not only one is able to go through and that's because when you have multiple instances so say we had five instances of our service if you had a 100 requests split across those five instances Let's Pretend equally you would have 20 requests going to each service okay when that happens each one of those Services has Stampede protection and that means that out of 20 going in parallel what's going to happen is within the process only one of the 20 is able to go through at a time so instead of a 100 requests in in total sneaking through to go Downstream or in our case a 100 of them sitting here for one second what's going to happen is that only five one from each of those processes will go through and that way you're not having what's called that Stampede go through and hit that Downstream service you've very much limited the impact of when something expires from the cash or there's no cash entry so like I said if that's confusing you want to read more about it I think it's very important to understand if you're starting starting to build things with caches especially in distributed systems if this is just something for you to go build for fun like a hobby project on the side and you don't have a ton of traffic these types of things will introduce a ton of complexity probably not for a good reason but if you're scaling up systems or want to understand that please go read more about this stuff distributed cash does not give us that protection there is something else that we can use that Microsoft has it's not yet included by stand in the current version of net that we have at the time of recording so it is in pre-release maybe if you're watching this in the future will already be included but it's called the hybrid cache and as you might guess by the name of hybrid cache it will take the IM memory cache and the I distributed cache and give us the best of both worlds so let me go ahead and show you that code all right so a couple of things to discuss here you can see that I have ad hybrid cache I have these pragmas around here because if I go take them off just so we can see it's complaining at compile time right so it's saying this stuff is only for evaluation purposes the time you're watching this if it's in the future which I think by definition it must be in the future depending on when though this stuff might already be included you don't have to worry about these pragmas but if you're watching it sort of right after I release this this is what I'm doing to make it compile just to mention as well if you want this included you can see that I have this package reference right here Microsoft extensions caching hybrid I think in the future they do plan to include this directly so you may not need a whole separate package but you'll notice here that it is in preview okay so I just wanted to get that out of the way so you know where it's coming from but same idea we're using dependency injection we're registering the hybrid cache okay and I'm not configuring you can see like there's no other hidden code here right I'm not configuring like redis or some other distributed caching system so out of the box right it's only going to have the memory one you don't even see me adding memory here explicitly but I'm certainly not adding anything that's extra any other process or anything like that to use it it very much is just like that memory cache example we saw right I have hybrid cache here right on the minimal API again we can access this because it's part of dependency injection so we can get this past in for us looking at this syntax here right it's not a separate get and set so this in the signature is giving us that uh Stampede protection kind of API at least something that will help support it here get or set we have to use a key so this one is also using strings for the key of the cach this is going to be the factory right same pattern that we saw earlier if it's not in the cache call the factory to create the item and put it into the cach and then we have some options so here what's pretty cool is we have expiration and this is going to be for the distributed cash part and then this is going to be a local cash expiration I don't have a distributed cash right you can see there's no other setup here so we should essentially only be using this 3 second cache but let's go ahead and run this we should be able to see that we can get a cash working just like this out of the box so if I press enter we should see 1 second there we go and if I keep calling it it's act like it's already cach but now that I've waited a little longer you can see that it's taking some time cached again and now uncashed there you go so again this is I know some of the examples kind of look sort of boring or repetitive I'm just trying to show you that we get the same behavior and that way you can tune these things to your liking but this is cool because if you wanted to go add a distributed cach now the best part is that you get this API that looked like the memory cache that distributed cash will be another layer of cash behind the scenes that you don't have to think about when you're using this cash and that's really powerful right CU like I said we get the sort of helpful API that we saw with the memory cache but we have the power to have multiple levels of cash in our applications because this hybrid cach like I said said at the time of recording is in pre-release it's not yet included by default there is one more caching Library we're going to look at this is what I've been using in my SAS called brand ghost I may switch to the hybrid one when it's publicly available sort of like by default I'll have to see if there's a good reason for me to move over but this next one that we're going to look at I had a lot of success with and it's called Fusion cash to kick things off you can see if I go to the package references we have Ziggy creatures that's an awesome name and fusion cache uh I've just pulled in a preview one uh not for any specific reason probably just because that allow preview check box was checked on and now I'm going to go back to the code and kind of show you what's up with Fusion cash I think something that they've done a really good job with for Fusion cash is that the apis that they have are very similar to what you have with memory cach and hybrid cache they've thought about Stampede protect from the beginning they have really good documentation online to be able to explain how you can set up what they call like their back plane for syncing your cash data together it's super cool I think they've done a really good job I'm not you know I'm not being paid to promote them or anything like that but I'm sharing this with you because I felt that it's worked really well for my use cases and if you want something else another tool in your tool belt aside from the hybrid cache this is an opportunity so very similar like I said add Fusion cach for dependency injection right makes it very nice to not have to configure a lot with fusion cach and I think probably with the other caches that we have through dependency injection if you wanted to have different cash instances like named caches you can go configure that with cash providers we're not going to walk through that in this example because it's just a very simple API that we're caching so nothing to really be concerned about there but if you wanted to have different caches you can go do that but what we're doing here is like we saw before on line 12 we're basically able to use dependency injection to get the fusion cache and this should look very familiar I realize like I said some of these examples probably seem boring because you're like hey it looks the same that's kind of the point is that they've done a really good job making these very helpful uh familiar apis right so get or set should help us with Stampede protection in terms of how they've designed this so that's great you can see that they have the factory method to go create something if it's not in the cash already and then we have cash options as well so you can set the duration there's other things too like there's fail safe so you can decide or tell your cash if the sort of the factory method throws an exception while you're trying to create the thing should it allow you to access the previously cached thing should it sort of clear the cash there's different Behavior so a lot of this stuff is very tunable again I will have other videos on this stuff but this is really all that we have to deal with and if I'm going to go press play we can go check in the browser and see hopefully the exact same behavior so this one is every 5 seconds I just have to remember that so that was one right if I keep pressing enter it's cached if I wait just a little bit longer and I press enter now we should see that it has to go calculated again spinning there we go right so the whole point is that it does work the exact same and we didn't go run this uh this code that frequently throughout these examples but because we're ending off on Fusion cache let's go basically blast our API and see if Fusion Cache can keep up with it like I said I think some of my examples are kind of bad here for the timing just because we're doing it from the same process but we should start to see some of these coming back so yes I know some of these are pretty high but we can already see some of them coming back if we were to maybe get rid of the duration right we just keep things in the cache because if you think about it while that's running things are getting uh removed from the cach let's go back and watch if I did this from a separate process we probably wouldn't see so much delay but let's get a little bit lower here right once we start leveling off we should see fewer of these ones coming in with these high uh these high times but um I think a better way that I could have sort of put these examples together and I apologize I'm just trying to do it for convenience is I should have made a second process that I could have kicked off and then just run it against uh there's probably some way to do this in Postman very easily or there's probably other tools for generating traffic but I thought I would take a shortcut just to make it simple but I realized that can make some of the examples a little bit confusing um and I also realized I was probably highlighting the port sometimes um which is even more embarrassing so anyway um I think we should probably be able to see that aside from the very beginning here we have pretty good um like that's a pretty high latency but if I scroll down lower high latency still high latency but we start to get into this territory where the latency is very low um these ones are still pretty high but I hope you get the idea right it's like when we're kicking off all of these things at the same time like the process is uh we're only able to run so many threads at the same time so some of these things are a little bit more delayed but this like I said at the beginning is a bit of a hack but the idea hopefully that you saw is that we have these very similar apis to use when we're working with these different caching libraries personally the one that I've been building with for the last little while is Fusion cache and I think that I will try to explore the hybrid cache from Microsoft as soon as it's sort of included by default and no longer in pre-release hopefully you found this helpful I will try to add more videos into the playlist around cashing so you can explore that more in depth and that way this tutorial that's already sort of close to an hour you don't have to watch a full hour's worth of content and we can start drilling into more specific situations that you'll want cashing for them so thanks so much for watching and I'll see you next time