How To Create A Multi-Tiered Cache In C#

in this video I wanted to talk about caching and specifically a multi-layered filtering cache that I'm trying to put together for my asp.net core application so we're going to be looking at the first part the multiple layers and how that's structured and then the second part is about filter sets of IDs that come back from when we're running filters against a database so before I jump into the code I just wanted to touch on the point that I made in another video and I'll put a little link up here where there's opportunities where like you know as software Engineers someone has probably written the thing that you're trying to do and I think in a lot of my projects I do try to take the the the the hard way right I try to do stuff on my own so that I can learn about it and when I don't find something that fits exactly what I want sometimes I'm like hey this is a good opportunity for me to try it for me to learn about it and then on my Journey if I come across something where I'm like actually this fits better I can always swap over to that so I just wanted to share that perspective because this would change a lot if I had paying customers or this is for my my day job and I needed to deliver faster I would probably spend less time building on my own and more time sort of um finding resources that fit so I wanted to share that first just so it's a hit that's a little disclaimer before we start looking at my code and with that said let's go to visual studio and start looking at the multi-layer cache all right so on my screen I am showing the interface for an I cache layer and it's pretty simple it's actually really just um kind of copied from what we have accessible in the distributed cache that comes from asp.net and really we just have the ability to try pulling things out setting things refreshing them and removing them so really simple API uh kind of you know the basics that you would expect on a cache but what's the deal with the multi-layer part well when we're thinking about applications we are familiar with things like in memory caches right you could have you could quite literally have a dictionary somewhere that is a basic cache I personally like to use I'm just going to jump over to it this uh this Library called bit faster they do have a handful of different variations for different types of caches um I really like this fast concurrent tlr U cache um personally I just find that it fits almost all of my in-memory needs for uh for really basic things and you can see that all that I've done here is I've wrapped all of the API methods off of this cache layer around the bit faster cache methods that kind of match up to it they're not perfectly of the exact same but it fits it works and for the most part this will be a really basic in memory free representation that I'll be using for my multi-layered cache but what other layers would we consider because if we already have something like a in memory one I had another example I'll just pull up might look almost the exact same but this is just a dictionary um if we have two variations of in-memory caches like what else would we want as a layer well what about a distributed cache so there's lots of flavors of this but in particular I wanted to start looking at backing up my in-memory cache with something like redis so that I had way more opportunity to store things in the cache and not really stress my application server by slamming more things into RAM I could have another you know instance in the cloud that is my my redis cache and that way I can have dedicated resources for caching and not have to worry about other performance implications for my my application server just to jump over to this other project I have this distributed cache layer this one is a little bit more complicated but what I thought was really handy was that I can basically wrap anything that is implemented with this I distributed cache that comes from this Microsoft extensions caching distributed namespace so a lot of the asp.net core cache is that you know the packages that people might want to use redis or other things a lot of them will implement this I distributed cache so I thought this was going to be a super handy way to basically say if I want redis or something else I can pass in that dependency here and then through these other methods and you can see that I have a couple other things like for converting the keys and the values this way I can interface directly with an i distributed cache like redis and take advantage of that so I'm not going to walk through all the specific details of how this works because for the most part you can consider this almost like a pass through it's just that there's a little bit of extra logic to be able to go convert the keys and the values because how you store those things in redis for example it might be a string or a byte array and that's going to imply some type of serialization whereas when it was in memory if I jump back to the bit faster one for example I can actually declare in memory that the cache is of type whatever key I want and whatever value I want so no serialization necessary but with something like redis I will in fact have to go serialize okay so that's a couple of different layers of the cache that we might want to look at but what about how we join them together so I do have this multi-layer cache object and how this works is that it takes in a list of the different layers and still implements the same type of interface where it will actually go have all those methods like try get it has the set method a refresh and a remove and all that it's going to do is basically do iteration over those layers and it treats them in a particular order so for example when we try to get something it will try to get it from the in-memory cache first and then from there if it can't find it it will fall back to the other layers now what I thought was interesting and this is where someone else who has a you know their own dedicated library for this that's well used there's probably a lot of things like caching's quite complicated so there's probably a lot of other Behavior to consider but one thing that I was thinking of was that if you were to pull from a particular layer of the cache that was later so for example the in-memory cache misses because maybe it is smaller and has a shorter time duration that things will exist in the cache and then let's say it hits the cache and redis I have this method or this part of the method here rather that with an option we can turn on if we missed the cash in earlier layers I can actually go set that value so depending on your caching needs if you're finding that you're hitting the cash in a sort of a further out layer or a lower layer that's a little bit more expensive right going to Something in memory should be basically instant having to go to redis is still going to be very fast but it is going to be slower than directly in memory that would mean that if you know in your particular case that you want to have you know faster access the next time you can sort of populate the the earlier layers of the cache and it is an option because maybe you're like I don't care at least hit the cache if uh if someone wants to go set it because that's how your application's flowing then the next time you go to set it will set it on the uh the inner cache so with that said let's go jump to the set method so set a sync you can see here all that I'm actually doing is running in parallel for all the layers we have just to go set um the key value pairs right so uh it's just a little bit uh this is kind of gross I do have a this interface problem that I couldn't really figure out because I can set things like the time intervals the cache entry options per entry or not but regardless if you just look at one of these code paths the idea is that per layer in parallel I'll go do the setting so that means on a particular set of the cache it will go populate all caches and like I said if I go back up to the get you can opportunistically go refresh the inner caches if they were to miss an outer or lower layer cache were to hit but it's an option so I'm going to play around with this and when I do some performance analysis then that way I can validate whether or not this is Handy all right so that's a quick overview of my multi-layered cache I did say that we were going to look at these result sets and I'm going to follow up with that in the next video because I think it was important to get this out of the way so you can see the different layers understand why I want a multi-layered cache and understand why I wanted to build it on my own to kind of experiment and explore but in the next video which I'll link at the end of this we're going to look at what we do once we're trying to query a database with a particular filter and how we'd actually like to Cache those results and you'll see how the multi-layered cache comes into play when we look at the implementation of that so thanks for watching and hang around for this video right here