Vertical Slices & Plugin Architecture - Principal Software Engineering Manager's Approach

all right today we're going to talk about vertical slices and plug-in architecture and these two things aren't the same thing but I'm going to talk about them together I had someone recently ask me about this in the comments section and I figured this would be a good video to make and I had someone else comment recently that they wanted some longer form videos where I actually take more time to explain things in depth I've been trying to make some shorter videos so I'm going to make this one a little bit longer go into a little bit more detail and I'm going to walk through some code that I'm creating for production and I'm going to share with you some different things that I've been looking at with respect to plugins and vertical slices so with that said let's jump into Visual Studio alright so I'm here in visual studio and I'm going to be looking at an asp.net core application and I'm going to talk about a couple of different entry points and some setup here that I have that kind of lend themselves well to plug-in architecture some vertical slices that I have within here and I'm going to show you a couple of different variations because the primary part of this solution is that I do have an application that is a web API which we can see here this runs in Docker but then I started creating some tools on the side and I found that I gravitate towards plug-in architecture for these as well so with that said I'm going to start talking about the calrate web API whoa that is a lot of stuff going on in this project but in this project I actually have it set up such that I have plugins and almost the entire application itself here is just a skeleton for launching a web application and the rest of the entire stack for this is located in this micro Services folder and I have all of these different micro Services um and I'm I should be using air quotes here microservices but these are the vertical slices and the plugins that I have that kind of compose the rest of my Apple location so what does that mean when I start talking about you know a skeleton for the web API project well if I jump into the program.cs I'm going to have it pulled up here I'll zoom in a little bit more if we look at the program CS file basically I just have a main entry point I'm going to be essentially using Auto fact to go resolve all of my dependencies um I'm I'm running something right now and I don't want to stop it I just wanted to record this video but I have a couple of code changes it's obviously not compiling right this moment but what I do is I resolve the web application and I run it here and then I just have a bit of a wrapper for doing some logging but okay well what is this application actually do well if we start looking through the other content in this project I just have a folder for logging I have a folder for some exception handling and then I have another folder for auto fact modules so really between these two modules this routes module and this web app module this is where I mean the entire body this skeleton of this web API really exists so just to kind of touch on this code right here this is going to be what I have for my own setup that's going to allow me to load in all of my plugins so I have this discoverable route registrar types and I'm pulling them out of a container so for those of you that are not familiar I'm using Auto fact which is a dependency injection framework and with a container from Auto fact I can register all of these things from my plugins and then I can resolve them so what I'm doing in particular here is I'm looking for all the types that have a discoverable route registrar attribute once I have those I'm actually able to register those particular types and make them a single instance and when I go to actually run this application and it's resolving some of these extra steps that I have here so this one is called post build web application dependency what I'm doing is I am looking for these discoverable types and then I'm actually registering them to this app so this code in particular allows me to have all of my plugins register their routes to the application so for most of us when we're spinning up an asp.net core application especially if you're using minimal apis and things like that now your entry point for your program is pretty lightweight and you can already start declaring your routes right in your program.cs file because the minimal API syntax makes that super easy for us to do however because I wanted to split my application out into different plugins and I'll touch on that a little bit more as to why each of my plugins needs to be able to say hey what's the application that's running and I want to be able to say here's the set of routes I have and register them into the application being able to structure code like this is kind of interesting because you have to start thinking about things a little bit backwards from typical now what I mean by that is usually if we're thinking about creating a set of apis in our web application you might be thinking holistically about all of the different apis that you want to have in your project and of course if it's nice and simple and lightweight you could just go at all of the minimal apis right into program.cs right so if I jump back over here you might have in your your main method here or if you're not even using if you're just using top level statements you can quite literally just start registering minimal apis right on your app and that might work um it's you know if things are simple you can just register a bunch of routes like all right in this one file nice and simple but I wanted to be able to basically have a skeleton application that I never really have to go touch so the amount of times that I come back to touch this particular code is almost never I have it set up and if I want to extend my application I never really have to come back into this calorie web API project I quite literally just go build new projects in this micro Services section and then I copy the dlls into the bin folder and at runtime all of those plugins will dynamically register and I can get entirely new functionality in my application without having to come back here to register new routes so let's go have a quick look at what some of these individual micro Services look like and I I'm going to touch on a little bit more about why you see so many projects here I should actually link a video that I made about following some anti-patterns I thought that was kind of an interesting video to make following some anti-patterns and basically taking things to what I would call an extreme to see how they work and see their limitations and this is certainly one of them so under my microservices folder I have a food service and you'll notice there's one two three four five six seven eight nine ten projects inside of this for this one micro service and if I go expand the others for you you'll see that it's quite literally the same type of thing where each one has a whole pile of projects inside of it now what I was trying to do here was go a little bit overboard with separating out the different uh call them layers that I would have in my plugin and this is where I want to briefly talk about vertical slices each one of these micro services that I have is a plug-in for my core application however each one of them is also a vertical slice and what does vertical slice mean well traditionally if we're thinking about sort of like classic architecture that we look at for programs a lot of the time it's done in layers you would have something like a user interface layer and then some business logic and a layer sort of below that and then below that you might have a data access layer and one of the sort of trade-offs you make with this layered architecture is that you're grouping up functionality in terms of the layer itself but that could span multiple domains and what do I mean by domain well each one of my micro services that I have here is what I would call a different domain for my application so I have a domain for food I have a domain for Brands I have a domain for being able to do tagging unit conversions recipes all sorts of things and when you're dealing with a layered architecture you might have all of your let's say your SQL logic is in one nice layer except if you want to go add new domains you have to go into that layer and go add more logic for that particular domain instead with our plug-in approach and taking advantage of vertical slices all that I need to do is go create a new one of these folders for a different micro service so let's pretend I'm adding food I have a project for routes and this is where I was kind of showing you a little bit earlier I have a registrar here there's that discoverable attribute and it has this register method so at startup it will go register all of these routes that I have here and from there if we were to dive into what one of these looks like you can see that I have some logic here doesn't really matter what it's doing here but this is going to call into the way I have it set up another layer but that layer is very specific to my domain for food so my route will call into my service layer and I've just expanded this project on the left here you can see there's a ton of stuff inside of it and in there I have uh like in this case I have a Foods service which has a whole bunch of different stuff this generally maps to a lot of the function calls that we would see on the routes themselves and then going back to this layered architecture approach I actually have a data folder I haven't broken it out into another project this is where for me personally I started to draw the line between hey look like I'm making a lot of projects here uh the only thing that really needs to care about this and access it is this particular project itself I don't need to share it so I made a folder instead of a whole new project and dropped in like my SQL uh you know logic but in theory if I were taking that same pattern of blowing out projects for everything this data like I might have a calorie foods data project and then another project specifically for the SQL implementation of things so you can kind of see how that pattern can get a little bit extreme it blows up your code base pretty fast but in particular this is where I kind of Drew the line just because I didn't really need to worry about it you might notice too that I have these client projects as well and these are just ways that I have my other domains able to access this service so for example when I'm dealing with recipes well recipes have to know about Foods however recipes don't need to know all of the details about Foods they don't need to know about the the you know the data layer for Foods they don't need to know all of the intricate behind the scenes stuff going on for foods but they do need to be able to access food information so my recipes microservice that I have if I scroll down to that my recipes service project is actually going to be able to access the foods client so that it can pull food information but like I said it doesn't have to know the details about it so for me when I have all of these different domains interacting I'm using a client interface I'm just going to go back to the foods microservice I'm using a client interface and if I look at this project there's no implementations I just have this interface and the rest of this stuff they're just dtos to be able to interact with this interface so any any domain that I have that wants to interact with another domain they go through this client interface now the cool thing that I think about this client interface is that I can have different implementations of the client interface and if you look here on the left I have a monolithic one and a web-based one so my application my calorie web API is actually a monolith and I know you're going well what the heck Nick you have a micro Services folder here and you're talking about plugins like how are like what's going on like how is it a monolith how does that all come together well I think maybe the term is like a modular monolith so I do have a monolith it's all deployed together I don't have these Services running separately but my monolith does have plugins for all of these services and because of that I can actually use a monolithic client that basically calls directly into the different service projects that I have and that way I don't need to make web requests between itself it would be kind of ridiculous for me to go run a web server and when I want to go interact with my own domain on that same server that I have to go making HTTP requests so as a result I do have like this concept of a monolithic client and if we go look at this we're for example this food client you can see that it references this Foods service directly if we were to go into the web client and a lot of these I don't actually have populated you can see it will just say it's not implemented for almost all of them but you can see here this is a web-based one so it's actually going to make a get request to be able to go perform some of these actions so if I wanted to go make other applications that interact with my web server this is where I would use one of these web client implementations but all of the referencing code just gets to look at this API that I have in iFood client so let's pause for a moment just to reflect on some of the concepts so far I did talk about having this calorie web API entry point project and it is a modular monolith I believe that would be the term used all of the routes that I have are loaded from plugins and I just kind of walked you through the food and the recipe ones for examples and each of these that you see in this micro Services folder actually has a plug-in that is basically going to be able to register the routes for that service now at some point in the future if I wanted to I could go deploy each of these micro services or combinations of them as microservices instead of this one big monolith but I'm not there right now and for me it doesn't make any sense at this point to go scaling out different containers for all of these Services when I can just have one single instance as this web API that's working totally fine right now I'm going to jump over to one more project before I talk about one more anti-pattern that I've kind of introduced into here just to contrast with what I've been talking about so this calorie debug console is something else that I treated like a bit of a skeleton application so what do I mean by skeleton application again well every time I want to extend the functionality of this debug console I don't ever have to come back into this core part of the code and change it all that I need to be able to do is register a new tool so you can see here that I have this I command line tool Runner and this tool Runner if we go look at what it does leveraging Auto fact it's able to pass in an inumerable of I command line tool so anytime I'd like to extend the functionality of this application if I go to this tools folder all that I have to go do is add a new tool and as long as that tool implements I command line tool and if I put this attribute on it to make it discoverable and automatically registered it will quite literally just get taken up by the skeleton program and all of that functionality will become automatically available for me now right now all of the plugins these tools they are embedded inside of this calorie debug console so that does mean that I have to go rebuild it technically but that's because it's just simple right now if at some point in the future I wanted to have different projects that could go build these different tools so say this clone recipe tool I could quite literally go move this to another whole project build it independently and as long as I drop that dll into the calorie debug console it will automatically get picked up and I don't actually have to go rebuild calibrate debug console so why didn't I go the full stretch to start with well I don't want to go make a whole bunch of projects again like I've like I was kind of illustrating uh earlier when I was going through these micro Services folders you can see that I am the type of person that will go separate code out into tons of projects I don't actually recommend that people go do this it's just something that I've kind of adopted because I like organizing my code that way but I do recognize that there is sort of this trade-off of having so many projects and it becomes a little bit unwieldy so like I said at some point in the future I might go take these different tools if I need to and go put them in separate projects but for now now there's quite literally zero benefit to doing so so far we've looked at two different skeleton applications one of them is a console application where we can have plugins for different tools the other one is an asp.net core application where we have plugins that get automatically registered for all of the routes and I demonstrated that with those micro Services again with air quotes around that that each of those micro Services is its own layered architecture I have a vertical slice for each micro service so that I can go deliver functionality end to end and by that I mean from the route that you would access all the way to the data layer and I have that self-contained in a little set of projects that get registered okay so what about that anti-pattern that I was talking about earlier well this application didn't quite start this way to begin with this repository is almost 10 years old and I took a bit of a break before coming back to it but when I first started off this repository I did start doing a layered approach and I had a data layer that I started with and you can see that I have calorie data data for SQL and then a my Sequel and then a my sequel already s for Amazon like I I started breaking down like these layers here and I'm just going to demonstrate to you how I started combining all of these different domains into like the basically this base data layer and why that's kind of caught me off guard now that I've switched to like a more plug-in style architecture so if I go into this data SQL you can see like I have food nutrients I have Foods I have nutrients these are all these cached repositories right I'm going to go over to um just the data implementation you can see a calorie data and then I have Foods nutrients recipes and if you recall I had foods and recipes as separate micro Services right so in the micro Services folder I have food and I have recipes these are supposed to be completely independent from each other because they are microservices and when they want to interact like recipes accessing food they go through that client interface I mentioned but if we look at this layered approach that I originally started with years ago we can see that I have foods and recipes in here and that means that the foods and recipes data is actually all compiled into one spot calorie data and that makes it a little bit messy because each of these things share this dependency over time as I started creating these other micro Services I was actually able to migrate a lot of stuff out of this data layer the remaining stuff for foods and recipes is still kind of stuck because I have some code that's actually going to be sharing access to these and I need to find a way to decouple them and I guess that's just going to be a work in progress right so another kind of quick reminder for folks is that you know your code is is living you're going to be iterating on it I I mentioned that I had started with this layered approach and I'm not a fan of layered architectures not like this and I'm not a fan but I used to be I used to write code all of the time in layered architectures and this is just sort of a remnant of me doing that and that's okay right it's not like I have to start from scratch and you know go blow all of this away no I was kind of already explaining that I was able to pull things like units and stuff out I think I pulled out I didn't pull out nutrients recipes and Foods because those are pretty coupled currently but I had uh you know the ability to make meal plans and like units and stuff and tagging all of that stuff got pulled out so it's doable it's just going to take time and right now if I were to look at you know this code and I'm reading it and I'm explaining it to you and I'm saying saying you know I kind of feel like hey it sucks that I have these things coupled together in some layer the reality is that it's not actually causing me any problems right now it's not like I have different teams of people trying to deploy different versions of this code and you know everyone's getting upset because you're stepping on each other's toes there's quite literally no issues with me having these things coupled currently so I'm not stressed about it I'm not like losing sleep at night trying to find a way to refactor these things because all the stuff that I had to move already it's moved when I need to I will come back to this and I'll find a way to refactor it that might mean that I have to change up some of my queries because they're expecting data you know to be in tables that they can know about but they can no longer know about it for some boundaries for the domains I might have to find ways to duplicate some data across different databases I don't know yet but that's okay I'll cross that bridge when I get there all right so that was just a walkthrough of this calorie solution that I'm working on just to recap what we talked about I showed you two different entry points a command line application and an asp.net core application both of them I have designed to be like a skeleton and they load plugins so the applications themselves are very lightweight they only have a little bit of stuff for registering dependencies maybe doing some logging and that kind of stuff just so that it's kind of set up and you have a shell of a program but from there I showed you different ways that I I have plugins and with the command line tool those are just other classes that I have right inside the project I didn't separate them out into anything fancy so as long as they need an interface and have an attribute they'll get automatically picked up and that way I can keep extending my tool very easily the other version the asp.net core application this is where I was showing you that I took a pattern to an extreme I have this folder for each of the micro Services you know micro services and with that I have a bunch of projects in there and each one of those sets of projects for the micro service is kind of its own layered architecture but it's a vertical slice so it goes from the routes to the database layer and back up so it's a full end-to-end implementation of a set of features so that covers plugins and vertical slices and then to wrap up I showed you an example of how I start started with a layered architecture and my data layer is still kind of hanging around I have a little bit more to clean up that'll get to that at some point so hopefully you found this interesting I know it was probably a little bit longer than usual and a lot of me just blabbing but that's uh some insight into some of my projects so thanks for watching and we'll see you next time foreign