How My Tests Have Evolved In My Asp.NET Core Application

so today I wanted to talk about testing and specifically the different types of testing approaches that I take when I'm developing software so for a lot of my other content and especially people that I work with and that I you know Mentor in person one of the things I focus on a lot is unit testing and really how you can write code that's very unit testable and I've found that over time I am writing fewer and fewer unit tests and I figured this video would be important because it's going to talk about why I'm kind of Shifting my strategy but why I still write code that is unit testable so before I jump over to visual studio and start looking at some code I just wanted to take this opportunity to discuss what I mean by unit tests and then the other types of tests I write and I think this is important just because people might have different terminology and different expectations so I don't want you to watch this video and then go well that's not what a unit test is by my definition so I'm not here to debate or argue what the right definite edition of a unit test is I'm just going to tell you mine and then I'll talk about functional tests right after so when I'm talking about unit tests my ideal unit test is really something that's going to execute very fast it's ideally looking at a single method and the other things that are important are that it's not touching the environment so that would mean like it's not going out to the Internet it's not writing files to disk or reading files from disk it's not um you know connecting to actual databases or anything like that the idea when I talk about unit tests is that it's focused strictly on the logic inside of a method and everything else is isolated from that so for me that does mean mocking things out so if you see my videos or if you're looking through code I write I do use a lot of interfaces um you know some people will some people will say well you shouldn't put interfaces around everything I I try not to put them on everything but when it comes to writing unit testable code and the fact that I mock basically everything in my unit tests I do end up leveraging interfaces for that now do you have to use interfaces to to mock things out I'm not entirely sure all the Frameworks that are out there for c-sharp that allow you to do Mocking but basically from what I've seen using mock moq um I believe I need to use interfaces and I follow this pattern it really helps me write unit tests so that's the unit testing part now the functional testing part it's sort of the opposite end of the spectrum these could be longer running it might be some people might say like it's an integration test right I'm I'm touching multiple parts of the system connected together and uh for me these are leveraged not to execute and exercise code inside of a single method but rather like some complex workflow and I guess it's also worth distinguishing that with a unit test I am asserting very specific things inside of the function but with my functional tests or integration tests however you'd like to call them my assertions are a lot more simple um and that's because if I had a lot of assertions to try and you know validate that every step along the way was perfect in my integration test that would be extremely brittle to maintain and in my opinion my unit tests are the more brittle thing where if I'm touching the inside of a function my unit test is probably going to break by functional tests probably won't unless I'm changing the overall Behavior so that's the terminology I'm going to be using when I get into this and I think with that said I can jump over to visual studio now so on my screen I have Visual Studio open I'm in my calorie asp.net application and on my screen I'm showing some tests that are result set tests and these are something that I would classify as unit tests by my standard and just to kind of explain what's going on I'm using X unit as the framework so you can see that I have fact here on top of my method which is the test method and fact is just what x unit uses to denote a test method and the system under test in this case is my result set and I basically if you were to scroll through this I just have a bunch of different scenarios that I'm going to be executing across my result set now in this particular case this result set object actually doesn't really have dependencies so you'll notice like if I go up I'm right at the top here I don't have any using statements for mock because I'm not mocking anything here so this is actually like one of the most simple variations of unit tests so just to kind of jump into this class to show you um the method let me jump back sorry the method that we're going to be looking at is try get page so let me jump right to that and you'll see that I have this code written here and we're not going to go through the details of this but really my unit test is just exercising this code so it's very specific to one method and like I said in this particular case I actually don't even have dependencies that I'm wiring up and passing in to this result set I wanted to contrast that set of tests with another set of unit tests that I have and this is for another class that I have called a quantity normalizer so you'll notice here that I actually do have Mock and a mock repository so I am going to be using the moq mock framework and X unit together and this is just going to demonstrate I'm just going to go through it briefly like the fact that I have these interfaces that I'm mocking out so you'll see in the Constructor here I'm using a mock repository to go create three dependencies right and then these three dependencies get passed into the the system under test so the quantity normalizer and really this is just a simple way of saying that the three dependencies that I have for this are not going to be real for the tests I want to have full control over everything that's going to be executing inside of this quantity normalizer for my unit tests so that I can actually ensure that the logic I have inside of here is operating as expected so just to quickly jump into this class this quantity normalizer this is a pretty nasty um class that I have that's uh used for basically uh trying to deal with unstructured information and this is something that I really wanted to be able to unit test because um the complication with some of these other things these dependencies they would be a lot to go set up and ensure that they're all you know functioning and sort of uh operating in a test environment so for my case I wanted to make sure the quantity normalizer worked and I said I don't care about the behavior of these other things I have lots of tests on them in other spots and one could argue for sure that I should have some functional tests that go across all of this together and test the integration of them for sure but in my case this is something that I wanted to basically have unit tests over so that I had a lot more specific control over what was being tested so this is a little bit different than what we saw before and in this case I'm using a theory from X unit and for those that aren't familiar a theory will basically allow me to have a set of data that's passed into the test and you can see like this test actually has parameters on it and it will pull that test data from this method I'll look at that after and we'll go through that and basically it's just going to exercise sort of different inputs and expected outputs for this method and the method that we're going to be looking at is try get quantity async looks it's a pretty simple test as you can see but that's because there's a bunch of other stuff that's actually in this uh this method that generates the data so I'm just going to jump into here to try get quantity async right so we're in this method now and the I mean like what we're looking at in particular is not super important but what I do want to call out is just how the dependencies are getting called here so you can see that I'm accessing a cache so this is a dependency that's passed in we saw that as one of the mocked dependencies the logger itself is also a dependency and just to kind of prove it if I scroll up a bit you can see on the Constructor I have these three things passed in they are interfaces and we saw in the test setup that I have them passed in his mocks and then if I scroll a little bit lower you can see that I have the third dependency accessed right so if I get them I know if I can get them all on screen I might have to zoom out a little but yeah there we go so food client I have the logger and the scraping lookup cache so what I was testing in my particular case is that I'm able to sort of flow over this logic and if I jump back to the tests you can see that I have a setup method on the scraping lookup cache and on the food client and actually on the logger itself I don't have anything set up and I'm gonna call out like a pattern that I've actually um this is something I don't recommend it's just kind of interesting because it's on my screen right now um this this test very well um at this point in the evolution of this project may actually be better set up as a functional test and I just want to explain why so every bit of code that I have is something that's going to evolve over time and in the beginning when I had this uh this is sort of part of a a scraper tool that I have and one of the reasons why this is probably set up as unit tests is because setting up the test infrastructure like I was saying for these dependencies like having a an actual food client set up um the the scraping cash is not so complicated but this is actually backed by a database and the logger is super simple but these two things together are actually pretty complicated systems behind the scenes at this point in the Project's Evolution and I'm going to jump to a functional test after to show you I actually do use Docker containers and stand up a bunch of dependencies but at the point in time when I was writing this I probably wasn't leveraging any of that so this is mocked out and just to jump back to the thing I wanted to call out here is when I'm mocking things I almost never will put something like this where it says it dot is any and the reason for that if I try to get this on the screen too you'll see that my mock repository is set up with mock Behavior strict and what this is going to enforce is that any time we're calling a dependency from within the code that's being executed it must be set up exactly the way we need it because a mock behavior of loose will actually just kind of allow things to get called and if he has if it has a return value it will just return the default but I'm actually trying to prove that things are called exactly as I expect when you start using things like it dot is any you're kind of surrendering to the fact that like it could be anything so I'm missing out on an opportunity to validate something here and again this is probably just because of at the time I wasn't able to set up good functional tests for this and when I wanted to go make this a theory which I'm just going to show you in a second to go expand this data um passing in the parameters the right way probably would have been just kind of messy but you will see that on the food client here this mock setup is actually not using it dot is any so it is looking for you know a particular food ID it's actually going to to prove that we have the cancellation token and stuff up like passed in here so that's all great but uh and like I said the logger is not actually even set up because we're not expecting the logger to get called it would mean that if something in the test uh sorry something in the system under test did end up calling the logger this whole test would fail and that's because a logger setup with mock Behavior strict and I'm verifying at the end as well so that's actually in my opinion a way that I have a bit of protection here that if that logic changes inside of this method here try get quantity async and say I started logging something or we called some other method on the food client I actually would expect this test to fail because it is a unit test so by my definition my unit tests are brittle because they're testing very specific logic if I go up to here just to expand this innumerable of the test data this is a little bit nasty but it's basically all the different scenarios that I'm trying to go run inside of that method we were just looking at so every time we have one of these yield returns with a new object array it's basically a new test scenario so there's two three four five six seven there's a whole bunch of them in here and that way I can test different variations over the same body of code but this is an example of maybe not a great example but of a unit test that I have that has mocked dependencies so next we're going to go look at a functional test all right so on my screen I have a set of functional tests and this is going to vary a ton from what we were just looking at with the unit tests and that's because there's going to be no mocking and we're actually standing up an entireasp.net application with dependencies and it's going to get a little bit interesting here so actually I'm going to show you this one and then one more after that's a little bit more complicated for the setup so this set of tests and I want to talk about the evolution of my test as well in this set of tests is a copy paste pattern from something I had early on and this kind of came out of not knowing how to properly set up Docker for some of my tests and get some dependencies stood up so what we can see in the Constructor of my test this may not make sense to anyone who's not working by code base which is fine but uh to briefly explain it I'm basically setting up a local client that will go connect to my application server but from looking at this you won't be able to tell but I'll explain that we're actually not even launching the application server so you might ask yourself okay well how are you testing it then and the answer is that it's actually going against my live server and running tests against that which is a little bit scary but this is all because I did not know how to set up Docker originally for my dependency so I have a mySQL database a mongodb database I'm going to have redis so I have a bunch of dependencies that I just didn't know how to stand up so I started writing these tests so that I get some coverage and in my opinion some coverage was better than none you can even see that I was documenting the tests with these traits that would mark them as slow that it was live data and then I would skip it on continuous integration because I couldn't even set it up in continuous integration properly but what I would do before committing is I would go run these locally and if things looked good it was at least some confidence that I could go check in my code so of course these aren't great but just to kind of show you um you know I want to be able to test a post method with an empty body and if I hit the API and if you see up here the base route is actually here recipes list enhanced if I hit that with a post I do expect that it fails with a particular error code and these tests are all kind of like that this one at the bottom is sort of the happy path so if we were to go hit this route with these parameters I actually expect it to succeed and then we get some response that comes back so the reason I wrote these tests like I said is that I wanted some confidence that I didn't blow up everything and the nice part about these tests even though it's against the live server which I do not recommend anyone do the nice thing about this is that it would basically test a particular path from the routes right so hitting a web API all the way down through the database and then back up which meant if I was hitting you know multiple micro services along the way if I was doing multiple database queries hitting caches things like that all of that stuff is getting exercised as part of this test and I use the word exercised on purpose because that is different than asserted so in this case this test would exercise a lot of code which meant there was a lot of code that was touched um and you know so a wide area of coverage but in terms of what's actually asserted you can see that I only have um like two things as part of this test this assert up here is just to prove that we could actually log in but this assertion is very simple it's not actually checking any of the logic inside so this is uh sort of one of my early functional tests I kind of explain why I have it and I'm going to go pull up another one that actually has a better pattern where I have the dependencies actually stood up with docker all right on my screen now is a similar pattern to what we saw except this one is not going to exercise code that's on the live running server instead I am using these fixtures that you can see here and one of them is for MySQL and you can see that's kind of passed into the Constructor here and then we saw a similar pattern like this Builder pattern and the other test but this time we're going to be setting up a local server and actually using the MySQL container so this test will go launch docker and it will go launch Docker for MySQL and it will launch Docker to actually go run my web API locally and that web API that runs locally will connect to that local Docker so two containers talking to each other now the huge benefit of this is that I'm not actually hitting my live server obviously that's not a great pattern that I want to be following but this is sort of the evolution that I've arrived at where I'm no longer required to be writing unit tests over very specific things and if we kind of scroll through this this is it's very similar to the other tests we just saw right it's almost the exact same pattern like you know we're testing if a method's allowed or not if I scroll to the very bottom there's sort of like this happy path variation so all the same reasons for writing tests like this that apply to the last scenario we looked at it's just that I don't have to run them against the live server so every test that I have that looks like the last set that does go against the live server I am going to be going back and refactoring so it will have these fixtures and the setup just like this one so now that we've looked at a bunch of code including unit tests that basically had no dependencies to mock out unit tests with dependencies to Marco some functional tests against a live server that I don't recommend anyone does and functional tests that actually run against some Docker containers with my dependencies stood up I just wanted to talk about sort of the different flavors of these tests that I have why I have them the evolution of that and kind of tie it all together with why I'm for the most part moving away from unit tests so this application that I have is growing significantly over time and a lot of it's in flux and what I mean by that is that as I'm creating routes and apis um the the other members of the team that I'm working with we are finding that sometimes a particular route we might have to blow away where we need something else that's new so we're always iterating and changing things and the reality with the unit test sort of setup that I was describing is that those are brittle so I could go write unit tests on all of that stuff but because it's changing so much a lot of what I need to be exercising is going to require that I would rewrite those unit tests so what I've been finding is that when I write these functional tests again trying not to hit the live server anymore but standing up the dependencies with Docker I'm able to get a way broader coverage um you know better bang for the buck so to speak when I go run these tests I know that it's going through my app stack I can test basic functionality that someone on the the front end calling my API they might expect that a sort of result to come back and just overall it's giving me like I said better coverage so does that mean that I've stopped writing unit tests entirely no I still write them I still write my code so that it can be unit tested and because I have these patterns that I've adopted over time that allow me to write unit tests easily so things like passing in all of my dependencies as interfaces that I could mock if I ever found that part of my system was solidifying more and I wanted to have more specific coverage over particular methods I'm set up to go unit test those very easily that would mean that the functional test could remain in place give me some high level confidence that you know the whole world didn't fall apart because I accidentally changed a SQL query somewhere um so I'd have pretty good confidence that things are mostly okay but for more complex areas in areas that are more solidified I can go back easily add unit tests and get that much more confidence that those um you know those really important areas are covered and the logic is sort of it's well understood and well well asserted on so um I think that's mostly it for what I wanted to cover in the next video that I put up I'm gonna walk through in particular my cash test that I wanted to unit test because I needed to have a lot more confidence in the behavior and I think that'll be interesting to contrast with the follow-up to that which is some of the functional tests and I'll step through some of the stack of that so we can look and analyze to see like why that's valuable to functional tests so hopefully this was useful I just wanted to compare and contrast the different types of tests I have why I do them that way and sort of how that's evolved over time is my application has grown so thanks for watching and we'll see you next time