ASP.NET Core Tests That Once Saved Me Are Now WRECKING Me!

in this video I wanted to talk about some of my testing approaches and in particular I wanted to focus on how things that I focused on have changed over time as my project has evolved and one of the takeaways that I would like you to have from this video is that your priorities in your project and what you need to be focused on and developing to make progress will change as your project is growing and evolving so in particular for my case I'm going to be talking about an asp.net core application and the test that I wanted to write for this application so I talked about the test for this application before in other videos and I've talked about this concept of like you know progressing and kind of doing what's right now and kind of evolving that in other videos as well but I kind of wanted to combine them together so I want to jump over to some code and I want to talk about how these tests for me really helped when I introduced them and why they're the biggest pain in my butt now and what I'm doing about it so let's jump over to visual studio and check that out together alright so I'm here in visual studio and I have some of my functional tests pulled up that I am using to go test a recipe service that I have now initially when I was creating these tests I was having a challenge where there's a lot of the stack for this micro service essentially that needed to be tested but I'm rapidly iterating on it so what would happen is that I had some functionality in place and I needed to go introduce another route let's say and communicate with the database and add more records pull other records out and this kind of stuff is evolving really quick because the other teammate I'm working with is working on the front end of our project so one of the things I was struggling with and I talked about this in another video was that the way that I unit test things in order to essentially focus on the dependencies being isolated and mocked out those are really valuable for me when I know that a system is not changing much and the reason that's the case is because the unit tests become very brittle as I'm changing the code so for me to iterate fast it wasn't sufficient that I go have unit tests and just to kind of prove it to you if I jump into the implementation of this look how many things are now passed into this class as dependencies so you might look at this and say okay well Nick like obviously the solution here is to go split this class up and I mean you're right this needs to be broken up now but this has evolved over time so that's one of the things I want to call out is that this particular class did not always look like this there's been more and more dependencies added into here like that's a ton that's a ton of dependencies how how big is this file it's almost a thousand lines long right so this class I totally believe should be broken up into smaller pieces more purpose built at this point I think that makes a lot of sense but where I'm headed with this is that I was not able to adequately in my opinion write tests that would cover the important parts and be able to do that with unit tests so I'm going to jump back over to the test that I want to look at here um and again you'll see that I'm talking about functional tests here they're actually in a functional testing project and I don't have I'm using X unit but I don't have a mocking framework because I'm not going to be mocking anything here these are going to be tests that are testing functionality like an integration test if you will but I actually had another challenge which was that in order for me to go test this stuff so if I'm talking about unit tests I'd have to go you know Ma call these pieces out it was a lot simpler at the time so I didn't have all those dependencies so I have to go mock those pieces out and it would be brittle or I could do these functional tests but what I was not able to figure out early on was you know using something like Docker to spin up all my dependencies um if I had like we have multiple databases that we need to stand up I need to seed some test data like there's all this stuff to get it going um because that's the other trade-off with functional tests that you can have right so unit tests can be very brittle the way that I write them I know some people are going to be throwing up their hands going well no if your unit tests are brutal you're doing it wrong when I write unit tests I'm controlling every line of code that's being executed so by that definition it quite literally means that they are brittle to code change so my unit test would be brittle but functional tasks are going to require a lot of setup and at the time I just didn't have the infrastructure in place in my testing projects and to be honest the knowledge to be able to go stand up those dependencies get that test data seeded and then be able to write tests against it so I came up with a hybrid and it pains me to you know go make this video and explain to all of you like what I had to do but it actually served a purpose and it worked really well so what I'm going to explain I would not recommend that other people go do but it worked for me so I came up with a hybrid approach which is not unit tests and instead of having functional tests where I go stand up all of my dependencies you'll notice that I actually tagged these uh tests with a category called live and that's because these tests actually go talk to my server that's running in the cloud so I'm going to pause for a second and think about what that means but yes when I go run these tests this is actually hitting a live database I have other tests that are marked as live that are not going directly to the database they're going directly to my API so I have a mix of hitting my application server I have a mix of application server and database directly all of that stuff is in my code base and that's pretty scary in hindsight because that means that when I go to run these tests I'm basically spamming my service my live Production service because I did not set up a replica at the time to go handle all this stuff this is just the evolution of how my project was working but I needed something that would give me some amount of confidence in my changes so again I'm going to pause and just do a quick little review of why these tests are here but unit tests would be too brittle the functional tests that I wanted to write and have Docker containers spun up for databases I just didn't have the testing infrastructure and knowledge to go do that properly and I needed something that would give me confidence in my changes from the database all the way up to the routes that I have that I'm not going to go break the other stuff in our application so my hybrid solution was actually running against the live database and live application server so that's what these tests do the details of what's actually happening in here not super important but um just to briefly explain um I am setting up my dependencies here there's Auto fact behind the scenes I'm getting a lifetime scope and then I'm getting the actual service that's registered so this I recipe service is going to just going to click back over here it's going to get this actual class because there's only one implementation of that and then what I'm doing is I have some actual data about user IDs and stuff like that that I'm pulling back I have these marked as uh actually this is just incorrect that needs to be category is slow but or category is as smoke I'm not sure what the intention was there but um the idea was that this would just exercise some of the code that I have in my application and I use the word exercise and not assert because I'm not actually asserting very much in here at all again this is another contrast with unit tests versus functional tasks in our unit test we assert a lot of specific things and again that makes them brittle but in these functional tests the reason I use the word exercise is that it's giving me broad code coverage but I'm not asserting a lot of specific details so the value that I was getting out of that was actually very high because it meant that if I were to change something like a a database query if I miss something like a semicolon right just to give you an example I missed a semicolon or I added um an extra semicolon whatever it happened to be and now two queries got merged into one or something whatever it happened to be I could have a test somewhere and I did and I probably still have a few left over that would be hitting a route and they would go you know do some uh interactions with recipes and that would go right down to the database level and it would blow up and it was cool because it just gave me like enough confidence that I could keep making progress so that's where these tests got me to but what started happening is that if I I don't know if I can pull this tab out but I have over 1100 tests now and those 1100 tests are running I'm trying to get my continuous integration jobs working properly in Azure devops and just kind of build out um you know the continuous delivery Pipeline and just make that feel more robust um and what's happening is that the amount of tests trying to actually talk to the live server if I'm running say the local agent for the continuous delivery job on my machine and then I'm also debugging and running tests on my machine I could be totally blasting the server with requests and even if that was not the case we have rate limiting on our server which is good what's not good is that when I'm running all of these tests in the continuous integration pipeline I didn't set up the rate limiting to know about the pipeline so those tests just start failing sporadically because the rate limiting is being exceeded so you know the solution there in my opinion is not oh just go Um You know open up the rate limiting so that that pipeline can hit no I mean that is a solution I don't think it's necessarily the right one I think um my projects reached a point of maturity where I don't need to be doing this anymore I don't need to be hitting my live server so I'm going to contrast that test with another one that I've actually basically ported forward to the new format and this particular test it's similar right it's talking about the the recipes microservice this is actually at the route level so it's a in my stack it would be one level up from where the actual service runs um and what's important to call out here again the details don't necessarily matter but I'm using a builder pattern that I was able to create over time and actually stand up my dependencies so I run a local server I use a local MySQL container so I'm using test containers for that in other tests I have to set up multiple databases but I use this Builder pattern to be able to set up those dependencies all locally and they're they're torn down after the test is run I have other Builder methods to go put in dummy data so again this was built up over time and now it's really cool because I'm at the point where I have all this stuff in this Builder pattern and I can stand up my dependencies and I don't have to hit my live server so it's really cool because all of these tests that exist I'm just going to jump back to this other one that had the live category on it what's cool in my opinion is that because I used functional tests and not unit tests for this particular case I actually can go look at what the behavior of what was trying to be accomplished in these tests was and then go say cool like the test still makes sense I might have to change a couple of things in the test the test itself still makes sense but I'm just changing where I'm actually getting data from and where I'm you know communicating with so this pattern here with this test fixture is essentially just set up to go talk to the live server the other Builder pattern that I had over here in this test is actually setting up communication with a local server and a local container for the database so back to this test I can go change this I can go you know put in the local stuff with the local dependencies and then the test itself is not going to work because some of these things were depending on real data and what's being seeded in the database now is going to be dummy data but the cool part is that I can go change this part of the setup I can still go resolve a recipe service and then from there I can go look at the different tests that I have here I can change maybe the the user IDs or the recipe IDs whatever it happens to be and still kind of apply whatever the test was trying to go exercise and make sure that that's still covered so again the benefit here a couple couple things um it's not going to be hitting my live server so I'm not going to run into funny rate limiting stuff on my my continuous integration Pipeline and Azure devops and the other thing too is that I don't have this dependency on a live server so I can change my dummy test data it's actually going to give me more flexibility now that I have some of that infrastructure in place so that was just a quick video to demonstrate how my testing has evolved for my personal project that I'm working on and to kind of quickly summarize I was going from having a testing approach that I don't really recommend other people do that I'm hitting my live server because unit tests weren't a good fit and the functional tests I wanted to write I just didn't have the right infrastructure and knowledge to go Implement so this was a good middle ground for me to get that confidence and iterate fast now that my Project's evolved a ton I've actually been able to build up some of that testing infrastructure for my containers locally and I'm at a point where my continuous integration pipeline is actually being held back by these live tests that once gave me a lot of confidence so I'm at the point now where I'm just modifying these to Port them to the new format and then my continuous integration pipeline will be in a position where I can rely on that for confidence so I hope this was a useful example you might be able to apply some similar Concepts where if you're stuck on a project and you're going I can't do the perfect thing well that's okay come up with something and just understand that it's going to evolve and you can always make changes later so thanks for watching and we'll see you next time