Avoid These C# Testing Patterns... Unless They Actually Work!

and I don't care if it's an implementation detail if you find Value in this write a test that proves it testing and software engineering is something that gets a lot of attention and it's because people have a lot of different opinions on the best way to do it there are so many different ways that we can test and so many different types of tools that we can use to test with that you have so many options and picking the best one can be really challenging but this challenge is compounded when you have people that are experts on the internet telling you that there is one best way to test all of the things now on my channel I try to take software engineering Concepts and simplify them and I try to give you the tools that you can make your own decisions about what's the best way to go forward in the situation that you have so in this video we're going to talk about something a little bit controversial when it comes to testing because a lot of people don't agree on this and that's testing implementation details and I can see my YouTube engagement curve already right now plummeting into Oblivion because people are going no way am I going to watch this this is going to be stupid but hear me out I I think that there's a really good time and a place for testing implementation details just like I think that there's many situations where we consider things that might feel like anti- patterns or go against the grain a little bit so in this video we're going to look at testing a small little system in C and then I'm going to walk you through a couple of different situations that I feel like testing implementation details are very valuable before checking out the code just a quick reminder to check that pin comment for a link to my newsletter and let's go dive into Visual Studio in Visual Studio I have a small little system here but I'm going to start with a couple of interfaces and walk through the system code that's very lightweight it's contrived like most of my examples but I'll try to paint a picture for what we're dealing with here so starting with the external system so this is the one that we're going to be interacting with from our own it's just this interface and it has a method on it called get something external extremely creative and true Nick Fashion and beyond that we have a logger that we're going to be working with so when we have any type of issue or any information we're just going to call the simple log message so again I'm keeping this super lightweight there's not tons of different log methods and different things going on it's just enough to get us through these examples next I'm going to expand our system here and yes I'm using a primary Constructor whether you like it or not I find it very convenient for this kind of thing so I have a primary Constructor with a logger and the external system passed in and the only method that's public on here is this method called get something and you'll notice that it's going to return a nullable string and it's a little bit different than if I scroll back up this one here on the external system has an integer and the method name is clearly different as well so just want to call attention to that as we go a little bit further so what you'll see is in this method where we call get something I have something is going to equal try to get something so we have yet another method inside of here that we'll check in a moment and we see if something comes back and it's null we're just going to return null right we're not going to go any further we'll exit early however if something comes back and it's not null then we're going to try to do a conversion and if you check out what I'm hovering over here you can see that the something variable is an integer that's nullable so the nullable integer will get converted here in this convert method to a nullable string but we're not going to do that conversion if it's already null we'll just bail out early so if it's null exit if not convert pretty simple now let's go check out the implementation of these two methods before we go over to the test that we want a right to cover it in try get something we have as you might expect if you're used to this kind of syntax and this naming convention we have a try catch block around something that we want to do so we end up having just a return here on calling this method really straightforward nothing else is happening except for when we have an exception so we have this catch block we're going to log the message and then return null and whether or not this is a pattern that you like to follow want to follow that's not the point I'm not saying you should use a you know what people call a Pokemon Handler where you're trying to catch them all right some people like to have very specific exception handlers not the case here it doesn't matter it's beside the point and whether or not you want to log messages like this also besides the point I just want to call that out in this particular case we're assuming this is behavior that we want in our implementation you got to go with me on this it's just to make an example because there's a million different ways we could go do this now the other method that we have is called convert and we looked at where we called that earlier it takes in this integer and what we're going to do is just use a switch expression and if we have one two or three it will return the strings one two or three in their string version otherwise it's null so it doesn't matter what other number we have coming in at all it's just going to be null okay again does this make sense not really it doesn't really matter it's just some type of implementation that our system has when we go through contrived examples like this I need you to just understand that some of the things we're looking at Yep they're silly but it's just to make a point because there's so many different variations we could look at so if I scroll back up get something if we just have a quick peek at this method here this is what we're going to be writing some tests on and I'm going to pause for a moment because when it comes to testing what a lot of people will say and push is that you should only be testing the behavior right so you should ignore what's happening inside of the method and you should be only testing to try invalidate that you're getting the behavior sort of treating it like a blackbox and the reason that a lot of people say this and I think there's a good reason here is that if you have the system treated like a blackbox what you're able to do is change the implementation details and still get the behavior and that means if you want to refactor code and reorganize the internals you still have tests that work and that's very powerful because you truly can go refactor code and when you think that you're done if you run your Suite of tests you shouldn't have to really do anything else ideally and those tests should tell you whether you're good to go but the thing that I want to challenge here is that people stop there they say this is all you should be doing because as soon as you start testing implementation details you are now coupling your test that you're creating to the implementation as it might suggest and when you do that it means that your test and your code becomes very brittle because as soon as you change anything to do with that implementation detail suddenly you don't have a test now the reason I have a problem with this is because when you ignore testing any implementation details there's a lot of value that you could be missing and when I talk about value in testing bringing this back to what I said in the intro right I want to give you the tools to think about different scenarios that you can do different things and get more value so with testing it's for building confidence and I don't care what you're trying to build confidence with it's just that you write tests to build confidence it might be that you're building confidence in a bug fix it might be that you're building confidence in how you've implemented a feature and the way that you work on your teams you want to demonstrate to another stakeholder on your team look this test proves that the behavior I coded does what I said there are many different flavors of this and if we only look at the behavior sometimes we miss opportunities we're testing imp impementation details could be interesting so what I'd like to do now is head over to the code we're going to look at a simple test we're going to change it up a little bit so that we get some more coverage so we're going to look at testing the behavior right kind of what people would say to do and stop there but I want to show you why sometimes you may want to continue a little bit further because you get a different type of coverage and my point here just to really drive it home is not that you should not write the behavior-based test and only write this other one and it's not that the ones I'm suggesting are necessarily better in fact I do really believe that if you only could write one type of test the behavior-based ones are probably of most value but there are times and places for these types of tests let's go back to the code and go through some tests so that's the method we're going to be testing have another quick look at it before I get rid of it and we'll head down to the test that I've started writing down here I am going to be using mock for this you can use literally anything you want to be able to do this this is just for demonstration purposes so I'm creating a mock repository these are the dependencies that we pass into our system and this is our system under test and usually I don't use short forms like this I'm just trying to save some space on the screen for real estate and it's the same reason I've collapsed a lot of this code to put it all on a couple of lines here just so you can see it all at once but I create the mock repository with a strict mocking behavior in uh theq nougat package mock having a strict Behavior means if we call a mock that has not been set up it will throw an exception it fails the test so that's important when you're writing unit tests in my opinion otherwise you're not really validating that you're having coverage on something being called properly you're sort of surrendering to the fact that it could do anything or not even be called or be called when it shouldn't be you may want that in some cases but not for the types of tests that I like to write so this is going to be strict then we're going to create our two dependencies they're going to be mocks that come off of the mock repository and then we instantiate the system under test with these two dependencies so pretty straightforward stuff here but let's go look at one test that we could write to check the behavior so what I do in this test is I'm going to set up the mock so the test itself is called we're calling get something and the external system returns one and the expected value we get back is one written out all right so that's what this test name means here so we set up the mock so that external system as the test name suggests will return the value one then we should be able to call get something on our system under test and assert that we get the word one back very straightforward and you can see I have a green check mark here if I right click and run this it's the same thing that I did before I started recording just so I made sure it works now this would be one scenario right when we talk about the behavior what this method is supposed to do this is one variation now what we could do because there's a handful of different options in terms of what this method is supposed to do from a behavior perspective is we could go ensure that the other parts of this return 1 2 3 and also null that's the other type that can come back right so let me go quickly refactor this into a theory an xunit that lets us do parameters so I'm not copying and pasting this test and we'll have a quick look at that all right I've gone ahead and made that change into a theory and just to explain what I've done especially if you're not familiar with theories in xunit uh this is the original scenario that I had right so the external system returns one and the end result that we get on the method we're calling is the word one and just to explain how this part changed before going into the other Parts is that I now pass in the value that we're supposed to be getting from the external system that maps to this parameter here the first part of inline data and then expected value is this second parameter that we have right here so if we look at the other inline data entries that I have here these are just different variations that we can run of this test so numeric value two coming back gives us the word two same kind of thing with three but when we saw that switch expression we know that once we go beyond three we should be getting a null back and technically I could go right 5 6 7 8 whatever you do what you need to to have confidence I would personally stop here that's good enough for me and then I wanted to kind of go to the other side right I could have put zero in here as well maybe that's maybe that's one more variation that would be kind of nice to round it all out okay so we have one two three which gives us a string back and 40 and some random negative number one 2 3 those give us null back so if I run this just so you can see it happening run hopefully that doesn't turn into a redex and we're good to go so this is testing the behavior of this method but there's some important details that I want to be able to call out to you that we can't quite test obviously if we're only thinking about that external Behavior I want to scroll back up and walk you through a couple of different things that just aren't getting good coverage in my opinion and now there's a couple of things in particular that aren't covered but my point with explaining this is that I don't want you to think about just necessarily oh we didn't cover these lines of code because in my opinion having 100% coverage is a bit of a pipe dream there's some diminishing returns at some point if you can do it great but that's not what I'm trying to suggest you chase what I'm suggesting you chase is that if you have some uncovered lines or some branches that are not covered that you think are very valuable you should come up with a way to test that and the first part that we're going to walk through is talking about code in that circumstance that does not need you to refactor it in order to test it that's important because if you need to refactor it in order to test it now we have some other tradeoffs we have to work with so the first one that I want to be able to look at is we have nothing that's currently covering this this TR catch block is an implementation in detail about how we've set this up that means that we don't based on the theory that we wrote have anything that's getting this external system to throw an exception or particular exceptions right the fact that this is a a catchall exception Handler you could have changed this to catch two specific types of exceptions but what happens when your system throws some random exception right should you expect that well you didn't catch it specifically it should bubble up probably I mean probably not what you want I would think I would think if you're trying to handle exceptions you probably want to catch all the important ones that this can possibly throw it's usually why I do end up going with something like a a Pokemon exception Handler whether you like it or not now the other thing that could be very important and I want to explain why is that this logger getting called could be very valuable in a situation that you might not have experienced yet and this is a bit of story time cuz this is really critical and it's related to what I do for work sometimes people think that calling a logger just doesn't matter right it's just for debug information who cares there's not a lot of value in testing that most of the time you're probably right to take that approach if you're not really that concerned with whether or not something's getting logged who cares right but if logging is a critical part of what you do on your team because that's really valuable in your team you may want to test it now to give you the story part here for what I do for work is I manage two deployment teams at Microsoft we deploy to hundreds of thousands of machines across the world for all of the M365 Services there's a lot of machines there's a lot of services there's a lot of deployments if something goes wrong in a deployment which it never does because we're perfect right but if something goes wrong what we need to be able to do is work with logs we need some type of information for my deployment team all of the engineers to be able to go through understand where things happened and make sense of it so when we're designing features and we're trying to be defensive we want to make sure that if we're integrating with other systems kind of like an external system in this really contrived example that if something can go wrong we have information about what's going wrong and that way when inevitably something happens again not that it does right but inevitably when it does we need to make sure that we can go through those logs and have the information needed you could find a lot of value on a team like mine to make sure that you're logging stuff if the way that you've written your code makes it very difficult to even test your logging you're probably going to find that it's not worth it for you however if you value it you would write code such that your logging's easy to test but the logging itself in this example is an implementation detail it has nothing to do with the API that we're implementing so a lot of people would argue don't go test it so when you take things as absolutes like don't test implementation details because it'll make things brittle in my case you're just missing out on having confidence that you want to make sure something's logged properly so I want to go back to the code show you how you can write a test like this and then we'll talk a little bit further all right this is the first pass of a test that we could write for this system and it's if you have key eyes and you scan this already there is an issue with it but I just want to show you what this could look like to start with so to explain a little bit what we're going to do is mock our external system to return or throw in this case an exception instead of returning a value so that's this line right here on line 108 we'll throw an exception with expected message in it and then we do expect on the external behavior of this thing that we should get a null back that's part of what we might expect from the API because it's nullable now interestingly enough nothing enforces that on the API even though it looks like it should be intended for that but there is no sort of compile time enforcement for that anyway if we were to go run this this is going to fail so let's go do that and we'll explain why it's failing so there's the Red X right it's failing because the thing I talked about wanting us to test is that we have it logged right now if we didn't care about that right if I made the the logger a loose mock this would pass and it would pass because we would call the log method the mock would go I don't care and then we would get down to this line here on 112 and we should get a null back that's what the code does and it would pass but I truly care based on the situation I was describing I want to make sure that when my team is putting out features and we care about logging that we have the logger set up and no I don't mean that all of the time I'm chasing my team down to do this I'm just giving you an example when logging is important or something else that seems trivial to everyone else is important to you you can go test it and I don't care if it's an implementation detail if you find Value in this write a test that proves it if I go run this now we should get a green check mark there we go so this is a test you could write if you want to have confidence that your logging is working for exceptional scenarios that are handled inside of your code totally an implementation detail and if we go refactor that code this test is going to break it won't make sense anymore if we've moved that mock and the try catch and stuff around but if you mix it with the other tests you have regression coverage my point is do them both if they're both valuable now one final thing to end this video and maybe a bit of my rant that's going on here I didn't mean for it to devolve into a bit of a rant but I've also heard people saying to not test private methods in this code we looked at we're technically testing a lot of private methods but I think when people say that my understanding is that they don't want you to um sort of find a way to from the outside call a private method and you shouldn't be able to conventionally we have some options that let us do it and I wanted to talk about that and I wanted to talk about it because I agree with them like 99% of the time you should not be finding ways to call Private methods explicitly as the entry point to your test right your system under test has some public methods on it you should be trying to go through that and I would argue that if you want to test some of the private methods it's a good opportunity that those likely need to be refactored and pulled out onto other classes in fact in this contrived example I'm going to go back and we're going to talk about testing this private method and this is a good example of one you could pull out onto a dedicated class so it's a poor example to use but I wanted to frame it up for why you might not have that option and how you could test it how it goes against the rules and why it could still be valuable so to explain what I mean this is the private method here called convert and most people and this is including me would argue that if we were trying to test our system under test we shouldn't directly try to find a way to test this method because it is private but my argument that I already made was that sure like maybe this belongs on a converter class right we move that out to its own implementation that's a new public method there then we can go test that directly that would be great but if your code base is not perfect and you have challenges in the very shortterm extracting things and it's worth it to you in the very short term to write a test because you really need confidence on something right right it's a pretty rare circumstance where you're unable to move something as especially as simple as this like I said it's maybe a bad example but if you found that you really needed to be able to test this right so to give you an example you had to go fix this bug where four was not handled right someone missed doing this this was part of the functional requirements oh no but for some reason and I don't know what that is in this example but for some reason you can't go refactor this out does that mean oh no we can't write tests it means H you probably shouldn't do all of the time what I'm about to show you but you could and you could do it if you find that there's going to be more value everything is situational right so this is just an option you have I don't necessarily encourage it but I want you to have it as an option you can make decisions about when this makes sense and before I show you how this works I name this this way on purpose because I just want to remind you I don't don't necessarily encourage this this is something that you could do if you found that you had no option and you really needed to write some tests on that and what's going to be the secret here some of you might have guessed it it's going to be reflection so I'm going to show you how that works so that you could write a test on that method but I do want to have the disclaimer here that why you shouldn't do this is because it's so easy to break this test because we are going to be using St rings to go look up method names that means that unless you're refactoring things very carefully you're going to have a bad time if you even change the name of that private method so let's go ahead and look at it I just wanted to have the disclaimer this little bit of reflection here is going to be the center of what our test does so I wanted to pause for a moment to explain it because if a lot of people aren't familiar about using reflection we can ask for the type of our system which is where this method is located we will say get method and we'll ask for it by the name convert that was that private method we looked at now there's these things called binding Flags you can leverage in reflection in this case we'll use instance and it's non-public because it was a private method once we do that we're now able to go execute it I should call out that co-pilot wrote this and it technically added in this bang operator here that's supposed to tell us we don't expect this to be null by the time we leave this line so that's interesting so if it is null it should throw but let's find out what we do next so in order to call this method we would call method to test and then there's this invoke method that exists when we're doing reflection and using this return type from get method but what's interesting about this is that because it's an instance method we do need to pass in an instance still and we have one it's this undor suut for system under test the next thing we do do is we pass in this object array that's going to map to the different parameters in this case it was just one parameter we need to pass in and we can start with the number one so what comes back well if we look at the result type here it's a nullable object that comes back so we could do a couple of assertions on here to make sure it's what we expect now this might be a little bit redundant but you could write something like assert that it is the type string coming back because we're using reflection it's a little bit weird and then we could check to make sure that it's equal to the string one but technically that part's a little bit redundant so we could go run this and this would be a test on that private method let's go ahead and do that so this passed and if you wanted to see how this could give you good coverage if someone came along and they accidentally deleted this part here commented it out that means this test should fail right but this test can also fail for other reasons which is why I don't necessarily suggest doing it so if I put this one back here and then I said well it's not called convert but I want to rename it to something else it's going to be called convert two just to make it up if we go back to our test this test if I run it it's going to fail but for a different reason and it's going to fail because it can't find this convert method even though the rest of the tests are going to pass so this is a type of test that I would use extremely sparingly but if you are in a pinch you're talking with your stakeholders you really need some confidence in the thing that you're trying to have coverage over you could write something like this and I can't wait to see the comments that come back after this video goes live and people say never do this there's situations for everything all right so in summary for this video we looked at the traditional way that people like to test for behaviors of things and that is we are trying to exercise some type of system and and we're treating it like a black box these types of tests are very valuable for the most part because we get to ensure that we're not coupling our test to the implementation details and when we do that it means that we can refactor code and change it inside and ideally we can even reuse those tests to ensure that our refactoring worked it's a really good way to add confidence over an entire system however the thing that I wanted to call out in this video is that if you just stop there you're missing out on potential other opportunities and these other opportunities might not be the norm I get it but they are opportunities and these are situational things that you could leverage the first one we looked at was things like throwing exceptions or logging and making sure your logging's working and then I even showed you how you can test a private method if you really needed to and while I don't suggested I've seen situations where it's really difficult to go pull out something to refactor it you don't have tests on the rest of the systems that are going to be functional to let you refactor it in the first place and you had to make a change inside there you just wanted to make sure you had some confidence so you can ship a critical bug fix and you want to tell me that you don't find any value in that I'm not sure what to say so I hope you found this useful I hope you found some different situations that you could think through to write some tests that might not be the conventional way of approaching things if you'd like to debate about these things in the comments I'd love to hear your thoughts and I will see you next time