RPG Dev Log 7 - Unit & Functional Test Decisions

so i'm going to talk through some testing challenges that i just kind of encountered and it was a bit of a interesting scenario for me um just based on the types of tests i was trying to write for something um and i kind of kind of give you like a real concrete example for a situation i was trying to handle in this uh this fun game we're creating and uh and sort of the trade-offs that i was making in terms of how i had to structure my tasks so for some context for the actual situation that i'm trying to create in the game is that the character or like you as the user i guess should be able to take items that you have in your character's inventory and based on the type of item if they're stackable you should be able to drag and drop items that are stackable such that the stacks combine so you know as an example if you had you know a stack of five coins and another stack of ten coins if they're the same coins and there's like stack limitations that are met you should be able to drag and drop those coins onto each other so that's the idea that i'm trying to accomplish here and i already have this system in place for being able to drag and drop items in an inventory and equipment and so on and so forth so i figured okay i'm about to go mess with this to make this stacking logic so this would be a really good opportunity to start adding some tests in i hadn't already done this before for this part of the code so i figured before i get you know to changing this behavior i should i should definitely go cover this with some tests so i don't break it and uh i guess a bit of like hindsight at this point was i i don't think i acknowledged up front that some of the code that i'm about to go alter um isn't actually designed uh like as well as i would like and i'm gonna maybe try to draw that out kind of briefly what i mean um or maybe i'll maybe i'll just kind of point to it in the code that's probably easier so i'm gonna jump to the bit of code that we have that's responsible for handling drags and drops you can see already just by the the structure of this there's lots of like you know of conditional checks for certain things but the i guess the specific things i want to call out are maybe i shouldn't collapse those but uh there's a whole bunch of logic in here specifically you know i'm gonna highlight this section here out of this section this really handles um if you have two containers of items each of these will go run this block of code to handle swapping something in and swapping something out and that way you can think about if you drag and drop in an inventory if you want something to go to the other position they kind of have to trade places um if you're moving something into an unoccupied slot uh technically you're swapping with nothing that's the idea here however this block of code handles the base case but then there's also um like already you can the this part is the thing i just added and this part is some other thing that's a bit unique and the the thing i don't like about this code is that when we're talking about our entity components uh any component system framework that we're using basically i had to go modify this class to have knowledge about some entities and components that you can use for socketing and socketing is just like being able to take a gem or something and put that into a weapon like that that's the idea there and that's through dragging and dropping so i had to go modify you know this which should be kind of generic um uh sort of item container to account for this new behavior that we're adding into the game and for me uh that feels like a wrong pattern it should be kind of inverted where i don't have to go break open this class to modify it to extend the behavior and i knew that i was going to end up doing the same thing for item stacking because it worked well here however you know i'm going to have to find a way to extract these things and kind of invert that uh like who knows about the other kind of thing so that uh this class that we're looking at does not have knowledge about these specific things so anyway this is something i should have realized before jumping into the test but it didn't take me too long to kind of figure that out so i'm going to jump back to the test for this class and show you what i started to write in this particular case uh i was jumping into some unit tests and unit testing from my perspective is going to be basically focusing on you know parts of a method that i want to execute it's going to be a lot of mocking so you can already see here i have mocks being set up i use uh this moq framework mock and x unit basically for all of my my c sharp development they're just two tools i use together that i'm very familiar with so in this particular case i'll kind of explain a little bit about how this unit test is set up so if you don't have prior knowledge it might help kind of understand the code we're looking at but in x unit when you label something with a fact this is the test that's going to be run so this is an example of a test method labeled by fact so that x unit can identify this is a test to run the constructor that we have for our test class this constructor will run before every single fact that we have in here so just to quickly show you there's two facts that you can see and that means before this test runs we'll run this constructor and before this test runs we'll run this constructor so i can put common setup code in here and not worry that um like this bag item set you see here this will not be reused between tests it will actually be recreated just for full transparency i use strict mocks when i'm like what doing what i consider true unit tests and this is because if something is not set up properly it doesn't silently just run it will actually throw an exception and say that you forgot to set this up because we should be explicit with our unit tests and then otherwise we're just kind of creating the the mocks so these are the dependencies that our bag item set is using so hopefully that's kind of a brief setup for what's going on and then i'm just going to walk you through this first test that i wanted to create so something really simple so when we're swapping you have to specify what you're swapping out and what you're swapping in so on this method call here this is the method that we are testing and i'm basically saying that this test is going to be handling when this is null which you can see up top i should have probably defined it a little closer here but um yeah this value that we're passing here is null and we're actually passing in something some actual reference as as the thing we're swapping in so if we're swapping out nothing and swapping in something that's really just adding that's what this test is for and then we've got a whole bunch of other stuff kind of going on in here so because this is a unit test i need to actually mock out the dependencies that we have inside a bag item set to do the right thing so this is sort of a the first initial kind of trade-off with this unit test and there's a benefit to this and a drawback the drawback is really like i have to i have to set this up explicitly i have to know the assumptions about what this is supposed to do when certain things are called so for example i know that when this is called on this dependency if it's doing the right thing it basically has a contract that it needs to raise an event but i had to know that i had to go code that because it's a mocked dependency and but the benefit is like if i didn't set this up if i didn't set up this whole thing here like this this type of test actually ensures that the code i've written is being executed the way that i expected so it's you know it's a a pro on a con there i guess so i set out to approach it this way and if we kind of walk through what bag item set does in swap items i'm just going to jump back to that class now if we go to the very top here first thing it's doing it's trying to find an item to swap out uh i don't want to explain the details specific to the the game here just because i want this to be more around the test but basically we need to give it an id so this uh this item set will go get us the item for that id with a null that we're passing into here we're gonna get a null out i'm just explaining so i don't have to dive into each part of the code um and it's not going to be equivalent so we're not going to do this part we're going to go into this block here this is really just something that suppresses some background events from firing so because item to swap out is going to be null based on everything i've just said this is going to be the first part of this method that we're exercising that has a dependency which you can see is highlighted item container behavior where we call triad item so this is why in the test that we were looking at i have to explicitly set this method up so hopefully that makes sense i had to go say look we expect to call this we expect it to have this parameter and then i'm going to mock out what's happening there then if that succeeds which we control in our test right so in the test i can actually flip the return value around and say hey look what what should happen if it doesn't succeed like that's the power of this unit test is that we can kind of control the flow of the logic in uh the system that's under test so in our current example this is going to return true so we're going to skip over this exception throwing because it's like a valid pass and then we're going to invoke this event and return this value so that's what this test back over here is supposed to do so we call the test method it's going to end up getting to this dependency like i had mentioned it's going to call triad item do this callback because that's what that is supposed to do under the hood and we should expect on our bag item set that we do get items changed raised and this is just going to keep track of how many times that happens so at the end of all of that we should expect that we get this return value we should only get one of these events raised and the other thing we're checking here is that our mock repository we're just going to make sure that everything we've set up was actually called so that's the sort of the inverse part to a strict mock a strict mock will tell you it will throw exceptions if you didn't set something up but if you set up too many things right so if you have extra setups in your test this verify all will catch that for you so anyway it's a super verbose way for us to just try and add an item but i set out and kind of did this first test and then i said okay what's going to happen if we do it the kind of the opposite way so in this next test if we have an existing id for an item that we want to swap out and no item that we're swapping in this is not an addition this is actually a removal right so i'm just trying to cover these two basic cases and i'm going to quickly show you how i kind of ran into my first kind of hurdle here and and why i decided to switch my mind for going down this route so this test is going to look very similar but we're going to have this item that we were instead of like trying to in this example we were swapping it in we're going to be only swapping it out but based on the scenario i'm trying to swap out an item that already exists in our item set so to make it a little bit more complicated i have to basically assume that this item is already in the item set before i can pull it out so already that's a little bit convoluted i have to kind of do some extra setup here so um in this particular case you can see that i end up calling this to swap in the item here and this swap out item is set up up here as well so if we kind of i it's be a little bit more obvious if i kind of just walk through the setups that are going on so item id to swap out is being set up we are assigning that to an item identifier and it's actually going to be the same one that we're swapping in so i know that sounds a little confusing and this is hopefully it already a good indicator as to what i was kind of going through initially um like this is really however many lines of code that is that big block that i'm highlighting is really just to set up this item that i want to move around okay so the idea is that we we put the item in first and then we're going to try pulling it back out and i only want to start checking events when i go to do the swapping out portion of this so hopefully that makes sense i have to set up my container by adding it first and then i'm about to pull it out um but what was really confusing about that is that because i own these dependencies i already didn't have to do that i have full control over them so i i already kind of made a misstep here i didn't have to go add the item first i can actually set up the dependency such that it knows it's already in there so that's you know like i write unit tests all the time and i kind of already messed this assumption up so i'm just kind of showing that's a little bit confusing if that wasn't obvious yet um but the next part that made it even worse was that i said okay i'm going to jump back to this code here i have to start like an item to swap out that's not going to be null anymore that's actually the thing that we're trying to pull out of the the container in this case so we're not going to go into this block of code um this code wasn't here yet so pretend this is not there because um i guess i should leave the other part sorry this block of code wasn't there because this is what i was going to add after i wanted to test that this was working as expected ahead of time so then i had to start saying okay like because i'm about to access this dependency again i have to go set up items and then i realized oh crap i have to go make sure that i can handle all of this socketing logic and in this particular case i don't want it to execute so i have to go understand what's in here i have to go open this method check it out make sure that i can cause it to return early because we're not socketing items in this example so already you can see that i had to really start understanding all the inner workings of this class so that i could truly drive the sort of the code execution that i wanted it to follow now you might be looking at this and saying well you should have designed it differently like this should have been extracted you're right it should have and if i would have pulled this out this probably would have been a lot more easier to unit test so um i mean there's that but it's not like that yet um and this is why you know uh kind of upon reaching this point and going i don't know enough about how i kind of want to set this up it's probably it's probably worth that i kind of skipped the unit tests and go right to functional tests and that way i can actually kind of guarantee the expected behavior we want and then i can go refactor this then i can actually go pull this stuff out flip it all around and prove it still works so if you're still with me i'm going to go back to these tests here and i'm actually going to show you not these unit tests we're going to get rid of those and i have all of these functional tests maybe i'll put them back and then we can compare them but i'm going to comment them out okay so those are out and these ones are coming back in and we're going to look at the difference here and hopefully you'll see that these are a lot more straightforward to read and this is by the way this is coming from someone like i love writing true unit tests and mocking things like i love doing it i don't know why but this is a really good example of where it just didn't make sense i'll kind of reflect on that at the end again so let's go back to this is essentially a functional version of the first test we wrote in functional tests we're not going to be mocking anything out we're actually going to be using real things from like real classes that we have set up so i'll explain what's going on here for our setup because it's a little bit different i'm using auto fact for all of our dependency resolution so we have a container that's going to basically have all of our services and things that are registered that we want i need a game object factory because this is what's going to make items for us so i store references to both those things you'll notice this is static i only need this stuff to run once for all of the tests in here but just like the unit tests i need to make a new bag item set for each test that we want to run otherwise it carries the state between the tests and we don't want that so between tests i need a new one of these and what you'll notice is that it's truly a real bag item set it's truly wrapping a real container and these come off of our container and they are also real implementations so that's important here it's a big difference the other case for unit tests these were mocks these ones are actually what's in the game so hopefully that kind of is already explaining some some major differences here and because they're real things that are in the game instead of me having to manually set things up the trade-off is that i have to sorry manually set things up in terms of the things i'm mocking i don't mock but i have to ensure that any dependencies are in a in a state where i can use them and right now we're pretty fortunate in this case where we don't really have others set up so if we jump into this test again this is the functional version of the first one we looked at and you can see i need to give our item an identifier and we're just calling it the item being swapped this looks a little verbose just because it's based on uh some of our other uh sort of like game architecture but uh we haven't gone through and kind of put like a a nice clean syntax on stuff like some extension methods to make it pretty and easier to read and use so a lot of this is verbose but anyway this makes an item that we're going to move around so it's pretty simple this is the same type of thing that's going to be uh checking for events because we're going to count how many of those we fire um and that's it like we actually just tried to do the swapping which like the first example we talked about a null to swap out means we're not taking anything out but we're putting an item in so this is a test that adds an item to the container but you'll notice there was no setting up of dependencies that's gone we didn't have to do that here so at the end of this we can already just start asserting that the right thing happened so did we get the right return result did we get one event because we only expect one we're asserting that we're only adding like there's only one thing in the item set after and we're asserting that it truly is our item so hopefully you can see that you know there's way less setup code here in this particular example we were able to assert all the things we wanted and overall is very straightforward the next task was the one that i didn't even complete the first time around and again let's have a look at it very similar in terms of setup to this test here right we're just creating this item that we're going to be swapping like i explained earlier we want to have the initial state of our test be that this item is already in the item set because we're going to be testing that we can pull it back out so we do this as part of the setup now but look how simple this is right there's no dependency setups here i'm just kind of i'm setting up my initial state so that's like that's going to have to happen in either of these tests but i didn't have to set up the dependency which was nice we're going to be checking for uh events and then this is actually exercising the test method and then we can assert so overall like i didn't have to go set up any of my socketing dependencies uh i didn't have to go know about that um and because i'm not explicitly setting up dependencies in either of these tests i now have the the luxury of being able to go refactor the code inside of swap items i can refactor that and not necessarily worry about these tests um like not compiling or or breaking because of the setups they might they might break if i break the logic in here but i'm in a position where if that logic in there breaks these tests are going to tell me hey look you broke expected functionality versus with the unit test it's not that you necessarily broke expected functionality it's that you you have um the wrong things being set up now inside your unit test so they basically cover very different things but quick reflection just on this part in particular was that it made a lot more sense for me to use functional tests because i know already that i want to go refactor this this bag item set class i know that i need to go refactor and change the internals of it knowing that ahead of time to me was a really good reason that i should have started with the functional test because the unit tests i'd have to throw them away if i ended up going to refactor all of that so hopefully that makes more sense going forward once that's all refactored the way i like it depending on how separated out things get i might go add unit tests for some extra validation and to touch on that just some quick examples i think this was a pretty good spot here you know unit tests in my opinion are nice for this kind of thing where if i want to make sure that i can kind of force my code execution to try this out or to try this out or this like i can tune my my setups to be be such that it navigates down these paths but there's in my opinion there's too much code in this class to make a unit test set up that's really helpful for this because to get as an example to get to this part of the code i have a lot of setup on dependencies to have to get this far and that kind of sucks okay so i think that's probably all i wanted to say about that i'm just going to quickly pull this up sorry that that's probably pretty bright and flashed in your face but i actually put this other video together just quickly highlighting unit versus functional tests i put this together essentially before this video because i thought this would be a really good opportunity to touch on it but um i'll i'll put a link to it and it's it's like two and a half minutes long so if you made it this far you can definitely make it through this other one but it really just kind of touches on maybe some thoughts around when you want a unit test and why you might not so some cons and then the same thing for functional tests so when you're thinking through these different things it might help you kind of pick and choose when you should be writing a unit or functional test and i think the other thing to mention that's important is that um you should probably have a mix of both it's not that one of these is better or worse in my example that i gave today the the the moving of items in between this item set with uh you know like adding and removing things that lent itself really well for functional tests because because of this one sec uh because more resilient to refactoring right in my situation i knew that i was going to have to do this so that's probably why i should have started with a functional test so anyway hopefully that was a good example i'll try to put some links to the i think all the code should be on our github too so i can put some links to the actual code uh hopefully that's helpful if you want to go peek through what that looks like and otherwise yeah hopefully that was beneficial and thank you for watching