Xamarin Forms View Model - Unit vs Functional Test

i wanted to talk today about um an example i came across and some code i was writing where i wanted to start testing some functionality that i have in a view model um so for a little bit more context i'm making a xamarin forms application and i have sort of an mvvm parrot i'm going on so i have a view model that gets wired up to a view and the view itself has bindings back to the view model so anyway the the idea was that um as i was putting this view together with this new model i started to realize there's a bit of complex behavior um a little bit more complex than i'm comfortable with and just assuming it's right um and if i'm going to be iterating on it it felt like it was in a good spot where i could say okay seems to be okay right now um maybe let's take a snapshot of the functionality by putting some tests in to assert the expected behavior and then from there if i need to do any tweaking i have some sort of test that can kind of be used to regress against so um for myself usually for people that have worked with me or worked on coding projects together i often talk about i i really like focusing on on unit tests and i i define a unit test as something that um is really like a black box test or sorry i guess a white box says totally opposite sorry um so you're essentially with the unit test um i want to be proving that every line is being exercised the way i expect within a method um so they are very brittle there's a there's a bunch of setup that has to happen you're really trying to prove that everything that you're calling is exer uh is exercised and executed as you expect um and then conversely like a functional test would be that um it's less about the the details and the exact lines that are being called and everything like that um it's more about does it have the expected um outcomes so again i kind of said it inverse the first time there but a functional test being like a black box test and a unit test being like a white box test so as a programmer and for myself writing a lot of code where i don't have let's say like maybe people doing manual testing on the code i'm writing i really like to be able to have a lot of unit tests so that when something's broken i can i can really pinpoint the exact spot in the code um there's obviously trade-offs for doing that i'm not saying that only writing unit tests is the only way and this video is going to be in my opinion a great example of a situation where i said you know what i can't just get away with unit tests a functional test would be much better for this so with that said i'm going to kind of go through some of the code i got here i'm going to start with this view model that i have and i'm going to talk about some patterns in here that that i find lend themselves well to unit testing and some that start to make it a little bit trickier and kind of get me to this point where a functional test made more sense for the thing i wanted to cover so um personally i find that if we're passing in dependencies as interfaces in a constructor depending on the language you're using so this is uh c-sharp if you're not already familiar with it there's you know the concept of like an interface where essentially it's it's not a concrete implementation of something it's just the the api that we want to be calling i like using interfaces a lot because when i'm using mocking frameworks to mock out an interface we're essentially saying by asking for the interface here that this class does not care what the implementation is as long as something can exercise the the methods that were in the properties we're planning to call so this for me lends itself really well to to unit testing and the reason why is that i don't have to new things up and an example of that is right here so when you new things up when we talk about doing a a white box that's like a unit test suddenly once something has been um you know nude up inside of here so we've created an instance in this example of a command once we do that in here our test doesn't have granular control over this and the reason why is the test wasn't responsible for making this right so in a test i can pass in dummy implementations of this i can pass in real implementations i can have some sort of control over seeing what's getting called on these things because the test creates it in this constructor though we're actually making a new command so when i see the word new generally i'm not saying that you can't ever type the word new and you suddenly can't unit test something but uh that's an example of something where when i see something like new command written here it's a good indicator that if i wanted to have you know if i wanted to check at some point that this command is doing what i expect i'll have to think a little bit outside of the box compared to my normal unit testing behavior another thing that's interesting on this one is that in the constructor i generally try not to do this but we're calling some methods in here so in the constructor we have these two dependencies that are passed in they're view mo they are view model validators there we go um i don't know why that was hard to say but um we call validate property on them and they're going to the reason why i have it in the constructor is that they're going to sort of set some default state for us generally i don't like to call extra methods inside of a constructor because um it's not like again it's not something like it's a hard and fast rule like it suddenly makes it wrong it's just that um especially when we're going to unit test things and check behavior essentially just by creating like if we had to go make the system under test which would be this join group view model just by doing that we're already exercising some code um some logic on our dependencies pardon me so it's not that it's wrong it's just that like usually it's a bit of a sign for me where i'm like it's going to make unit testing a little less pretty another reason why i generally don't like to do this is that i know personally that these methods are very lightweight however if you get in the habit of um you know in your constructors calling methods you might run and i have examples in this code actually somewhere else where the the law um the logic that's behind this method might actually be something that uh doesn't execute like fractions of a millisecond or something and suddenly it becomes expensive to make new instances of things that's generally something you want to try and avoid in my opinion constructors for objects should feel like instantaneous so anyway this something i try to shy away from but in this particular case not the end of the world not a big deal so this is going to bring us to two properties in particular that i were interesting for me and they're linked to another property and i'm going to try to explain how all this works and hopefully this will illustrate why i wanted to um to test this so on this view model um essentially the the view we have has two text inputs and they're bound to this group code and to this username property respectively so two entries that for text and they're both bound to these two properties um what i have on here is that we have some attributes and this validation code and essentially that coupled with this code i was talking about up here this is this really cool thing that as a user is typing or entering text i can essentially put on a couple of annotations here to to do checks to see if the user input is valid i thought this is a pretty cool way to do things it works out pretty nice in the ui it makes some bindings pretty easy that's all cool but a couple of things that make this um not ideal are for unit testing at least or that we have these attributes on here um what's tricky about that is that um the kind of like when i was talking about newing things up technically putting an attribute on here is like newing uh up a new attribute we don't get control over it right so when this if i want to try something out with these regular expressions and these attributes and stuff it's going to use the real regular expression so if i want to like i have to suddenly have more insight about uh what i want to be testing not the end of the world it just can make things a little bit more complicated um the other thing is that if i'm writing unit tests and i'm putting in mocked validators here they they don't have any intelligence if they're mocked to go pull information from these attributes right so so suddenly by trying to mock some things out because maybe maybe the setup to make this validation stuff actually work in a unit test is really intense um and if i'm like you know what i just want to make sure that this method's being called you know it's being called with this parameter and this parameter and i'm happy um there's there might be particular cases where suddenly the setup for this becomes really complicated in my unit test so i want to avoid it but the result of that is essentially that if i mock out this validator suddenly it's completely detached from these attributes um so you know i might write some unit tests and someone someday comes along and they delete those attributes my unit test was never doing anything with those anyway so um the test would still kind of do the right thing right it's not actually covering as much as i might want it to um i could write some unit tests that do reflection to check for these attributes again so there's ways around it but it starts to make things a little bit more complicated by putting these attributes in place so this other property down here for username is it's almost identical to this so i'm not going to touch on it too much but the other thing that these properties do is that when we have a resulting change for the property being set we have some other properties that can essentially potentially change state so i'll give you an example there's this can join group property and we are saying that if someone sets the group code and it happens to be different which is the only way we can get into this if statement we want to go make sure that a property change notification happens for can join group and the reason why i'm going to jump to can join group it depends on whether or not the group code and the username are valid okay and those will depend on on the state of these validators so essentially if someone set a new username it might change the the state of the validator in in which case uh the condition about whether we can join a group or not would be different and the same exact thing happens for these right so if the username has an error or this username validation error is the actual text that's associated with the error so one's a boolean flag and the other one's the actual textual error so anyway these properties basically depend on some state from this validator um so it gets a little bit complicated in that when someone sets a username a whole bunch of different stuff can happen right it's the same thing with this oh sorry i'm on the wrong properties so the group code this one here and the username okay so very similar they have the same complexities associated with them um but one thing that i was kind of set on was like in our application users can navigate to this page and i want to make sure that the default state of this view model is that users cannot um press the button to join a group which is determined by this property so it always must be false by default okay so that's something i wanted to have a test for it sounds really trivial but um it's it's really important to me because if the state because these are complicated like these uh you know the validation state the actual text that's in these two fields um i want to make sure that i never have to worry about it and i have a test that will prove that it's always in the right state so i thought that would be a great thing to tackle so i said okay cool let's start writing some unit tests so i'm going to jump to the unit tests right now and show you how i started doing them and then why i actually had some trouble with unit testing this property okay so um i've gone through in my professional career i've gone through a few different iterations of how i like to set up i use mock so moq and x unit for my my tests um huge fan of both of these libraries and these frameworks so how i like to set these up for the most part there's always exceptions to to rules but um i like having an instance for a mock repository and an instance for my system under test um and then for the most part any dependencies that are passed in as mocks i like to also have instance variables for them the reason why is that in x unit per test your constructor is called again so if i had you can see i have one test here one test here and a theory which is made of a third fourth fifth and sixth test so the six tests in this file to go run all the tests this constructor will get called six times one for each of those tests um so i like making sure that i have essentially one spot that can make a um a system under test for me so in this case a join group viewmodel in the same way and that way i don't have this code copy and copied and pasted um you know six times for all my tests that's uh it's a lot and if i'm refactoring things so say one day i decide i want to add one more parameter to this or i want to change the order um you know maybe this one comes out um if i had that in six spots i'd have to go update six spots to go do that so in this in this uh in this setup there's only one another um yeah so if i if you remember that i was talking earlier i'm going to jump back to this join group view model i was talking about some of the stuff in this constructor not a huge fan of in particular these two validate property calls essentially just by making this system under test i have to set up um these two mocks with a call to validity property i have to do it or else if i don't because there's strict mocks it will explode so i won't even be if i took this out i can't even run any tests they'll all fail so i have to do that ahead of time feels a little silly but because there's strict mocks that's how i have to do it and for the record um i will almost always suggest that if you're doing or trying to do a true unit test that you do use a strict mock instead of loose a loose mock essentially when you call something with a loose mock you're essentially saying i don't care what this method does or returns i don't care what it does i don't care what it returns i don't care if it was called and in my opinion if you're doing that like you might be better suited for writing a true functional test um because when you have a loose mock you're not only saying like that you don't care about a method being called or the return value of it uh you're also it's like a totally imaginary type right like these are mocked iview model validators they're not even a real type like that would be in my application so a loose mock to me feels like i don't have a lot of good examples of where i'd want to use it if you find yourself wanting to i would say maybe you just want a functional test instead so title is kind of the same thing as these guys i had to set it up because it it's at it's accessed in the constructor so these feel kind of gross to me to have in my constructor for my test but not the end of the world okay so to get started with some some really simple tests i said cool like why don't i check by the the couple things i was interested in we're talking about the username property the group code property right because on here again i'm just scrolling through these are the two properties we were talking about a lot right because they have these um attributes and stuff and then this other property can join group right so i said well let me write two really really simple tests that i can check the default username and by default it should be null right it seems like maybe like in your opinion it seems like a really stupid test to have but it's super lightweight right i'm just checking that i made a new group uh model right because it gets created in the constructor i check this username it's null and that everything i've set up with my repository my mock repository if it's been set up it's been called so it's two lines of code and this just proves to me that a default state is that i don't have a username set exact same thing for the group code and then what i went to do was i said okay those ones are easy i want to make sure that the default can join group so i literally wrote this test the default can join group state should be false okay then i was all excited because i'm like great it's two lines of code that's gonna be exactly what i want and i'm gonna go run this and we're going to see that it's not happy and it made me not happy i'm just going to pull out this this window over here i'm still waiting on that but i should finish in a sec it's just visual studio being slow because obviously uh it wants to be slow when i'm recording and streaming so okay do something something's changing here we go okay so it failed um and then i was like okay well that sucks why um and it goes well you didn't set up these these mocks i was like oh okay well yeah that makes sense actually because um can join group right that's the property we're checking it checks these two other things which essentially asks the validators if they have errors or not but here's where the problem is and it's that if i go to do that um i have to actually tell the validators because they're mocked i have to tell them what their state is because i have to have full control and suddenly i was like oh crap i can't actually test the true default state i have to make assumptions right i have to tell it oh your um you know your username validator and your group code validator don't have errors to start with or they do have errors to start with whatever it happens to be right i have to make an assumption about it and then that will determine the state so i went crap like you know a unit test in this example is not going to do what i want um and it's because i want to know what the true like the whatever is set up in the actual application i want to know what that i want to know that actually works without any kind of having full control over it my test so that meant it wasn't a good candidate for a unit test however um i decided like why not try and do it anyway with the unit test i kind of did this after the fact so i came back and did this but here's an example kind of like i just said i have full control so if you're not familiar with theories a theory is like a fact an x unit a fact is just a single test a theory is just multiple tests and these parameters here so false false true correspond to this this and this right so there's four different uh inputs here and i'm using them for whether or not there is an error on the username and group code and what we expect can join group to return to us um so i have all the permutations false false true false false true and true true and then whether or not can join group should be the right thing so you can see that if i went to go do that other unit test i was talking about i would have to make an assumption about what the actual default state is for these and that just didn't feel right um so i said okay if i can't assume the default state why don't i do all the permutations um and this kind of tells me that you know based on if i ever want to check can join group it's really only checking the username and the group code validators if anyone has a keen eye you'll notice i don't this is kind of funny but i don't have a verify all call on here right it's just completely omitted from the test um a bit of a tangent but the reason why i don't have that in here is literally just because of boolean short circuiting logic uh which sounds super fancy but it's not at all um essentially uh just because i'm lazy on these these test scenarios i always set up these two validators right i always do it however i'm just going to jump in here if is group code valid returns false because this was false right so one of my mocks was false it will never even try to check this one because it's i it's called i'm assuming this is like a standard thing maybe i just made it up by accident but like it's short-circuiting the rest of the check because of boolean logic if this is false it doesn't matter if this is true or false because because of this and here so it never even exercises the rest of this if is group code valid is false and if that's the case what would happen is that if i tried to verify all when this was false is that it would say hey look you set up your username validator and no one ever checked it and it's not wrong like no one ever checked it because of that short circuit logic so this is just an example of like where i remove a verify all call but i will almost always try to put one in okay so that was a unit testy way to kind of approach checking can join group and what it does but again i was really set out to find you know a way to test the default state of it and the default state means that i i don't want to make assumptions about um you know how to set these things up then i'm kind of cheating because i want to prove the default state so i said okay i'm going to make a functional test for this so i've jumped to the functional test now um similar kind of thing here um i'm not going to dive too much into um why it's exactly set up like this but one really cool thing i like to do now in my functional tests is i use auto fact an awful lot for like almost anything i make i kind of i start with putting auto factor in right from the beginning i love it um and what it allows me to do is you can remember i'm going to jump back to the unit test here scroll to the top to make my join group view model i had to pass in three dependencies right one two and three um if i go back to the functional tests um i'm basically this is a little um shorthand i have a class over here for helping with autofact but um with these two lines i can basically set myself up to then go resolve and i join group view model and this is giving me back an instance of i join group view model that set up exactly how the real application would do it and the really nice thing about that is that so if i were to copy i'm just going to try and do it super quick oh sorry wrong file so what happens when i try to do it live um if i wanted to make a new one of these right so the alternative would be that i need to make a new join group string provider i need to make a new um viewmodel validator right so this right that line there and this line should in theory be the same thing should but the the one really big difference here is i have uh typed out three specific implementations to pass into this join group view model this might actually mirror what application is currently today at this very moment however someone in the future might say you know what um i'm gonna go register this other um we're not using the viewmodel validators anymore we're using a fancy viewmodel validators now so they can go change that in the core application and so that would be somewhere in auto fact for us and if i didn't change the test i would still be using these old view model validators right and then the risk there is like suddenly i'm out of sync with what the real application is setting up so anyway i really like using uh this approach because i can say give me whatever the is the real thing the application has um and then you can see that um my actual test to go check can join group is back to being like a a one-line assert so i just checked the property and it's false and to me that that's like exactly what i wanted to achieve and wanted to achieve in the unit test but i couldn't because because the dependencies i passed in i would have to go set up and i would have to make assumptions about um how those things should be in their default state um so yeah that was sort of most of what i just wanted to get across in this example of like um you know i think that you can absolutely have scenarios where uh you know if i could have unit tested this and it would have made sense great but instead of trying to fight with it and be like you know maybe i'm putting comments in about here's the default state and like here's what i'm assuming like instead of doing that it feels kind of gross and it's potentially just going to be out of sync with the application at some point i said well i can go write unit tests still for some things and they might make a lot of sense and i can have really granular control over what i'm setting up how i'm executing things that's great but the one thing in this example i was really after was like just tell me that the application will by default see can join group on this view model as false by default that's all i wanted to prove so really like that much code um is what got us that answer so a functional test over a unit test check this for us um and yeah that's mostly it um i think that's all i wanted to get across so yeah you can use unit tests and functional tests um another little side note is like technically i'm testing stuff in the ui uh a lot of people might you know as soon as there's something in the ui of an application they're already thinking about okay well what kind of clicking test do we need maybe we need screenshotting tests to like to check the state of the screen like in my opinion avoid that as long as you can because if you can check it in code without doing anything um fancy with actually uh exercising the real view the real application the better so i could check that those all work as i expect okay thank you