A Beginner's Guide to TUnit and Microsoft Testing Platform

As C developers, many of us have been using XUnit to test our code for ages now. But we do have other options available to us. Hi, my name is Nick Cantino and I'm a principal software engineering manager at Microsoft. In my previous videos, as part of this playlist, which you can check out right up here if you haven't already, we're looking at Microsoft testing platform and then comparing and contrasting the different testing frameworks that we have to work with. I want to put together content on TUnit and walk through how we can leverage TUnit with respect to Microsoft testing platform, but I wanted to make this quick video to do a bit of an introduction to TUnit by going through some of the repository and talking about some of the features. So, if that sounds interesting, just a reminder to subscribe to the channel and check out that pin comment for my courses on domain. Let's jump over to GitHub and see what TUnit is all about. I'm at the GitHub page for TUnit. This is by Tom Hurst. It's quite popular already. So, this is really cool. I know other content creators have made videos on this already, so I'm behind the curve on it, but this is something that I wanted to do as a precursor to the next video. We'll keep this one pretty short just so you have a bit of an intro to TUnit, the modern testing framework for .NET. It says TUNIT's a next generation testing framework for C that outpaces traditional frameworks with sourcenerated tests, parallel execution by default, and native AOT support. So, one of the things that's really stand out about TUnit is that there is source generation, which is pretty cool. I think traditionally a lot of the ways that this works, this is coming from someone who loves reflection. I use reflection all the time, but a lot of the traditional methods will use reflection at runtime to discover tests and do that sort of thing. So, this is something that, you know, throughout my software development career, uh, even hobby projects going back for forever, I've been very comfortable with reflection. So when I think about, you know, testing frameworks using reflection to go discover and execute tests, it just makes sense to me because that's how my brain works. But the fact that we have source generation for tests is pretty awesome. Parallel execution by default is also interesting. It's not that the other test frameworks don't have parallel execution. It's just that they made this sort of by default. It's sort of the paradigm for these tests. And then the other thing is native AOT support. This is not something that I have spent time exploring it, but I do know especially with talking with more people doing like more modern web development like AOT is really top of mind for them. So I think the fact that we have that in here is a really cool opportunity. But they have this comparison chart. If we look quickly test discovery like we were just saying runtime reflection for traditional ways like XUnit absolutely does this and then TUNT is compile time generation. Again, if you're not understanding the benefit of that, that's totally cool. Just a quick note, reflection is slow, right? So, it's not something that you want in the hot path. If you're doing it sort of once on startup, maybe it's not so bad. I write a lot of my code to this day using tons of reflection for plug-in loading once on startup, right? So, you pay the penalty once, but I get the flexibility and how I want to structure my code and load things up. But if you have to do this on a hot path or if you're thinking about running tests in a CI/CD pipeline over and over at scale, sometimes this kind of stuff can really add up over time. In little bits, not so bad, but in repetition can really add up. So execution speed, they say sequential by default for the traditional frameworks, but they're parallel by default in tun. This one again, like I think this is more, it's not necessarily like a oh that means tunit's way better. I think it's just calling out like this is a paradigm shift because otherwise it is sequential by default for the other traditional testing frameworks. They say that you know limited AOT support on the other frameworks. I've not explored at all AOT for the other ones. So I can't comment on that. But again I'm mentioning this because if you are interested especially in AOT this might be something where you're like okay Tunit's going to be the option for me. That doesn't mean uh that you need to completely, you know, ignore the other options like NUnit, XUnit, MS test. I would just make sure that you're cross-checking on those to see if they support what you need. But it seems like AOT is sort of like a first class thing with TUnit test dependencies. It's not supported traditionally in the other testing frameworks. This is super interesting. You can see they have like this depends on attribute. I I hadn't I don't know why I hadn't even thought about this until I was looking at some of the T-Unit documentation on this page. I'm gonna pause that thought until we scroll down a little bit lower, but I thought it was pretty neat. And I have some use cases that I'm going to consider for this. And then resource management. They say manual life cycle and then this is intelligent cleanup. I'm not totally sold on this yet. Um because like I feel like if I haven't told Tunit what I needed to clean up, I'm not sure how I can expect it to be intelligent about that. Sometimes when we hear about like magic features for me, I get a little bit of a hesitation because I'm like when there's magic, if I don't know how it works, then like that becomes more scary than helpful magic. So we'll look into this a little bit more. But I think that there's some interesting options here. I am pretty excited to start using it a little bit more. So, we can see for the quick start part, um, I called this out in a previous XUnit video, but make sure you get templates installed. I am personally like for someone who was using XUnit for ages and ages, I would just, you know, go make a new project, a class library. I'm not taking any templates or anything. Just go add them and then I go to another test project that's set up exactly how I want. I go to the project file, copy the contents over. It's just how I operate. So, the fact that there are templates, like you should try to use them. They're there for a reason. So do that. You can add a new test project that way. The key features, there's a bunch here. I really want to focus on the code examples. I just want to pause on this part to see if there's anything that stands out. Talked about source generated tests, parallel execution, native AOT. It says optimize for performance. They have some benchmarks. I think the AOT ones stand out. In my very simple example, which you will see in the next video when I'm going through code in Visual Studio, unless I change something between right now and when I film that right after this video, the XUnit ones were faster. Um, the tests are very trivial, but the XUnit ones were 10 times faster. Kind of interesting because there's not many tests I and they're so simple. I don't know if it's just like once we start scaling those out, we'll really see the difference or if I really need to use AOT to see the difference with TUnit. They're all like we're talking milliseconds here, but the fact that, you know, I could consistently see like 10x improvement on XUnit. I don't know, right? Like I'm not discrediting TUnit. I think that this is something that they're keeping in mind, but I have not yet seen that benefit. So, just a heads up. Rich data and assertions. This is pretty cool. Like in the XUnit video that I put together, I I hadn't even thought about matrix for uh test data. So that's pretty cool that we have that in TUnit as well. Fluent async assertions. I think that many of us are really finding that the sort of fluent syntax is something that we gravitate towards. We'll see that. It's on the bottom of the screen right now, but I will scroll to that in just a moment. Smart retry logic and conditional execution. I'm not totally sold on this idea yet. Um, maybe I need to see more examples of how this works, but smart retry logic seems weird to me in tests. So, I don't know if there's some interesting scenarios that I've not yet had to work through where that would make a lot of sense, but interesting to see. Let's go to the other side. Advanced test controls. This depends on thing. We I'm excited to talk about that. It's one of those things where my mind hasn't like totally figured it out yet, but I know that I have some things for that. Parallel limits and custom scheduling. Again, for brand ghost, I have some some challenges with spinning up test containers and having a bunch of Docker instances running. So, I'm very curious if I can explore something in TUNI to help me with that. Built-in analyzers, compile time checks, of course, custom attributes. Yeah. So, these are things that like we do get in like XUnit and stuff as well. Just calling it out because I want to do a bit of compare and contrast as we go through all of these videos in this playlist. So, and then developer experience, right? Full dependency injection support. I feel like this is the kind of thing that we really need to have as like a first class thing in all testing frameworks. I don't know for especially junior developers or people just getting started in C. If you if you're not familiar with dependency injection, if you hear it and you're like, "That sounds scary. It sounds complicated. I don't know what it is. I'm just pausing for a moment to say please go learn about this early because I think that when you get exposed to this early, it will save you so much more time. I think it was years into programming before like it clicked for me. And I truly mean this. I cannot think about programming in a way that doesn't use dependency injection. Now, I can't I can't wrap my head around how much more like verbose and obtuse my code would be. So, please, if you're seeing this, whether it's this video or other things or like dependency injection, oh, I don't know what that means, please go spend a little bit more time and try to uh I'm not saying you have to understand the nuts and bolts of all the dependency injection frameworks, but please spend some time and try to to understand that a little bit better. IDE integration, of course, right? This is I think we need to expect this out of testing frameworks, right? if if someone publishes a new testing framework tomorrow and it's not going to work in Visual Studio or VS Code or Ryder being, you know, another super popular one, it's not going to get used, right? So, I think that's helpful. I haven't gone through all of their documentation. Anytime I see documentation on a GitHub repo, I will shout out Jod Denetti for fusion cache having the best documentation I have ever seen for a project. So, shout out to Jod Denetti. Okay, let's go through these simple examples here. The thing that I wanted to focus on is like simple attribute up top, right? This is pretty uh you know expected for testing frameworks where we can mark the tests versus just like a helper method in the test. It's async, right? So we have async usage in here and then we can see that we have like they called out the fluent syntax that is also async. So when we say fluent syntax for folks that aren't familiar with that, the idea is that it starts to read more like a sentence and you can chain things together. You might have seen things like link of course where you're doing like a you have a collection then you're selecting or you can say where or call to list on it. This is a similar type of fluent syntax but for assertions right so you can read it like a sentence assert that the user created at is equal to the current time which is daytime now and then within a minute. So this would mean that uh we're comparing these two things and there's sort of like a minute of of like leeway or flexibility in the comparison. So for some people I think some people hate this kind of thing because they would rather see it more like actual just like uh I don't know like a formula comparison right and that's totally fine but I think there's many of us that are gravitating more towards this. There are other fluent assertion libraries and stuff available, but it's pretty cool that they just have this built in from the get- go. So, I think that's pretty cool. For myself, just for context in when I've been using XUnit for ages, I haven't really been using Fluent Assertions at all until probably I don't know the last 6 months or so, I've added I'm trying to make sure that Fluent Assertions the Nougat package is added in and I can make that more of a common use case for myself. So, this is growing on me. Um, I am the kind of person who's not sold on this quite yet, but um, I think that as I write more and more code like this, I can see myself shifting that direction, especially from a readability perspective. When it reads more just like an English sentence, I think that there's probably less cognitive load. I'm personally probably just paying that extra cognitive load because I'm so used to the other way. So, let's keep going. Data driven testing. This is like I'm going to do some comparisons to XUnit here because probably people are familiar with XUnit, right? So this is kind of like a theory in XUnit and this would be like inline data. In XUnit, we have fact or theory. A fact is no parameters on the test and a theory is with parameters. In TUnit, it's the same, right? We just have test. This one up here was kind of like a fact in XUnit because there's no parameters. You can see and here there are parameters and it's still just test. And then we use arguments like inline data from xunit and now we have a parameterized test. Pretty cool, pretty straightforward. I like that it's just the same attribute. I know this is a very trivial thing, but from a quality of life perspective, you know, there's times where I write a test and it's a fact in XUnit and then I'm like, oh yeah, I probably want to parameterize this and then I forget to like I need to change it from fact to theory and like, you know, go to build and like it's not working. Just don't have to think about that anymore with this little thing. It's a quality of life thing. Down here, we see matrix data source as well. I think this is super cool. Um, so we can add in these attributes rate on the parameters. And again, if you're not familiar with what this looks like because for me, I have not yet used this, but it's one of those things I said in the XUnit video. What I used to do and I need to break this habit now that we have matrix data is if I know that I have a set of conditions that I need to sort of multiply out. I end up writing out all of the data combinations and it sucks but I'm like whatever. I do it once and we're good. The problem is that inevitably in the future I'm like, "Oh yeah, there's one more scenario I want to add in, except it's one scenario for an input that I have to multiply out by the other inputs and then I'm manually typing out all of this junk and it feels ridiculous." So I think it's super cool that we have the ability to do this. And if you think about this, because it's going to be a matrix with two parameters, it's basically 3 * 3 in terms of combination. So you'd have nine different tests from this. Hopefully that makes sense. I think that's super cool. This part here, advanced test orchestration. This is something that my mind when I saw this, I I knew that I had scenarios for this, but my brain hasn't yet like adapted to it. So, in other videos that I've made around testing, I talk about doing a lot of unit tests. I don't think that, you know, if you want if you've seen that content before, you've seen me talk about unit testing and stuff online or in other videos. I write code that is very much able to be unit tested and that means that I will use interfaces and mocks and stuff but the point is that I write code to be able to unit test it. It does not mean that I write as many unit tests as I used to. Structuring the code in that way allows me to if I come across a scenario and I go oh that's pretty complicated. I would love to really make sure that I can build some confidence around this and write a unit test. Now I don't have to go rewrite the code. It's just it's already set up in a way where I'm like cool. I got my interface there or I can easily extract the interface now. You know, I might not prematurely do that these days, but I can extract it. Cool. Like I'm still not using a lot of static methods unless it's like just math or something simple. I'm not trying to, you know, do like file.open with like a static file class to be able to go do IO. So I'm avoiding stuff like that to this day. But the reality is that when I want to do integration tests, I have not fully sort of transition to this way of thinking that's on the screen right now. So here's the idea. Generally, when I'm doing an integration test now or functional integration test, however you'd like to call it, I'm exercising more of the system from like a, you know, end to end kind of scenario. That might mean that I'm spinning up a database or something like that. And just to give you an example, I might have something like a repository pattern where I would like to test if I can create something. So I would write my test method. I would call create on the repository. And to validate it, I basically have to call a get on the repository or write a SQL query to pull it back. So I just use the get on the repository. But here's where it's kind of weird. Now when I want to go write a test for the get of the repository, the test almost looks the exact same. I have to go seed the data which means that I'm probably going to go call the same method to go add something and then I have to call get again because these are two different things right I end up either having a test that looks duplicated or I have a test that's combining two things I want to check what we could do instead though is look at how we have these dependencies throughout the test right so this is like a setup so before class I I think that's kind of interesting. We do have some things like this in XUnit to be able to have some pre-est setup which is fine. But when we start looking at here like this is like this test here depends on register user. In my example with the repository I could have a test that's like create the record. I still don't know a good way in my mind how I would go assert it. But the next test if I made the get depend on the create now I would create it and then in a new test I would go get it back out. So I can basically separate out the concerns of what I'm trying to assert on. So I do really like this approach. I need to change my mental model personally for how I'm doing this. But I could see a bunch of different use cases, especially if I have more front-end code where I'm trying to step through things could be great or more complicated workflows in the back end where I would like to do some behavior checking at different stages, but instead of repeating all of the same setup later, I can just kind of continue on the same line of execution. So, I'm I'm really excited to lean into this more. I personally do not have experience leveraging this kind of thing successfully yet. So I'm very interested in looking more. We can see smart test control. So conditional execution. So Windows only, right? More and more these days, especially with .NET being crossplatform, we might have some things where we can only run on Windows or only run on a certain OS or in certain environments. So I think that's really valuable to consider. I see now where the retry stuff is custom retry logic. So retry on HTTP error. I I don't know. Um would love to hear from you in the comments on this. Truly, do you see room for this? Right? Like should your test be able to retry? Why is it failing in the first place? Why do you have tests that are like going to the internet or tests that have some type of thing that is not going to be consistent? I would, and I mean this genuinely. I'm not challenging you. I would love to hear about your real scenario with this or if you don't see value or see that this is kind of risky on the service seems weird to me but would love to hear from you. They talk I mean this is more about the same kind of things. Um they talk about the IDE support we saw the list earlier they have migration so the different guides here right? So they say if you're going from XUnit or MS test they do have migration guides for you which is great. And the last thing I want to note, and we we're not going to analyze them right now, but they do have benchmarks at the bottom of their GitHub page so that you can see sort of how they're measuring performance and stuff like that. In the next video where I go into Visual Studio and we look at some T-Unit tests, I will show you in the Visual Studio Runner. Like I said, unless something's changed between now and right after this video that when I go to run those, I am seeing that XUnit is still 10 times faster, but it could just be the scenarios that I'm running. So, we'll call that out. So, curious to hear from you. Is this enough to make you go explore TUnit? Are you still stuck in XUnit for forever? Or are you still using something like NUnit and you've never really shifted from that? So, love to hear from you. Thanks so much for watching and I'll see you next time.