2 EASIEST Ways to Run C# Benchmarks with BenchmarkDotNet

as software Engineers something that we focus on is being able to write high performance code and if you're writing in C we have access to a tool called Benchmark that's absolutely awesome for standardizing how we set up benchmarks and the output that we get with them but what are the different ways that we can go run benchmarks using benchmark.us and I'm a principal software engineering manager at Microsoft in this video I'm going to walk through two different ways that we can go run the benchmarks that we've created in benchmark.us and my courses on dome drain and with that said let's jump over to the code okay the first thing that I want to mention is that I'm creating a whole dedicated project here just to have the Benchmark that we're going to use in the two examples this isn't necessarily something that would be standard but I just wanted to reuse it so we can see the same Benchmark show up in both spots so what I've done is I've just created this benchmarks class it's marked with short run job and then I have this single method for benchmarking that has the Benchmark attribute and all that we're going to do is generate a lot of random numbers The Benchmark itself is not important I just wanted to show you where it's coming from as we go through the examples so the first one that we're going to look at is The Benchmark Runner and the way that this works is that we're able to run this from the command line or visual studio and what will happen is that it's just going to run all of the benchmarks that we have in this assembly here in this case because I mentioned that I was going to be working with a benchmark class from another assembly what you can see is that I'm referencing this entire class and then asking for its assembly and there's a handful of different ways that you could go look up the assembly so you could go scanning files or if you know the list of them already you could go create that list and have that added in here so I'm not going to tell you how to go find all of the assemblies because I think that's up for you to decide and if you're trying to figure out how you would go use multiple assemblies with this Benchmark Runner with the run method really what you want to be able to do is do multiple assemblies and then look for the types specifically in there that are your benchmark classes and that's because the other overload that we have access to here instead of taking in one single assembly there isn't one for just many assemblies what you need to be able to do is get the types off of the assemblies and then you can pass in the type collection in as this first parameter the other cool thing about this is that you can take your arguments from the command line and because I'm using top level statements here this args is really just built in so we don't have a program class and a main method explicitly defined here but the args come in from that implicitly so we can pass those args directly into the Benchmark Runner so if you need to spef spe ify things on the command line you get that for free but what's important to note about this is that when we go run this it will just start running the benchmarks there's no user input nothing else to do it just starts them off so before I go run this a quick note that what you're going to want to do is make sure that you're in release mode and you're going to want to run these ideally without the debugger attached will it work with the debugger attached yes but if you're trying to Benchmark things properly and you're doing it from Visual Studio it's more ideal to have the debugger detached so that you get more accurate results so the way that you can do that instead of just going up and pressing run because technically that's going to debug is what I like to do right click on the project here this is going to be a little bit misleading but if you go to the debug menu even though we don't want to debug right you can start without debugging to me this is a weird spot for a menu but that's what I would do in order to start this off and not have the debugger attached and as I pointed out I am in release mode when we go to do this it should put up a console that just starts running the Benchmark that I showed you in the other project and as you can see I'm not going to wait for this whole Benchmark to go run but it just launched and now it's already benchmarking things and I don't have to do any other user input I didn't have to confirm anything it just runs and actually that was super quick I blabbed not for even that long and it finished so you can see that it is running that method this is a test Benchmark that I showed you at the beginning of this video so let's close this off and jump over to the other way that we can do this which isn't going to directly kick them off off but wait for some user input so instead of The Benchmark Runner we have Benchmark switcher and it's going to be a very similar setup we can start working with an assembly we can pass in the args right from the command line but one fundamental difference here is that it's not just going to start them all it's going to give you some user input to be able to select the different benchmarks you want to go run in this particular case I only have that one single Benchmark so it's not going to look that creative when I go to run this but I just wanted to call out that that's the major difference here we do have a slightly different API to work with so not only does this one have from assembly but it has from assemblies it has from types so those are some different variations compared to what we saw before with the Benchmark Runner so a little bit of flexibility and then the other thing to mention is not only do we have a run but we have a run all which I think is just going to make it look like The Benchmark Runner we saw and then a run all joined which is going to put all of the Benchmark information into one single summary at the end instead of having it split out across so if you're using it like this with the run and I pass in the args this is going to basically do what we saw before except wait for some user input to be able to pick which benchmarks we want to run let's go try this one out again I'm going to go to the debug menu by right clicking on the project and saying don't debug it so start without debugging still in release mode and this one did not start running benchmarks right away but it does say we have available benchmarks and it's not listing out the Benchmark methods it's listing out The Benchmark classes so I wasn't very creative with my naming the class that I used I just called it benchmarks so it's not very obvious that that's the class name that I picked but if I go put in zero here because there's one to select from and if you read the instructions you can pick multiple benchmarks you can also do some it says console arguments with filtering as well so if you wanted to start this and pass in those arguments like I showed you in the code which is right behind here right so using these args here on line 10 coming from the command line you can get this all kicked off kind of skip that user input but if I enter zero here and then press enter it's essentially going to go do exactly what we saw before and if I wait for a couple more seconds it probably will finish off the benchmarks because we saw that that was pretty fast in the first example and there we go this is the second flavor where you're able to kind of interject and say Don't run them all I want to be able to select which ones to pick and those are two different ways that we can go run our benchmarks in basically just kicking off all of the benchmarks automatically and the second way was having some user input to be able to filter which benchmarks we want to run if you'd like some more information on using benchmark.us and I'll see you next time