LINQ in C# - Being Lazy with Mapping, Reducing, and Filtering

if you've been programming in c there's no doubt that you've heard about link but a lot of people misunderstand link and what it does link can greatly simplify a lot of really complex things that we would otherwise encounter when going over Collections and like they say with great power comes great responsibility so I think that it's really important you understand how link works and in this video I'm going to explain what it does and how you can write your own link methods as well before we head over to the code just a quick reminder to check that pinned comment for a link to my free Weekly Newsletter and courses that I'm working on all right let's jump over to visual studio so link stands for language integrated query and we're going to be looking at link operating on I enumerables now there's three different things that I want to call out that link can help us with and that's mapping things filtering them and reducing them from a collection down to one thing and we're going to go through examples of all of these before we see how we can write our own link methods and along the way I'm going to demonstrate something that's a little bit tricky with link so you're going to want to stay tuned to see how that works the first example that I want to walk through is mapping things and that means that we're going to be transforming different elements that are inside of a collection in this example I'm just using a list of integers and it's called numbers we can see that on line 11 here so this is a new list that just has five integers in it from 1 to five and it doesn't have to be a list it could be an array or any collection in C because collections Implement I inumerable so that's the minimum requirement that we need here as long as whatever you're dealing with imple ments I inumerable you can use Link on it so to see what this would look like without link I have an example from 13 to 17 here that shows creating a new list and because we're mapping things what I want to show you is going from a list of integers to a list of strings and what that would require us to do is have this new list that I declare on line 13 called numbers as strings and what we're going to do is go with this for each Loop where we take each number from numbers and then we call two string on it right here on line 16 and for each one of those we're going to add it into the list of strings that we've just created so at the end of running this Loop what we'd have inside of numbers as strings is 1 to five but as strings instead so this is how much code you would need to go write to be able to do this or what you could do is use the select link method which I have here shown from line 20 to 22 and what I've done as well is split out this code across multiple lines because to me this is more readable instead of having one long line of code which you can have happen pretty easily when you're chaining some link things together I like stacking it vertically and this is a pro tip because when you're working in code bases that have really wide lines of code especially if you have to use a scroll bar to go left to right it can make things really difficult to read and if you don't believe me spend some more time writing code and try seeing the difference when you have to scan really long lines left or right in many lines of code so what we have on line 20 is we take numbers and then we call the select method and this is a link method that we have if you haven't seen this Arrow syntax or it seems a little bit confusing what we're doing is saying that this is going to be a variable called number and this is just like when we have this for each Loop where we say int number it's the same concept we're just giving a name to the element that we're using on each iteration of essentially a four each Loop and then like you can see here I have numbered to string which is just like this part as well but the difference is that when I'm calling this select method I'm not putting it into a list like we have on line 16 here I'm not doing this part we don't see any of that here but what I do to put it into a list is at the end after we call Select I say to list and a little bit later in this video I'm going to explain why we have the dot select and still need to do a two list as well but to briefly mention doing do select here just gives us another I inumerable and doing two list is what's going to put it into a list okay so now that we've seen mapping things with list link which is essentially going from one value to another so we're just converting each element in the collection to something else filtering is instead going to be removing things so we just want to focus on keeping a certain set or excluding things that we're not interested in so to see what this could look like without link I have an example from line 26 to 33 here and what we're going to be doing is looking at taking that numbers list that we just had and we're going to get the even numbers only so what you can see on line 29 inside of this 4 each Loop so we still use a 4 each Loop just like before we're doing number modulo 2 and checking to see when the remainder is equal to zero this is a really simple way that we have to check for even numbers so every time we have this condition met we're going to add the number that we're checking into this even numbers list so much like before we're still going from one list that we're iterating over to another list that we're going to add new things into but as you might imagine because this is not going to meet every single number that we're iterating rating over it's only going to take the even ones which means the final collection that we have is going to be potentially fewer numbers and because we know its numbers 1 through 5 we know it's going to exclude things like 1 3 and five but again this much code is needed to do it without link now if we see on line 36 through 38 we have the same idea that we saw before with link but instead of Select we're using where so where is what we use to be able to filter things and we have the same Arrow syntax so we're just calling the element for each iteration number and then inside here we're saying when the number modulo 2 is equal to zero so this part that I have highlighted right here on 37 is equivalent to what we would have to do with this if statement and again we have to have A2 list there because just doing this where is going to do the filtering but it's only going to have it in an i inumerable we still need to put it into a collection and again if you wait a little bit longer I'll explain why this is so important but the final thing that I want to look at before we explain that little detail is reducing things so we've seen mapping we've seen filtering but reducing is going to take that collection and reduce it down to a single thing so if we wanted to get the sum over this collection of numbers that we have and you could do things like average or Min or Max uh but sum is just the example that I picked in this case again we're going to have a for each Loop but what's interesting just to kind of call it out early is that instead of having another list that we're putting things into I only have one thing it's just the sum it's not the list of even numbers and it's not the list of converted numbers to Strings it's just one element and like you can see I had this for each Loop so again very much like all the other examples and then each iteration of the loop I'm just adding the number to the sum pretty straightforward and if we look how we would do that with link on line 48 it's just numbers and then we have a sum link method so extremely simple there isn't a Min there is an average as well so you could do Min here you could do average um you have access to all that stuff which is great so you don't have to go write your own methods like this so it's really handy but what you should notice is that just like I mentioned how we don't have a list here on line 41 I don't have to call2 list after line 48 when we do the sum and that's because the sum the result of trying to calculate this is just a single number and that's what we mean when we say reduce we're going from a collection down to one thing okay so what's the big deal with doing the2 list thing well let's go into that right now so link methods are lazy which sounds kind of funny so what does it mean to say that link methods are lazy so when you're just doing do select or do wear and we don't call the two list nothing actually happens which sounds really funny right like what do you mean nothing happens well what I mean is that we're not technically stepping over all of that stuff and I wrote this little bit of code here because I want to show you that and prove it to you so this example is going to use a select here and you'll notice I don't have a two list after line 62 right there's just a semicolon there's no two list like the other examples we saw but what's unique about this one is that every time we're about to go return something as we're selecting it so this is just converting the number to string I'm going to print this to the console and what I want to show you is that this code is going to run we'll step over it and we'll see this line that says after link line for lazy numbers of strings but we're not going to see this get printed to the console and we're not going to see this stuff get printed to the console until after this line gets written and that's because on line 66 I'm going to be forcing the enumeration by calling to array which is just like like saying two list but instead of a list I'm getting an array so two array two list doing a for each Loop over it that's going to force this part to execute and I want to prove it to you so let's go run this okay and just like I said we see after the link line for lazy numbers is strings right at the top here followed by every iteration that says transforming so that's this line 63 running before all the stuff you see above it and then we see after forcing numeration lazy numbers a string which is printed right here and just to reiterate why this happens is that until we force a numeration so two array two list four each until we do something like that this select method it's the same thing with wear they don't run any code it's really just a function pointer so we're setting this up but nothing's evaluating it yet what's different about sum or average are the other ones that do the reduction is that those will automatically go execute and that's because they only give you one thing back those are not lazy but doing the ones that I just mentioned will be lazy until you go materialize them or iterate over them now this means that there's yet another catch with using link and this one is super important because I see people making mistakes with this all of the time regardless of their experience level because it's just something that's really easy to forget or Mis interpret or otherwise just plain mess up so because these things are lazy if you don't put this into a list or an array or otherwise materialize it and store it into something every time you go to access this to iterate over it or call something like two array or two list on it every time you do that you pay whatever penalty your link method is going to incur so what I'm doing in this example is every time we go to do an iteration ation of this select which is just like the other one we saw I'm still printing it out to the console I'm going to wait a full second here and if we look a little bit below what I want to illustrate is that I have two for each loops and each one is going to go over this I inumerable that's resulting from do select because remember I'm not calling two list here or to array so when I go to do this each time here and here we're going to go materialize this fully wait 5 seconds each time and we'll see it print out to the console so let's go run it and see as you can see it's printing out to the console here and finally there we go all done and that took 5 Seconds * 2 it took that long to go run them all now what will be different is if I call to array on it just once so instead inad of just doing this part I'm going to say2 array here what this will do is force this to be an array assigned into here and that way we pay the penalty just one time you'll see that the order is going to be different because when we call to array it's going to force this enumeration to happen so we'll see all of this stuff print before this part and this part but these two for each Loops are now going to go basically in instantaneously because we're paying the full performance penalty up here by calling to array let's check it out and I'm just going to add a console read line above this so that I can press enter and time this properly for us all right I'm going to press enter and we'll see that we have this part in the select go first followed by these other two parts basically instantly right so this is a very different performance profile than what we saw before so the difference here is that we pay the performance penalty one time but that does mean that we need to store this entire collection which it's only five numbers we do have to have a whole copy of that stored into an array so you do need to be careful that if you're dealing with tons and tons of elements if you had lists of hundreds of thousands or millions of Records you're going to be making an entire copy of that so you do need to consider your memory footprint versus the performance penalty of being able to go iterate or materialize these things and the last example that we're going to look at here is just making our own link method so what you need to be able to do to have your own link method is we need to have an extension method that operates on I enumerables and to do that we end up having a static class so I have a public static class just called my link this name doesn't really matter it just has to be unique because we're not going to call this part that says my link anywhere if you look here I'm call calling my link method but because it's an extension method we never have to write this out fully now the next part to get your extension method working is that you need to have a static right and then you also need to use this and the way that this is going to work is that we have the name of the link method or the extension method we want to create and it's going to operate on an i inumerable of type T and generally with link we use generics because we don't really care what's inside of the collection but there's nothing that says that you can't go you know write your own link method that only works on I enumerables of integers or I enumerables of strings you can totally do that but I'm just writing a very generic one and to explain it a little bit further this is going to be an extension method that takes in a type of T and it's going to return an i inumerable of type t as well so the input type of the I inumerable is going to be the same type as the output for the I inumerable and the other part that's interesting here and you don't have to use a fun like this which is going to be uh basically a delegate or a method that we can pass in that's how we get this kind of syntax here but this is going to be the input type so again I'm passing in an i inumerable of type T that means that the funk that I want to work with here is going to take an input of type T and this second T is the output of the function I'm going to run again you don't have to do this you could go write a for each Loop or something inside here that you know only returns strings for example Le and you don't have to pass in your own selector or function like this but this is just illustrating that you can be a little bit more generic with what you're doing but the real big takeaway here is that you want to have yield return combined with your I enumerable and that's going to mean that you are making what's called an iterator so these pieces are what give you that lazy behavior and that's the whole idea behind the link stuff is that when you have an iterator you don't force the evaluation until we have something like two array two list or calling a four each Loop so what Nick's fancy link method does is really just iterates over each one of these things it's going to write to the console and then it's going to call this selector so whatever function we pass in here and in this case I just made it multiply the number by two that's going to run this and return the numbers so what this technically does up here is it's going to take the numbers 1 to five it's going to say for each of the numbers from 1 to 5 multiply them by two right and what's going to happen is that the console right line will go for each one of them before it's multiplied so we'll see to the console 1 2 3 4 5 but the end result is going to be that we have an array that has 2 4 6 8 and 10 and to prove it I'll put another console right line in a for each loop after this so what we should see when we go run this is the applying select called from 1 to 5 and then we have the numbers like I said doubled printed right after let's go check it out and as we can see we have the applying selector run first for 1 to 5 and then that for each Loop that I wrote after from lines 101 to 104 that's going to print out 2 to 10 because it's all of the numbers above doubled I just wanted to put this back on the screen for you because this is a few different things going on here and a lot of this syntax can look very confusing if you've never seen it for the first time so just to reiterate a lot of the different pieces here if you want to have your own link method you need to make what's called an extension method that requires a static class it requires a static method on that static class what you're going to need to do is have an i enumerable and it does not have to be of type T you can force it to be a particular type but you need an i enumerable you will need this because this keyword is what makes it an extension method and that allows us to extend like we see some array some I nerable that we can add this method onto and then the other part like this here with this funk you don't need that but you do need to have something that is for each with a yield return combined with an i inumerable because those pieces are what give you an iterator and allow you to yield back one at a time the different elements that you want to return and that is Link in a nutshell so we got to see mapping reducing and filtering with link we got to see how we would do that without link followed by with link and we can see that it simplifies the code at least in terms of number of lines I also went over some gchas with that because if you're not careful about how you use Link and I enumerables you can run into some interesting challenges and the last part we looked at was making your own link methods and I highly recommend you go back pause the video have a look at the different syntax and the structure of the extension method that we had to create because it is a little bit complicated and if you want more information on that just search making extension methods in C and compare that to what I put on the screen because you'll start to notice the pattern for how you can make an extension method for anything and what you would need to do for an i enumerable to get something that feels like link if you found this useful and want to see more about iterators you can check out this video next thanks and we'll see you next time