Refactoring Social Media Assistant Code in .NET C#

what I have up on my screen right now is Jamal's code that he had pushed up and when we're going through a prototyping phase like we were doing what we're actually not really focused on in my opinion is just trying to structure the code in a really elegant way we're actually just trying to get things working we're exploring and to be totally honest when I was trying to drive Jamal in a particular direct Direction you know I had some some poor ideas that I was proposing we were getting stuck and when we finally got around to trying out selenium it was a lot less like you know hitting walls and getting stuck and so I think that was a great suggestion and as I was guiding Jamal through this he was able to try out some things that were I think a little bit more advanced that we were planning for but really what we have is kind of all of the code dumped into one file so that's the first thing to call out that I'm going to be showing you how I cleaned up um I'm anticipating that we're probably going to want to have some other scrapers as we're doing this and as a result of that I wanted to be able to kind of refactor out some of the Chrome setup for selenium here I did actually figure out what was wrong with this um and I should have caught it before but the Chrome options that he's created never get passed into here so I've corrected that in mind we'll demonstrate that in just a moment and another thing that's a pretty big change in mind which I will show you is that this method that we created wait for page Source contains that's actually something that I feel like if we're going to have other selenium-based scrapers we're going to want to reuse this and so we're not copying and pasting it everywhere I pulled it out into another library and I made it an extension method so that way when we're using it it feels a little bit more natural like it's built into the web driver so that was one more thing and the final part is that this code here which actually does the um sort of parsing if you will of the content of the HTML um I actually said this worked pretty well and let's kind of bundle it up into a dedicated Tick Tock plugin and furthermore while we're getting the followers what other information is pretty easy for us to get so I made a record that will have the profile name the URL the follower count and the following count just because it's available and why not kind of demonstrate that so let's jump over to visual studio and I can kind of walk you through those changes that I've implemented so I'll minimize this sorry that this was you know really bright white background if I jump over to here this should hopefully be a little bit easier on the eyes I'll zoom in a little bit more try not to cut off anything here but this is the same program.cs file that Jamal was working on and you can see that it's only a couple of lines long now it's not because I deleted all of his code I actually added more code I've just reorganized it so what we're doing is we're going to make a tick tock profile fetcher that's where I moved a lot of our code and then we're going to ask it to fetch the profile for Dev leader which is my what I use for my social media accounts and this profile info that comes back if you can see I know it's probably a really small font there but the the tool tip is showing that it is a profile info type so that's the data structure I created then we just print that to the console so clean this code up so that the program looks really lightweight let's start with um the Chrome sort of reusable code for selenium that I wanted to do so if I jump over to this file here in this other new project that I've added this is why I had Jamal create a solution that we could put multiple projects into and kind of name it social media assistant and then all of the sub projects below but to start you can see that I added headless and I actually reduced some of the logging it's very noisy in the console and would love it if it didn't log anything this seems to help and then you'll see that I pass in the Chrome options here so that actually makes the Headless part work which we didn't have in our last video so we can call this it's a factory pattern just so we can create an a selenium Webdriver so this is nice and reusable it's only used in one spot right now but I very much anticipate that we will be reusing this next up I have this iwebdriver extension method class I've put it right into this open QA selenium namespace though and that way anytime we're going to use iwebdriver this will make it feel like it's automatically accessible to us and kind of built into iweb driver without really much other tweaking this code itself didn't really change at all except for I renamed the result type and then I added just uh for I felt like it might be useful to actually provide in the result the text that we were looking to find not really necessary just felt like that might be a quality of life thing that will come up later so otherwise this didn't change and you can see that over here in this other file I actually have um you know it's the same record that we had before and like I mentioned I added this property as well okay so where did the rest of the code go well if I jump over to this other class here Tick Tock selenium profile fetcher this is organized a little bit different and I'm trying to make sure it's not all cut off for you here but if you recall we had one regular expression that we were getting the follower count I've just moved that up to be static um I don't think it's going to be necessary based on how we run this for now but um it I like kind of pulling out constant things like this that exist for the lifetime of an instance but also the lifetime of the entire type itself is static read only I can't make a constant here because a regex is not something that can be constant in my opinion this is the next best thing we could make it lazy or something like that kind of Overkill at this point but you'll see that basically between these two regular Expressions followers following so we're going to be pulling both those pieces of information next up I'm going to be using that Chrome web driver Factory that I talked about so all of that code that was pulled out into the factory we're going to use it and I just left a note here for the future that I'm going to be trying to show Jamal how we can use Auto fact to instantiate some of our classes and dependencies so generally I advise against this type of pattern where you basically create your dependencies with a concrete type directly in your Constructor my opinion when it comes to testability and stuff like that this is a big No-No but I wanted to leave this for later so that we can have an exercise to go clean that up if I go refactor it now it's basically zero benefit so we'll do it later next up um I created this method to fetch a sink right now it's not actually asynchronous at all this will be blocking but um I'm thinking a little bit ahead when we go to make these plugins I'm thinking that this might be a nice API that we can have on an interface and all of the plugins can follow a scheme something like this but we'll see as that evolves I added this little check here um when we're talking about social media handles some different social media sites actually um it's common to prepend the at sign and I figured like if someone has provided it into here let's strip that off for now and we'll basically force that we inserted into this URL here that way we don't accidentally have a double at sign or something like that um you can see that we're using the Webdriver Factory to actually create that web driver so that's great that hasn't changed this forces the navigation to the URL also hasn't changed and really the last little bit that's different here is that you can see I've kind of doubled up some of the code it looks doubled right because I'm doing follower and following count and actually I've messed this up which is great that I'm looking at this now um actually no no this is accurate so just to explain for a second um if you're looking at the structure of the HTML that comes back we're going to be looking for following count because once we find that not only is the number for your following right there but your follower count is directly after so you can see it looks a little bit misleading it actually tripped me up here we need this result in both looking for a following count and a follower count because it's just the position where this information starts and then we can look for each piece of information independently which is exactly what this does so we're saying from that spot we're going to look about 200 characters it's more than enough and for the following or sorry the follower count we'll look a little bit further also 300 characters is sufficient there and then the last part is basically doubling up this parsing where we're trying to get the strings converted to integers and last but not least we just bundle all of that up into a nice little record so really um didn't change a whole lot I just kind of refactored where the code is placed and then added following um as a as an addition so I'm going to go ahead and run this for us foreign you'll see that I don't have well you have to trust that it's not coming up on another monitor for me but I don't have Chrome launching anywhere you can see that this is already completed if I didn't have the the logging settings there this would have filled up with a lot of logs and stuff it's pretty noisy but the important piece of information I will highlight right here and let me zoom in just so it's a little bit more obvious but this right here you can see that this is the struct uh pardon me the record that I created has the profile name is devleader profile URL number of followers a number of following and it's great to see that as we've been recording this number is slowly climbing so that's awesome but there you have it just a small tweak and a little bit of refactor to some of the code that Jamal wrote for us so thanks for watching this video uh I hope you enjoyed seeing like a little bit of refactoring steps that I was taking there hopefully the explanations made sense if you thought this video was interesting please give it a thumbs up leave a comment below if you have any thoughts or questions on sort of the direction that we're heading in and of course subscribe if you want to see the rest of the journey for the social media assistant or other programming related content thanks and we'll see you next time