Unlock the Power of Quartz Job Scheduler in ASP.NET Core Blazor

so this is kind of neat I think that based on what I've seen what I probably will do next is start copying and pasting code this video is another entry into our build series for Blazer and the format of this video series is such that I'm walking you through my thought process for putting an application together from scratch as we build it and try to move forward in the last video we looked at building the foundation for a plugin system that we'd be leveraging for the different platforms that we wanted to integrate into our metrics application we settled on something really simple to begin with but we said in the next step we wanted to start looking at Job scheduling so in this video I'm going to be walking us through a little bit of an experiment getting set up with quartz for.net keep in mind that the format for these types of videos is a lot less cleancut and a lot more experimenting in me playing with code so you can see how I explore things before we jump over to visual studio just a quick reminder to check that pin comment for a link to my free Weekly Newsletter let's go look at some code where we left off was placing some code into program.cs and if I scroll through it pretty quickly you'll see see that the last thing I was talking about was some type of job that we might want to go build but we did say that we wanted to use a scheduling nougat package that's going to be quartz that we're going to look at together and if we scroll down a little bit further you can see that I was talking about building this facade for these social data Fetchers that we're going to have plugins for behind the scenes to go fetch data and I started talking about this idea that when we think about scheduling this type of work either we're going to have one job scheduler that's going to schedule one job at a time that will be responsible for going through all of the data Fetchers through this facade interface and that might be the case but we don't know yet we're just kind of playing around with it or we should be able to schedule individual jobs for each plug-in and if we do that approach then this facade sort of goes away beyond that we're just looking at a couple of dto for different things that we're working with here we started with Twitter and if I scroll down a little bit further you can see the Twitter impl mation of the plugin we started to build I'm going to take just a moment to start moving this code out of program.cs I don't want to totally kill it off yet I think there might be some helpful things that we can start to stitch together especially when we have the job scheduler so what I'm going to do is move these out to separate files I'm going to organize the code a little bit into folders in the solution Explorer here on the left and then we're going to look into quartz all right now that this code is moved out I'll just take a brief moment to explain how I've started to organize code and this is by no means the final way that it is intended to be organized we'll have to see how things evolve but what I've done is I've made a folder for features I mentioned in the beginning of this video series that I'm thinking about trying to organize things in vertical slices so I like having something like a features folder again this might change but one of the features that I want to have is the ability to get Social metrics and my intention here is that within social metrics I want to have different plugins for the different platforms we support Twitter was the first one that we looked at so you can see that I just made a Twitter folder and dropped our Twitter specific things into there and then the rest of the stuff that we looked at that was really for just generalized social metrics Gathering not plug-in specific I just have in the folder above the plugin folder so it's sort of like a at a sibling level to it within social metrics that's the starting point and I've done that literally just so that we could get back to having within our program.cs file because I like to use that as a pumping ground when I'm playing around with things we just have this job class left and this is also very likely going to go away but this is where we got to before we wanted to start talking about quartz and quartz is going to be a little bit different than how we looked at tweet Envy when we were doing Twitter and the reason it's going to be a little bit different is because I have literally no idea where to start with when looking at a job scheduling API with something like tweet Envy I've done interactions with different social media platforms some type of ooth and authentication I have the concepts of posts and things like that followers I understand roughly how that should work but when it comes to quartz I don't necessarily want to explore uh code first because I just don't know what I'm getting myself into I'd rather see if they have some type of documentation and examples so I'm going to start that way versus let me explore the API and code and feel it out so first thing I'm going to do is pull up the quartz documentation and we'll start there okay so I have the quartz documentation pulled up on my screen and honestly the first thing I want to look for is not just the API docs I want to just see a working example in code so looking along the left hand side they have a starting a sample application bit of code here and this is pretty cool it looks like this code right here is just a really simple way to get set up with quartz so let's have a quick read through what this code seems to be doing so we have a factory for auler and get getting a schedu a rate off of that we start it so this async a weight API in in quartz is good to know right we're going to be dealing with a syn8 stuff uh this is just a delay sure and then shutting down oh and it says with a job scheduler because there's going to be background threads and things like that it looks like when we call shutdown uh it seems that we call that so that we're good to go in order for the application to end and yeah so based on what this text is saying right here I'm going to scroll down and get this a little bit more in the field of view we have to wait for that work to finish right so you should manually block exiting of application if you want the scheduler to keep running because otherwise it looks like if we just let this continue on it will kill everything off so looks like that's been a a sort of breaking change as a quartz 3.0 so good to know um logging setup I'm not super interested in this right now um sounds kind of funny I think logging is incredibly important I'm just not interested right now I want to be prototyping this kind of spiking it out they have logging support good to know let's move on uh trying out the application so this is looking better uh so this is showing them running it and the output I'm assuming that's because if I scroll back up they have this logging attached so we probably would not expect to see that if we haven't configured the logging and oh they have a Hello job so hello world right cool this is this looks really similar I'm going to flip back over to visual studio just for a moment but we got pretty lucky it looks like it's a very similar API right so I had an async task called run job async uh it took in a cancellation token theirs is called execute you need to implement an interface I like that to start with that seems really good it's a good fit uh I don't like having to implement abstract classes for me that's a a bit of a code smell I don't think that I need an abstract class to accomplish what I would consider for job scheduling so that's nice but there's this execution context I'm assuming that lets us pass data into our jobs maybe there's a cancellation token and stuff on that so that's good to know and they just have a console right line inside so oh cool so we can see it getting set up all here right so let's read through this so Define the job and tie it with our hello job class so we go build the job and it takes in that typ parameter of the hello job so if I go back to our code if I had like a Twitter job or something right uh it would take in we can give it an identity so a job and a group I'm not really sure what the group's for yet that doesn't seem like it's an issue right this moment so trigger the job to run now and then repeat it every 10 seconds so what we haven't talked about is what frequency makes sense for us to be polling for our metrics right if we think back to our plugins we were mentioning that it could be that we have one job that gets scheduled and it's responsible for all of the plugins or we we have multiple jobs that get scheduled one for each plugin maybe there's a hybrid where they're in groups but what's the Cadence and do we need to have a different Cadence for different plugins so to give you an example do I want to pull my Twitter following every minute right or every hour whatever it happens to be but LinkedIn I want to do once a day do we need that level of granularity I'm not really sure but that's something we should be thinking about in terms of what's uh available to us and how we want to design this right now I don't see a huge need for that but if we can do it easily having the flexibility might be kind of nice so it looks like we can do this is with a simple schedule and we give it an interval and how many times to I'm assuming how many times we can do it also for forever to me this indicates with this kind of syntax we could still go both directions we could trigger one job that will go run across all our social platforms but what that will mean is that it's sort of if I can use like the word coupling how we can figure that schedule to run for all of the social platforms they're all tied together in one job that starts to feel I don't like part of me is like that doesn't feel great we don't really have a you know there's no functional requirement on this but to me it just doesn't feel ideal to couple that stuff together when we could have the flexibil it seems like we could have the flexibility to do each of these jobs per platform pretty easily as long as we just go build a new job and schedule it so maybe maybe we do want to change up our approach a little bit let's keep going and this is piecing it all together we've already seen some of this code right so I kind of skipped over the logging stuff but they have the factory and the scheduler that they start up this is the code we just looked at to go make that job and give it the schedule that it needs and then we schedule it we wait so this will wait for a minute and this is supposed to be every 10 seconds it goes so we should see by the end of this running that we would have that code going multiple times so this is kind of neat I think that based on what I've seen what I probably will do next is start copying and pasting code I'm G to take I'm G to get the Nate package installed and then I'm going to take the code that they have on this page drop it into program.cs and I'm going to see what we can do to get almost like a hello world job set up but before this video is over what I'd like to do is see if we can call into our Twitter like make a Twitter job basically um and if it's not a Twitter job then it's all of the social Med media uh being run so I want to by the end of this video get a feel for if we're going to do one big job for all of them or multiple jobs and see how that fits into the plugin system that's what I'm hoping this kind of Investigation is going to show us at the end so let's dive into it okay so well I'm going to break all the rules of programming even though everyone does this I'm just going to copy and paste all of the code that's on this screen and start massaging it into our program right so I know it's not going to work perfectly like like this but I'm just going to start bringing it in so we have it so I'm going to bring in this main I'm going to set up the logger as well I feel like I might as well and I'll even take the hello job to begin with why not that way we have something to to work and play with here so let me go ahead and put that into Visual Studio I'm going to ditch our job that we made before we're we're about to remake it anyway with the right API so I'll put that in here and I don't have quartz installed so let's go get Quartz by going to manage nougat packages I'm pausing here just so you can see that I'm going to be installing version 3.8 depending on when you're watching this video things might be different so let me go install that now that I have it installed I'm going to get the using statement pulled in then of course this code that I copy pasted in is not going to be quite right because it had a private class here but it also had a main method as well and I'm using uh no top level statement so it doesn't really make sense to have a main method so let's go ahead delete that main method log provider a pulling quartz logging console log provider is inaccessible okay let's it's probably because I pasted in the console log provider and it's private so let's jump to that that's the case I probably will have to make some of these things internal and all that but we'll revisit that is needed I don't need this console right line stuff and I think I probably have an extra parentheses this is a ambiguous warning because I have Microsoft extensions logging and quartz logging so this needs to be quartz logging by the time this code gets moved around and stuff I'm assuming that that's going to go away and really the way that this is done with a console logger probably not code that will exist long term but we'll use it for now and then I just got to get rid of that trailing curly brace and we now have the console log provider that's cool and then this is the job that is the hello job it's just going to write to the console so I think at this point we have everything but of course the way this is configured is I have the whole application starting up beforehand we're probably not in this video going to set things up to be um all ideal in terms of running in background services like really integrating asp.net core blazer with quartz we'll probably need to come back and revisit that put this probably into like a some type of like background service or something that will start all this stuff up we'll have to come revisit that like I said what I'm going to do though is I want to see this run I'm going to not run the application for Blazer we'll let the setup go but what I want to see I want to see that this uh this scheduler runs and I did this in the previous video this concept of like let me just see if I can get their base example working because if I can't even have a a starting point that feels good then I don't know if like I change something that's breaking things or or what I should be be able to have a nice common starting point and start massaging things but one thing that I remembered is I got rid of the console read line I think I will need that just for now or else this stuff here is not going to do what we want so I'm going to drop this back in place for now when I go run this I'm hoping what we see is just some console output that is not asp.net launching a web application in Blazer so we will see what goes on have that run Okay cool so we're getting something already we got the logging that was in there this is supposed to go every 10 seconds we just got the second fire and if I talk for just a little bit longer hopefully I'm not too annoying sorry we should see a third one coming up here and we go there we are so that means that's working and that's cool that's not probably super exciting for a lot of people but I've never used quartz I've never seen it in action so right now a couple of open questions for me are one how do we wire it up properly to asp.net I don't know if we're going to get there in this video cuz I want to research that a little bit more I think and I do want to see us ditch this hello job for something that make sense for either Twitter right that's the one plugin we started with or we start looking it if we make the this call the facade now as I've been talking through this I have been leaning more towards having each sort of uh plugin that we want to do for our social reach program here I'm thinking that I want each one to have its own type of job it just seems to be a better fit but that's going to mean that the plugins that we create each need to provide a job or a factory for a type of job and that's kind of interesting because the API that we started creating needs to adjust and that's okay we've we've only started with one so that's not not bad shape to be and if we had 50 plugins already the maintenance cost of that's going to feel kind of crappy so we're not there yet that's good news but maybe let's think about how the first pass of this I think I want to ditch that facade and think about allowing o the plugins to create their own jobs and I don't even think that I need to make a new implementation for each plug-in let me go back to the code and I'll explain why so I don't think I need to have something like a Twitter job a Facebook job an Instagram job I think that the job I'm going to create is going to be generic but what it calls inside of that is going to be unique now that's what I'm thinking but at the same time that might just be um like premature refactoring of things out so perhaps let's start with making a Twitter job um I might be jumping the gun by trying to come up with the generic one and pass in the logic that we run instead we could just Define it directly that might be better so let's go ahead we'll call it Twitter job and instead of writing to the console we're going to be calling some Twitter stuff so what will be interesting about this is that we need to have the Twitter things we built so if I go back to this plugins folder and pull up the Twitter plugin I mentioned before that we probably don't want to call this Twitter plugin anymore this will be like Twitter data fetcher so we have Twitter data fetcher we need this thing passed in and we did put this interface on it called is social data fetcher that's already seeming like we might not even need it if that's just going to be called within here so what I mean by that is if we were to look at creating this Twitter job if I had um I social data fetcher right like what I want like this is a Twitter job it doesn't want any social data fetcher it wants the Twitter one specifically so it's kind of interesting um depending on how we do dependency containers and dependency injection I could ensure that the only thing on the container is going to be the Twitter one I could do that but what I probably want is really like this Twitter data fetcher and if we have this and we're not using that facade which is over here the facade was the only thing really that needed to have this concept of social data Fetchers so truly this might just be a waste and that's fine we can go delete that right now this is what I'm thinking we want but I don't yet know how we do dependency injection and I don't know that cuz if I go back up here this job builder takes a type in like this these comments are kind of ridiculous now but if it takes in this type I don't know how the dependency injections handled we might have to go back to the docks and see how that works but I'm assuming they have something wired up um if we look at there's no other overrides um description I wanted to see if there's like dependencies uh so yeah we can we can go back and see because I guess first of all I don't even know if we're allowed to do Constructor parameter passing on these so let's go back to the docs and see cuz if we can do this probably all we want to do is something like this kind of thing here and I'm hoping context has the cancellation token and it does so I want something like like this this by the way what from what we talked about before will only fetch data but we're not doing anything with it we need to be able to store that somewhere so we'll have to come back to this still and then we should be talking about does that work belong in this job or do we need to have multiple jobs for reading and writing that might be just be tons of Overkill so let's go see if we can do the dependency injection with these jobs and that means we need to come back to the Court's documentation so I would like to see if they have anything for dependency injection integration so oh okay well this is cool we might as well have a a Peak at this but this is for ason at core integration because that's exactly what we're going to be doing and practical example of the setup I'm just really hoping to see that we can get dependency injection passed in I feel like the answer is probably I feel like it's probably um if it's wired up like this in asp.net so if I search for dependency in here it says there's one but I don't know where it is oh see the courts dep oh there we go I should have uh people watching this you were probably like hey it's on your screen just stop being an idiot and look at it so courts dependency injection documentation so let's see this it says courts comes with two built-in alternatives for jaw factor which can be configured bya either calling use this or this one's deprecated so there's there's one oh they're both deprecated as of Courts 332 all jobs produced by default job Factory are scope jobs you should no longer use these by default courts will resolve jobs type from container okay and if there's no explicit registration courts will use activator utilities to construct it and inject its dependencies via the Constructor so job should have only one public Constructor that's good to know this says we're not talking about the persistent job stores yet but let's have a quick peek here um when using persistent Job Store make sure you have a job trigger names for scheduling okay um so far not so concerned this is for the config sections okay I think we might have enough to be able to go back and I didn't think we were going to be integrating with asp.net core but maybe that's what we need to do in order to get the dependency injection all hooked up so let's go back to this documentation that was going to be about the integration and we can see base courts Schuler when shutting down we want jobs to be completed so I'm going to go take this and I'm going to put it back into this code it's going to look a little different though so so I think we want both of these so quartz dependency injection and generic hosting integration I'm going to start with just this one here and see how far we get there's dependency injection I pasted the code back into here and saw that it wasn't going to compile CU I was trying to add it onto here there's no quartz directly on that um so this is for maybe something a little bit different I thought it's a service collection but these these two things don't show up and that's because I pulled the wrong thing I need quartz asp.net core so let me go back and do that see this is why it's important to read the documentation I install that I think we should be in good shape so let's find out this might need another using statement pulled in fingers crossed we get that all to work but I think we want something like this interesting this add quartz one doesn't seem to be here but I might have misunderstood what's supposed to happen here so you can add quarts by invoking ESS method add quartz method sorry add quartz server on this this will add a hosted quartz server into it so maybe we don't need this but we want this yeah okay this is commented saying asp.net core hosting so I think we can ditch the other one uh and then this doesn't need to exist um but it's not happy and it looks like it's not happy because I'm doing this stuff out of order so I see I have this in the wrong spot so let's put this here but also this okay peculiar but uh I did didn't realize and this is because I'm new to Blazer I didn't realize that once I started doing this part here the return type is a razor components Builder and up here it's still a service collection so we need to make sure we're doing all this stuff first before we get into this part of this Builder pattern so that's interesting now I want to go back down here and see that we probably want to move some of this stuff around means we probably don't need this because we will be running the jobs for the lifetime of the application so that's cool we don't need this dummy delay anymore but we might want to move just to test this stuff out we'll pull this and this Factory and this log provider so literally all of it right um we want our app run back in but I have that stuff on my clipboard I think I want to put that stuff into here but let me see based on the documentation they have this Q parameter what's q q in this case I called it something different I called it C oh they call it Q for quartz I guess um but they're doing quartz and adding job okay interesting so uh they have a job key add job and they're using this so the syntax is a little different but I think we can still do this so I'm going to pull this out and put dump it into a comment just so we can play around with it the logging is interesting so I need to see where that's going to get configured I don't think we need to do this scheduler in the way that we have it but this is what they had in theirs right so let me drop this into a big comment just so we have it to refer back to and I'm not going back and forth to that page but it looks like they want a job key um I'm going to call it Twitter what I I'm trying to work through here is we're going to make it work for one which is going to be Twitter the only one we have and then we can look at how we would want to extend this and pull this information off of the plugin so add job we have a Twitter job and Twitter job we're going to run with the identity the job key that we have here I Chang this to be quartz CU I don't like single letter variables when I don't need them and the trigger is going to be different so we'll have this trigger and it wants it for the job key so that's cool with an identity call it Twitter job and I guess naming up here is supposed to have job in it as well and we have a trigger on it and then a cron schedule so this is supposed to be every minute but can we do with simple schedule cool let's let's just take this code right and put it here boom that looks good okay like that now my missing there we go added a job added a trigger and then do we just not need this stuff anymore that's pretty cool okay interesting but couple of things here one is that I changed the Twitter job I'm going to jump down to it it takes in this Twitter data fetcher we haven't wired anything up yet for dependency injection at this point so that's one part what I might do is put this back just for a moment and I want to see cuz I don't want to change too many things at once right I'm going to put this back so we can you can see console right lines coming in um so we'll put console right line I like Hello World why not put that and I'm not making it async I don't really care right this moment so we'll do this Twitter job what this is not validating is the dependency injection part I do want to come back to that but it seemed to be a better fit to switch over to see if we can just wire this up with asp.net core first then we can talk about the dependency injection so if I go run this hopefully it doesn't all explode and we can see print to the console that we have something going on so there's the app that's nice the console here hello okay cool we already have two hello worlds it's every 10 seconds so we should see another one roll in any moment now please there we go a third one great so this does work which is awesome I don't have the logging configured so I'll have to come back and revisit that but I did say earlier I don't care right now so I'll leave it commented we can come back I'm going to put a to-do in here and I like using to-dos uh my own person personal projects I have to talk with a team before I do stuff like this in other places but I I'll leave that in there for now and I think it would be good to make sure that we can get the data fetcher and stuff wired up and we can see if this is feasible what we might not do cuz this video is already getting super long is figure out and implement the API for pulling off the plug-in stuff here but I want to talk about it so let's just prove we can get the dependency injection working then from there we'll have a quick chat about the API that we might want to tweak to get our plugins working with this so this is a little bit embarrassing but I just spent literally an hour trying to work to get the dependency injection stuff put together I trimmed it all out because it was just going off the rails but I finally figured out what was going on so what I want to do is briefly talk about just this part here for the dependency injection for the Twitter data fetcher and then what I'm going to do is wrap up this video and we'll talk about some of the API considerations we might want to have about creating jobs out of plugins so it turns out that there's a really big problem with primary Constructors either with quartz with the net dependency injection framework or the combination of the two because this just does not work no matter what I do with this setup I can't seem to get this Twitter data fetcher reference passed in through dependency injection however if I switch this up a little bit and get rid of the primary Constructor everything seems to start working so I'm going to go ahead and do that I just wanted to call it out because I spent a lot of time trying to figure it out and for some reason this is really what makes the big difference if you're watching this and you're familiar with this problem and you know how to work around it please feel free to let me know this is essentially what we have to do and I just ran this to prove it to you but you can see when I use the hover over of the Twitter data fetcher on the instance variable it's populated with a Twitter data fetcher so we do have that working that's all good and all that I had to do was wire up the Twitter data fetcher right onto the dependency injection setup lines right at the beginning of the application along with the config file or just it's a config dto which later will load from a file and pass that in so those two things the config for Twitter and the Twitter data fetcher are registered on the DI framework and then once we get rid of the primary Constructor it all just works all right so the last little bit that I want to spend a couple of minutes on is about how we can try getting these jobs out of the plug-in API that we have so what we had done previously was built a facade that would get the data Fetchers but instead of being able to get data Fetchers out of that I think we want to change things up a little bit and one way that we can do this and it's a little bit premature because I'd like to see a couple of these done manually wired up before we dynamically start to load them but if I highlight this code and we start looking at what is Twitter specific and what things are parameters we could tweak it starts to give us a little bit of an idea for what we want to provide out of the plugin so you can see that we have something like a job key that needs to come out so some type of unique name we need to know the type that we want to have and that's because we need to load this up through dependency injection and do add job right on this quartz API right here so we need to be able to do that the trigger we could probably take this name right that comes out uh that we build the job key for and just append a dash trigger that's probably good enough for us and then it looks like we probably want to have some type of interval that comes off and I'm curious with interval we can use a time span so that might be interesting just to provide a time span off of that so the way that I would think about building something like this instead of going through the facade that we talked about before is thinking about those different pieces I just mentioned right so uh this is not going to compile if I add it right here I just want to type it on the screen so we have it and I'll scroll a little bit more into view but I would do something like an interface that we'd want to have but it might be something like I social job provider something like that U not really great with names and sometimes it takes me a little bit to come up with something but I think starting with an interface like that we would need some type of uh job identifier so maybe to start we have a string that's a job ID and interestingly enough a job ID might kind of make it seem like it's a single instance of the job right so maybe like a job type ID some type of unique identifier for the type of job and that would help us with this part right here for the job key as well as as the identity for the trigger because we can just append a dash trigger like I was saying we also need the type right so we probably could do something like expose a type right on here this is kind of interesting because I already am not a huge fan of this and I'll explain why but but the idea is that I'm thinking I don't want to have the responsibility of each of these plugins to be able to create the job and the reason for that is that I would think that I might want to extract this and put it in a common spot and then I can just call it from one spot and populate it based on plug-in data but the reason I want to do that is cuz I'd like to keep the plugin sort of agnostic to this concept of quartz but it makes it kind of tricky the way we have it set up already because that type that would need to come back is not really lending itself well to uh being agnostic to quartz because it has to implement I job that comes from quartz based on that it means that our plug has to implement or depend on quartz not a fan of that so far so we probably want to change that at some point but I'm just kind of thinking through this out loud right doing this kind of exercise right now is certainly premature um so we get the job key we have that from here we can make the identity for the trigger based on the job key we want some type of interval so we could do some type of interval like this and just have a time span and really I think that's mostly it that we need but like I said said the big problem right now is that if we want to have this type that comes off of here and that way we can end up passing it into the the other method that isn't a type parameter it takes in a type object we could create all of this and we could Implement that for Twitter but I'm going to stop that for now because I think that the solution that I might want to pursue for that means that we'd have to go create some type of wrapper class and then go back to this idea of some generalized job so within our core application that we have our Blazer app it has the concept of working with quartz I don't want to push that dependency out to the plugins if I don't have to it feels a little bit odd but if I make a generic job inside of the core application I can create my own API that I allow the plugins to implement and that way I can take whatever that is from the plugin and put that rate into our job it's really generic and I can create mult multiple instances of that one for each plugin that we want to do so I think that's probably the direction I want to head in but I think I'll save that for next time so hopefully you found this interesting I know it was a lot to go over and a lot of jumping around and if you're familiar with quartz this might have felt like a little bit of a waste of time but if you're not then at least we got to see how we can start to integrate a job scheduler like quartz into Blazer and then we started talking about how the plugins might fit into that so this is getting kind of exciting for me so thanks and I'll see you next time