NoesisGUI + Unity3D + Blend: A Crash Course!

hey welcome back to the channel um i had a request recently from a user called the electric spot about uh gnosis gui and how you can get uh set up and running with that so i figured it's uh pretty straightforward it's a great opportunity to make a video and they have some really good documentation on how to do it so i figured we'd just kind of walk through it together i haven't rehearsed this or anything so i feel like it's a you know kind of show you naturally all the little stumbling points i've only done this a couple of times so there's probably a few hiccups we'll hit along the way but they have as i mentioned really good documentation uh so we should be able to get this through this together pretty easily and yeah let's let's begin so um first thing uh jump over to their docs for how to get started and i'll have a link to this page in the description um but this definitely helped me get going and it's super easy to do so what we'll do is start by getting a new project made in unity i'm currently using i guess it is showing 2021.1.0 f1 and i'm going to make a 2d project in unity because my games are 2d based you can get away with 3d or kind of do whatever you want to do i guess but for this video we're going to make a 2d one i'm just going to call it new project or a new unity project i'm going to put it in my oh cool we'll call it um gnosis test and i'm just going to put in this location and create a 2d project last time i tried to make a project it seemed to take quite a while from unity so i'll figure to kick this off let it get going and then kind of talk about some of the steps we're going to have to take to get gnosis set up and running so uh first thing you're going to want to do is click this download link and download this gnosis gui unity package they do explain with some great pictures and stuff like how you're going to import that the times i've tried this it's worked perfectly there's never been hiccups at this stage so you're going to want to download this and once we have our project set up here and as i mentioned it's going to take i think it took about three to four minutes last time i made one which is kind of kind of crappy but we'll kind of go through that i wanted to be able to show you exactly what i'm doing uh so that it's there's no magic steps that are missing that you have to guess at what to do but anyway once this is open we're going to like as a first step basically go to this assets menu import packages and uh go to custom package menu and then point it at the unity package for gnosis gui so make sure you've started downloading this pause the video kind of get that going i already have it downloaded from before so i don't have to to wait for that just gotta wait for this unity project to get created but there's not too much more to do after that so by adding this custom package i'm going to leave the samples and stuff on by default when we walk through this but if you're kind of doing this for your own project it definitely doesn't make sense for you to unless you're just kind of fooling around to start if you're if you have your game kind of up and running you don't want to include the samples for gnosis directly into your project that doesn't really make sense because you're not going to want to ship or deliver any type of game that has these samples in it so just something to think about you probably don't need to have these samples pulled down but i do recommend as you're kind of learning and experimenting with what gnosis can do that uh definitely try out the samples it's a great opportunity to see how things work how they recommend structuring things and we're going to kind of go over some of that once this is all set up and running so um we're about two minutes into hopefully what's about four minutes so just to kind of touch on it once you've got that um that custom package imported you should see this gnosis gui menu um i think that's a good checkpoint that after if you try to import it if you don't see this or you get some type of error message or something else that comes up you're going to want to start debugging something else before even continuing with this video so just kind of use that as a checkpoint but it does look like we have unity open here so that's great um there we go it's finished so as it mentioned we're going to go to the assets menu import package and then custom package right here it's sitting right at the top so we're going to be using gnosis gui 3.0.11 and we're just going to go ahead and open that get a little spinner it's preparing the package now cool so here's the import selection that we get to pick from as i mentioned we're going to uh we are going to pull in the samples when we do this there's some editor libraries and the samples right so this is a point where if you're actually adding this to your game uh you can you probably want to consider unchecking that you can it's not going to hurt your install or anything if you decide that you've say it's an accident that you've checked this off and you're you've included the samples you can always just delete the content you don't need to have it that's totally fine um and they also do have a link uh like basically their whole github is set up so that you can get these samples uh if you if you mess up and you did want to see them and i think the other thing too is i think the way that the unity packages work is that you can basically re-import this and it will just let you have the opportunity to overlay the samples folder as well so you can't really mess up too much here if you've accidentally done this and you wanted it or the other way you've included it and you didn't want it but anyway we're going to include it for now i'm going to press this import button it's going to work its magic and add these files into our project down here and hopefully this process doesn't take four minutes because that'll make for a pretty boring video but again trying to make this as uh you know foolproof as possible i want to show you like all the steps that are actually happening so that you're not guessing at uh some steps in between you can pause and rewind this video and kind of do what you need to do so it's importing the actual files now i'll just wait for that to go through and we like i mentioned uh based on their documentation we should see a little pop-up window that says hey like you know here's no scooby like it's a little welcome screen and we're just going to close it off but like i mentioned that's a good checkpoint to make sure that things are working as we expect if you don't see that probably going to want to check on their forums and see what's going on and maybe there's something about your current install that needs to be figured out before progressing so with the samples i'm pretty sure there's a a lot of files to go through so i'll just keep waiting on that as that's going just want to touch on the fact that this initial scene we've added has a main camera added to it for this little walkthrough we're going to be talking about getting gnosis to work with a camera you can definitely i haven't tried this yet but you can set it up so that the gui that you're creating in gnosis actually renders to a texture instead and i think there's some really interesting things you can do especially with 3d games where you can essentially make some surfaces like in 3d render a wpf gui and i think that's super cool for the games i'm creating i basically only make 2d games because anything that's three-dimensional i'm already pretty terrible when it comes to art so 3d is just a stretch of that uh my inability to to have art so i stick to 2d i'm going to be using this this camera to set up the gui but as i mentioned they have documentation and uh and stuff like that on how you can set up your user interface to actually render to a texture if you prefer that method iteration two okay come on just when we thought it was done but here's some other samples coming in okay so while that's wrapping up let's quickly double check the rest of some of this documentation here we said we're going to get this menu so hopefully we see that we did select to have the samp the example so we should see these samples included here and yeah as i mentioned it says right here a lot more examples can be found at our github repository so definitely make sure you check that out hopefully you don't find yourself reinventing the wheel for any of this because i think they have some really awesome examples for you to walk through it looks like we're still going we're getting there um and then there are some settings we're probably just going to leave this to the default settings for now i think you can figure out what you want to do if you need to adjust anything i think another key thing that i don't believe they necessarily address in this uh documentation here at the at least an order um so i'll call it out right now that there is a a license and you can use a trial license so we're gonna do that um probably if it complains to us but i've seen it where it basically starts up it renders and then a few seconds in it just kind of kills the rendering of your ui and says hey you need a license but there the trial licenses are and stuff are free um i think that they have pricing and stuff that's uh i think for if you're not profiting off your game i think it's free essentially and but anyway you can check that out on their website so pull this back up here's the welcome menu that we said we were supposed to see and we see it so that's awesome um so now we can get started so i mentioned i'm going to leave the settings at their default um but what we're able to do is basically add the gnosis functionality to our main camera by adding a component that is called a gnosis view and that's total coincidence that i have noe type there for a gnosis view that must be from my actual game that i am working on so when we add this gnosis view component to this camera we need to give it some xaml that we're going to be showing as our user interface some other key things and i'll probably make a separate video to address this directly but i've seen that i basically need to have this box checked for enable i can't remember what ppaa stands for it's some type of anti-aliasing but what i've noticed is that when it comes to drawing uh very small details like like say one pixel width lines on some of your ui elements if you don't have this checked off then it seems to do some odd things like it anti-aliases your lines and ends up causing some of the lines to basically go invisible so i found that some of the details to show up properly do require this box being checked but i'm pretty sure if you search for gnosis and ppaa you'll find some information on exactly what this means so definitely recommend you get that done and we'll save this so we have a sample scene with a camera and a gnosis view attached to it but like i mentioned we're going to need to get some xaml created in here so that's kind of the next step we're going to look at i think if we just want to see a super quick way that we can get that working we can probably just grab a sample and just drag and drop some stuff in there so you'll notice as i kind of went into the samples folder then inventory we have some some assets and stuff in here and right before our eyes we saw that main window kind of switched over and uh became like a different icon i'm not exactly sure how it's operating uh under the hood here but gnosis itself will do some work to basically transform your xaml files into an asset that can be used in unity so i think it should be as simple this is from a sample that's an inventory for like a like what looks like a role-playing game um so we should be able to do that and drag this right over and i believe as soon as we press play we should actually see um like an inventory window kind of animate on it's super fancy like how easy was that right obviously all the work is involved in actually creating this user interface but to get this set up was camera which came on the scene add a gnosis view to the camera and literally just drag and drop this right onto this xaml property in the editor and you know we can you kind of watch this as i ran it i didn't do anything fancy here um but anyway obviously all of the work is kind of put into creating this xaml with animations and they have you know um like a sprite atlas to pull all this stuff from so they've done work outside of this and i think that's the benefit of using something like gnosis gui in my opinion at least is that i'm a lot more comfortable writing back end code i'm a lot more comfortable at using wpf to make user interfaces even though i'm not great at it for me it's it's like light years ahead of trying to create a user interface directly in unity and i would much rather spend my time doing that because i already have the skills there so as with this example all the other effort to make this happen was done outside of unity right and then you kind of put that all together and in a xaml file and and gnosis will make this asset for you basically automatically there's some interesting behavior like i'm not sure when it actually decides that it when you change your xaml that it will uh like re-import the asset i do know they have some hot loading hot reloading type of behavior i believe while your game is running you can actually do it but i haven't really dabbled too much with that but anyway they do reload this asset based on your xaml so you spend all of your ui design time outside of unity and uh that kind of process happens automatically for you so that's super powerful in my opinion i probably just said the same thing four different ways so apologies for repeating that but you know super easy way to get this example up and running so i'm going to go ahead and stop that and let's spend a little bit of time basically walking through the next few steps in their guide as well as some of the setup of the the projects and stuff i think i was kind of contemplating how we could set it up through blend but it um it's probably a better opportunity to look at their samples directly and kind of just walk through that so i'm going to flip back to their their instructions here i talked about these this configuration how i wasn't necessarily going to bother with tweaking anything there blend is the next step which is probably worth talking about so um blend is kind of like it's essentially visual studio that has a bit more focus on uh you know ui editing for wpf and some of the tools built in for animations and things like that i'm absolutely terrible like i've mentioned at creating gui so the animations and stuff i'm not great at but there's a lot of tools in blend to help you animate things and you get your your visual editor for wpf so you can see the xaml rendered as well as the the actual xaml code so let's go ahead and kind of look at that so and ext it's just to kind of read it an extension for visual studio is provided to create blend projects that are compatible with both wpf and unity and this is key so if you're used to using unity and c sharp you'll know that the solution and project file is essentially automatically generated by unity and that's kind of weird i think a lot of people that are used to managing solutions and projects know that there's a lot of effort that can go into maintaining the structure and that kind of thing that you want so having something like unity just dynamically create those things is to me totally bizarre but in order to get blend to work they have essentially a tool from gnosis that will kind of let you get up and running to have blend set up as well so they do recommend i've already done this on my machine so i'll kind of show you how it works but they do recommend that you get basically this extension so that you can get project templates um for blend and uh and gnosis to kind of work together with unity so just took oh this i just read hot reloading so if the flag hot reloading is enabled in those scooby settings panel changes perform will also appear immediately while in play mode so that's that's super cool um and project template so let's go ahead and get that in visual studio right so um i think i can do it blend for visual studio so let's do that so i have 2019 here as i mentioned i already have it so not a good example i'm just going to open up my my solution here and i think this got corrupt from something earlier but let's kind of check out the extension so it's saying go to extensions manage extensions and then in the extension manager expand online node and search for gnosis so i already have it as i mentioned right but let's go ahead and search it oh that's weird it's it says that i don't have it included but i've absolutely done this before so anyway you would go ahead and press download it says you're gonna have to restart visual studio and then that way what you're able to do is go file new and then pick a project template um just like this right so to create an application for the project template you pick gnosis gui here and then it will basically set up a blend uh solution for you in your unity folder okay so let's talk about a couple other things their their documentation is in a really good order here to kind of explain some of the things you're going to see um this part i don't really like about gnosis but um i kind of have this mentality that if you're going to have what i call like you know ugly code or garbage code or something like that that it's kind of confined to one spot so i feel like they have a good balance of this where i hate personally having um like if defines um and conditional compilation i i just can't stand it i find that it makes things super complicated really fast but i think they have a good balance of it and essentially this is just so that you can use blend and gnosis in unity kind of at the same time and have all your stuff work together so this is a great example here where you're gonna want this and i think they do it automatically in their uh and their template for you so you have an example of it but you kind of need to say that um your using statement for gnosis essentially replaces this um and if you try to have both name spaces in at the exact same time either blend won't work because it won't recognize gnosis or unity won't work because it doesn't recognize these other pieces so you do need to have this conditional compilation you don't need it in all your files but you will need it in your xaml files so actually let's go back sorry i'm jumping around here we're going to look at their solution [Music] so show in explorer i'm just going back to their inventory project right so sample inventory and let's get this organized by type so they have maybe that's not a good way to do it oh i thought they would have a blend folder in here and i guess they don't that's okay we'll use our own then so i'm gonna go to here open up this folder and actually let's walk through oh they're up here my apologies they put the their i figured they had blend projects i just didn't know where they were so they actually have them in the root of the project here and then a blend folder and then inventory okay so they just organize it slightly different than i did in my own project but theirs probably makes more sense to do it so they have samples blend this would have been created using that workflow that they talked about with the project template that we just talked about from the extensions that you're going to want to install now let's kind of highlight some of the things that going on so we're only talking about the one sample we're interested in which is this inventory blend one now let's talk about so just ignore the other samples that are included because if you kind of compare this to a game that you would have you're not going to have all these other projects in here you would probably just have one for your own user interface that you're working with but they have a blend specific folder and that's kind of to give you an entry point into a wpf application so let's go ahead and we're going to just launch this one so you notice i said inventory blend as the startup project and i'm going to press start here and if you remember what we had in unity we had just by dragging that xaml file onto the gnosis view xaml property basically right away we had a workable user interface so if i press play here this is going to create a window for us and drop in that same control so this is just an entry point into a wpf application with the same uh controls which again my opinion on this is it's super powerful so that you can actually work in your ui um kind of get things going how you like and not even have to worry about working in unity i think that's just you know incredibly valuable so let's go ahead i'm going to close this i just wanted to show it running but you know you have in this project you do have an entry point here your app xaml you can open that up startup uri is pointing to main window inside of assets gnosis gui samples inventory and main window right so this blend folder is blend specific it's just to give you an entry point into a wpf application but this folder is kind of what you would see in in unity so and i think maybe a good example of showing that is that if we kind of mirror the folder structure i'm just going to try to put these side by side to compare we opened samples blend solution which is what's loaded here in blend and if we're looking at the projects inventory blend is sort of at the same level as the solution right and then if we want to look at the actual files that are included in unity they're in our assets folder right then they're in gnosis gui they're in samples they're in inventory and hopefully you're kind of seeing as i click through that this structure up here right is the same structure that you have inside of unity right it's it is literally this folder um so that's just a key call out um and what else do i want to highlight here so let's actually look at the content of one of these windows because we were talking about conditional compilation um so what i'm gonna do is open the code behind and there we go we can see at the very top we do have that conditional compilation we are not in unity we're in blend right now right so this part's all disabled we do get this and then there's a couple other things going on here um they have some hookups for um some callbacks here so this part is kind of like specific to their um you know their their sample you don't have to do this this is what they do um same thing with this code here that's really just to uh on load assign a a view model to their data context that's all it's doing so you don't need that and you don't need this you can kind of decide how you want to handle setting up your view models but there are a couple important things to acknowledge here um and this is like ideally if you're using something like mvvm or some other ui design architectures you're not putting basically any code in your code behind except for in this particular case some of what we see going on here so let's talk about this um right now because we're in blend initialize component is actually hooked into the framework such that it will set up everything in your xaml right so your xaml uh how that's all defined that's going to get pulled in and basically converted into objects and stuff like that we call initialize component here so this is responsible as it would normally be in a wpf application for doing that work the conditional part here is that when we're in gnosis we don't have that same part of the framework that's going to be converting our our xaml into objects but instead they have their own sort of system for doing that we talked about having gnosis automatically convert the xaml into an asset for us and that's actually what this is going to do for us here so instead of um like when we're in unity because we can't rely on the built-in stuff gnosis has their imp their own implementation so uh this doesn't exist but we kind of define it while in gnosis right that's what this is for and then call into their framework to do this work for us um personal opinion of mine is that i actually wrapped this even more to have some automatic namespace resolution so that if i'm moving stuff around i don't accidentally have things in the wrong folder so just wanted to call that out is that you can do you can make some helper methods to make this a little bit more clean for you if you're ever worried about um say you're moving this control main window into something else that's not inventory like right so it's you've moved that into a cool ui stuff folder as long as you're keeping my opinion as long as you're keeping your namespaces and folders in sync you can do some cool stuff with basically making this a little bit more automatic to resolve anyway just a thought if you're trying to reduce some of the differences between gnosis and unity sorry gnosis and blend so that's probably mostly it for this particular file um trying to think if there's anything else it's really worth mentioning and the answer is probably if we look at the solution for what we just made um from the unity side right so we have blend opened this is going to be the blend perspective but let's open visual studio from um from unity basically right so this gnosis test solution is the one that unity created for our project so let's go ahead and open that and what i'd like to talk about is the files that you see included in the solution depending on depending on if you're in blend or if you're in unity okay so that's loaded up hopefully i can do this one on each side here okay so ignore the c-sharp editor one for now we're just dealing with this top one um you'll notice already right so we don't have a blend folder in here because that's blend specific we do have assets right and this is on the right this is created for us this whole structure is created for us by unity and you'll notice assets gnosis gui samples inventory you know same thing assets gnosis gui we have some extra stuff right so that's the first subtle difference so if i press show all files suddenly there's a lot more stuff right in blend there's a lot more stuff that's hidden from us first of all because this setup for their samples was just this one project for this specific set of stuff they didn't include all the other sample project code in this one so that makes sense that's kind of that's an obvious one second thing you'll notice is they don't include asset or asset metadata the reason why is that this is unity specific it offers no value here okay so don't worry about trying to include those they're not going to be of any value inside blend for us and let's kind of dive down a little deeper so plugins we were talking about the inventory one so not plugins we want samples inventory and then you'll notice that this folder is a lot different right inventory why like where where is everything else that we were interested in so i'm going to do the same thing and we'll notice that there are things that are included in unity and not in blend okay and the things that unity doesn't care about are the actual xaml files unity does not care about the xaml files that is a blend specific thing and as i've mentioned a few times gnosis while in unity will convert those xaml files into assets so that's why on the right here um like actually it's probably a poor example but i'll elaborate a little bit more of the the right side does not care about xaml files and assets aren't used in code but they are used in unity right so that's a unity specific thing and as i mentioned uh unity side does not care about xaml so they don't get included and the left side here is focused on xaml and not assets so that's why you see these included just it might sound complicated but you got to keep in mind that on the right side this is controlled by unity by default it's not going to be including xaml for you so you don't have to worry about remembering to include that or not it will just regenerate your project and kind of keep it out of being included in here if you try to include it depending on what your stuff looks like number one first of all unity will blow away you know this inclusion when it regenerates your project but number two if you do it temporarily i've seen sometimes where it just has compilation errors because it can't resolve certain things but it doesn't need to be included and it won't be as i've mentioned so let's go ahead and actually open up the code behind for it and this is the part that is important and you'll notice i have the same thing open on both sides here and because on the right side we are in unity we do oh i didn't save this so that was confusing there you go now they're mirrored um you'll notice that because on the right side we are in unity we get gnosis defined right using gnosis and then this block of code does get turned on on the left side it's the complete inverse right so if i f12 on this one on the left on initialize component to actually go to the implementation you'll see that this is like some some built-in stuff that's actually going to kind of get our xaml all built for us and convert it into objects if i f12 on this one on the right side it just goes to this line so if i don't have this essentially gnosis does not have a way to go pull the information from the assets for us so hopefully that kind of helps explain that piece and maybe with that said let's do a quick jump back to the the setup documentation and see if there's anything else that's probably worth mentioning so they talk about the xaml assets here it does say if you don't need to edit the xaml you can remove it from your project because it's not necessary for the run time right although this is not strictly needed because xaml files are not included in the final build produced by unity right because as i mentioned they're converting those into assets nothing's actually happening with the xaml files after the assets get created so they're not going to get output um let's see they talk about resource dictionaries and things like that so how that those get managed textures so this is actually an important part that we're going to want to cover pretty quickly this took me forever to find and i'll just kind of touch on it briefly because we're not messing around with uh creating a whole user interface from scratch right now but if you're dealing with uh transparency so if you have pngs as an example this is something you're going to want to pay attention to i'll probably make a short snippet video of this exact same thing for people to easily find but um this part's important if you have a jpeg no transparencies you want to use it as a background you don't really have to mess around with this but you will want to care about the alpha settings with pngs and i didn't realize this but you'll notice this icon right here it's a little tag right you're going to want to click that and then make sure that your pngs that you're going to be using in your ui do have this gnosis label on them if you don't do that your picture will still show up but what i've noticed is that it will have all sorts of uh different like odd artifacts in the background of it where they're supposed to be transparencies it's highly unusable that way because it looks like a completely different picture but if you're starting to experience that um i definitely like you know they have it in their documentation it's easy to skip over and it's even harder to notice if you're not familiar with some of these things in unity there is this little label thing here and you're going to want to set that to be gnosis i personally have not dealt with any audio or fonts yet so i'll skip over that they do have some input pieces here as well and how to configure that um and then they kind of talk here about setting it up with a camera which is what we showed right so we dragged and dropped um the inventory main window right onto this gnosis view that we added if you've made it this far and you're kind of curious about how you can do it with a texture instead of a camera again rate in their documentation i personally have not messed around with this but i think they have a really cool example here where uh they literally have a wpf ui rendered to a texture that's in uh you know a 3d space so that's super cool and then a couple extra points on hit testing i'm not going to dive into that and then how you can get that to work on these other players all right so i think that's super high level how you can set up gnosis gui with blend and visual studio um and unity to all kind of just work together um just to reiterate quickly i think there's a lot of power uh in being able to make wpf uis have them in unity and personally this is something i'm going to definitely take advantage of trying to learn how to make user interfaces in unity is challenging for me and i really enjoy being able to leverage you know xaml the the resources the styling the animations um all of that from from wpf in my in my game so um with that said i think that's it i will have a link to both this tutorial and probably i'll put in their their github link as well so you can see the samples and i do recommend you check out the samples because they explain everything that you probably are trying to do with some concrete examples and they're all really flashy and uh cool to use so hopefully this helps and if you have any other questions please ask in the comments i'll see if i've dealt with it or i can answer quickly or maybe make a whole video on it and hopefully this was helpful thank you