Fluid Architecture

hey today i wanted to talk about uh something i tried to come up with a clever name for it i'm calling it fluid architecture um uh i like to be able to stream this and record at the same time so that it feels like a little bit more natural for me to talk about and then i don't feel like i'm rehearsing it and uh it's really contrived but i did try to make some notes ahead of time so i have a bit of a template to follow um so i'm just going to kind of i have this open office little slideshow here it's nothing fancy but just has some some notes that i want to go through so i'm going to talk about fluid architecture what i mean is being able to develop software in a way that um it allows you to to essentially keep evolving what you're working on without having to necessarily stop everything you're doing spend a lot of time trying to redesign everything without delivering until say like months potentially years depending on what it looks like um and then sort of saying okay like here's the the new hotness or you know doing a complete rewrite um so kind of canning everything you have going on again not delivering uh incrementally and then saying okay like here's um the new thing we've created so fluid architecture is sort of the opposite of that where you're able to keep evolving without your system uh you know totally sorry not totally being compromised and you can you can keep delivering things as you're going so um that's what i mean by fluid architecture um so i kind of want to get into like you know why this is something i i feel that you should consider um i wanted to uh also i i don't think it's really fair like um to talk about uh you know here's a way to look at stuff and like here's all the benefits of it without addressing any sort of drawbacks i think that uh many times someone's gonna you know propose something to you you should be able to hear um the flip side of it right if everything always sounds like oh here's all all the perfect reasons why you should do something um you know then everyone would be doing it without asking questions but um if there's anyone that's going to question it you should really be um thinking like well there's got to be some sort of drawback so what are they um so i'm going to get into sort of the pros and then we'll follow up with the cons and then i have some notes on and i want to give some examples about how you go about this and like i said some examples that i've done uh in my experience so we'll get started with the pros um so the first one i've kind of hinted at it but like business continuity i think is a huge one so um if you're working in software teams um i'm gonna try to give some context to like software teams versus like your own personal projects um so business continuity i think is important if you're actually working in a software team where you are delivering software there are customers that are using it um and you know there's a business to support um i think that if you have a different like every software organization is going to be a little bit different in terms of how this works but um so say in like an agile software shop you might have product owners that are kind of the ones that will be telling the engineering teams like here's where we want to be spending time so that could be a project manager in some cases that might be a product manager like the titles and things might be uh different but if it does boil down to someone being able to make that decision i think that when you're able to talk about having fluid architecture um you likely will have easier uh an easier time getting someone to give you give you and the team the ability to work on that kind of stuff so if you were to say like look we're not going to be able to ship software for um six months but when we do it will be great versus like um you know maybe it's going to take us seven months or eight months to get it to to closer like to say almost perfect it's like the ideal state it's going to take us a little bit longer but the cool thing is we don't need everyone's time and we can still deliver other value as we're going my opinion in my experience um sort of that latter case is a little bit easier of a pitch to them because when it comes down to business if you're not able to continue shipping software generally that's a big problem i have another note here saying like a from scratch rewrite so if you're taking something and you're like well it's going to be a bunch of work to make this better so let's just scrap the whole thing and rewrite it either doing that or having a very long running refactor and what i mean by a long running refactors kind of from the previous example of like essentially waiting months before you can actually deliver value it's kind of gaining other value from being delivered from my experience those are very expensive um and some reasons why are um you know you have potentially an entire software team held up trying to um like as as your as time is going on and you're like say you're doing your your sprints again in like an agile software shop um you're not actually introducing any more net new value to the customer so you're sort of it's like you're paying off this tech debt and there's nothing wrong with that but you're paying it off and like from a customer's perspective they don't care they don't have insight into your tech debt it's not something they should ever have to think about because they're buying your software for the features the functionality right so um if it's like well sorry you can't get updates you can't get uh you know new features or bug fixes just because we're we want to take time to redo this like um that's gonna cost you um versus the alternative like like i'm explaining with fluid architecture being able to say like look maybe we're delivering features and bug fixes at a slightly slower rate but maybe it takes us a little bit longer to get to our ideal state um but you're not you don't have this uh huge expense of uh not delivering software as you're going um so that's sort of the first major uh pro i would say uh next one is um and i've talked about this in previous videos i sort of and you might have heard the concept already but this 80 20 rule so essentially software is never going to be perfect anyway um so if someone's looking at a code base and they're like you know what we're not happy with this um you know maybe some parts are brittle maybe it's not designed the way we want um and uh you sit back and you say well let's let's sit back and design you know the perfect system the perfect solution for this the reality is it's never gonna be perfect anyway there's two like almost in any uh real situation there's so many variables that you have to consider that um if you want to sit back and try to make something perfect by the time you actually get into making it and actually have people using it and you're getting feedback on it it's going it's going to have changed anyway even if you've already built something like this in the past um as you get into things there's going to be changes that come up so um the sub points i have here are just saying essentially like software is something that evolves naturally um so instead of pretending like it's not and trying to come up with something perfect and then when you find out it's not perfect saying well we have to scrap the whole thing or we have to do a whole rewrite or we have to spend all this time uh without delivering anything just so we can refactor i think like you're setting yourself up for failure like you should be assuming that it is going to evolve and then planning for that um it's like planning for you know the unforeseen things so being flexible and nimble um i the word agile is obviously uh it's in my opinion overused a lot in software um so i try to avoid saying it but you really want to be agile and nimble in the sense that you can let your software keep evolving as as new things come up um second point they're just saying like um as as you keep repeating things you're going to see new patterns come up um you or people on your team or people you talk to new patterns are going to to arise we're going to say oh man like that is better like if we started doing this that would actually stop us as developers from you know coding um something that might be brittle or hard to test or um you know is the cause of these certain types of bugs so you're gonna say well let's start introducing this new pattern to prevent that um it's it's just gonna happen naturally right so if when that happens you say well we have to stop everything so we can go change every occurrence of this pattern to the new one you're basically hitting the brakes for all of your software um say within this team that's doing this and then you're not delivering value um as it's happening so um the other part is just obviously like as you're as you're working with customers assuming that you have a feedback loop and you are taking customer feedback and trying to evolve software to to suit customer needs there's going to be new things that come up right so um you can do all the research you want ahead of time with product teams trying to figure out all the features they want but if again this is a lot of it's based on my experience the more conversations you have like that that's great but the the more that people start to use actual software that you produce the more they're gonna say like oh you know what like maybe maybe i was wrong about that actually this way is totally fine or um what you you did exactly what i said and it's not really now that i'm using it it's not actually what i want so like their requirements and their needs are going to evolve over time so um yeah in my opinion if you're if you have something like fluid architecture you can support business business continuity which i think is uh very important when you actually have a software business um to keep delivering value to customers um and it's sort of the second part is just really accepting that software has to evolve over time um it's just not perfect no matter how much planning you put into it so um just accepting that so little just a side note on like it so if you're not working in a say a software team or a business and it's a hobby project or um you know your business continuity isn't um you know isn't a huge deal i'll give you a personal example of uh something um where i i feel like i wish i would have done this for myself and my own hobby project so one of the things that i've liked to do just sort of to practice programming that might be a really nerdy thing to say but i actually really enjoy programming um and one of the things that i love to to use to practice is actually making role-playing game uh engines i'm really bad with graphics so i never focus on that i like making the systems in the back end um so say like uh item generation and how your characters can progress and all that kind of stuff i really love doing it um however uh what i had always done and i mean when i say always i mean for i'm trying like how old am i now so like over a decade probably um you know maybe like 15 years of this without exaggerating where i would take you know start something that's a role-playing game engine start building it maybe over a year and a bit and then get far enough in where i'm like oh you know what maybe i made some bad assumptions about this oh no i've kind of reached a roadblock here for uh kind of extending this engine well let me just restart it i've learned a bunch let me just redo it and what ends up happening is like i basically spend i'll make up a number but like say like 80 of my time just redoing everything i've already done um and then like i get about like a little bit further and then i reach this same type of roadblock so i spend so much of my time repeating the same thing and in my opinion and where i've tried to get to even like i'm saying in my personal projects is when you reach that point where you're like man i feel like uh this design has kind of reached its uh its maximum its limits we can't extend it anymore um instead of just giving up on it it might be a good opportunity to start looking at the parts that that do feel really good making sure that you can kind of keep those isolated and then um start focusing on the parts that you don't like so i'm going to come back to that at the very end for how to address it um i just wanted to share that um i do feel like this can be applicable in your own personal projects um but the weight of these might um you know business continue maybe you don't care you know i never had to ship my my role playing game engine to anyone it was just something for fun for me so this first point for business continuity did that really matter no um it just kind of sucked to spend like years on something and not actually get much further every time so great learning but anyway um so let's talk about maybe the cons of this um and these are things that like i picked up from other people over time when i have this conversation um and i you know for me they don't carry as much weight which is why i do still talk about you know why i think fluid architecture is a great thing however i think that they are important to talk about i think they're important to understand and then coming up with ways around them so the why nots um so cognitive load um this is something i've heard come up in the past um for me personally i've worked in smaller software teams so um this is less of an issue however um this concept of like so say you have a bunch of people working in a code base i think it's um if you've been programming for a while you probably notice that if you're in a code base and you're about to add a new feature into something or fix a bug whatever it happens to be generally programmers like to um like to look at other existing patterns and examples so they can say oh that's how we do it okay like i'll apply that in this particular case so when you start to have fluid architecture one of the things that i like proposing is that uh when you start to introduce a new pattern you don't necessarily go back and fix all of the old examples of it and sort of balloon the amount of time it takes you just start introducing the new pattern um so when i talk about cognitive load here i mean that the bigger the team is and the more complex the software is and the more examples of these patterns that you want to be um updating the more cognitive load is there is the more cognitive load there is there we go um for people to understand what the new pattern is what the new hotness is um so that they don't start accidentally reintroducing the old patterns that people didn't like so hopefully that makes sense but yeah essentially it's just you know the more complexity there is the more cognitive load there is for trying to keep up to date on all the the new patterns um that you want to be introducing in your architecture so related to that um again based on complexity of your code base and your team size and things like that um there is a load for communication and i just made a note here for reviews um so same type of thing but i'll give you an example so so say that you have a new member of the team maybe it's a co-op student maybe it's just a more junior developer that's come in maybe it's a you know a seasoned person in the company that's just switched teams to work um in your code area and essentially they uh are tasked with fixing uh a bug or they're adding a new feature and they do kind of what i just said of like they're going to go copy an existing pattern because they're like oh that's how this team does it cool like i can totally do that for them you know and they feel good about it and then they go put it up for review and then i mean if they put it up for review and however that works so let's take the example where they just they've added the new feature and it gets landed um and then later on so say like a week later two weeks later someone goes hey wait like why are we doing it this way i thought we said that we were following this new pattern um essentially if you're not keeping up to date on your communication and catching this kind of stuff in reviews um you start interleaving more of the old pattern back in with the new pattern and it's it takes longer and it's harder for this new pattern to sort of take over your code base um so there in my opinion there's certainly more load on so say people coming into the team and working on stuff they should be asking questions early about like you know someone asking someone else on the team like this is the way we do it right this makes sense um when people start introducing new patterns and saying like this is you know we've talked as a group this is the architecture we want to follow this is the pattern we want to follow communicating that out with people right i think having frequent open communication is really important on software teams and maybe you want to have you know patterns that you can put into your code to make sure that this is tracked so maybe it's um in for example in c sharp you can put like an obsolete tag on stuff so maybe you can say don't look at this example you can leave a comment sort of as you're going so that um maybe it's not time for you to it doesn't make sense for you to actually go replace one of the old patterns but you come across when you can leave a little comment just saying by the way don't don't use this little things like that you can decide as a team um but the point is that um having a more fluid architecture certainly does um add more load to communication um and what what it might look like for your reviews to make sure that you can catch this kind of stuff um and continue the new good pattern that you want to use so um and then the last one um i just have a note here saying too far gone um so on the previous slide i talked about the situation where um i was saying you know rewrite bad thing to do it's gonna be costly um but i don't like having um saying like absolutes like you should never rewrite software um because the reality might be that when we're talking about fluid architecture here you're maybe you're already dealing with a code base you've inherited that i'll just make up a contrived example maybe there was something in place from a couple of years ago hasn't really been touched and someone was like hey like we had this thing it sounds like it does what you want so you start um you know revitalizing and resurrecting this uh this project turning it into a product and as you're going through it you're like you know what this is pretty far off base um maybe maybe it is worth asking the question uh is a rewrite worth it right like um there is a balance between um you know your business continuity if it's not already being shipped maybe maybe it's worth taking the time up front and addressing like this is too far um off the path that we want let's take the time to redo it i just think it's it's worth asking the question um generally from again from my experience um depending on how out of date something is a lot of the time there are great methods for um basically starting to to introduce a fluid architecture and progress something and refactors you go however um i don't want to pretend like they're there's never a situation where you should consider a rewrite um so i just wanted to mention it because i don't think it's fair to not say it okay so those are some of the cons i've come up with i've mentioned some of the pros um i'm not saying these are the only things to consider but these are some of the the major ones i've had uh come up in conversation the ones that i think are important um i'm sure after i finish uh recording this i'll go oh like i should have talked about something else um but that's uh part of the reason why i like to do this live so that um uh you know it feels more natural for me to just talk about it and maybe i'll make another follow-up video or something on it later so we've talked about pros we talked about cons what about like how do we how do we do this um if we want to start incorporating a fluid architecture into our systems what do we do um so i have a couple notes here um i've talked about some of these things in different videos so i wanted to elaborate on them and then maybe give some concrete examples if i can either from personal experience in my hobby projects or generalized experience from my my career so i like being able to give some examples from real life in like software industries so that there's more weight attached to it obviously i need to omit uh specific details on things so that um i'm not sharing private stuff but anyway um i think first one uh abstractions and flexibility um i made a video before on uh sort of software pillars i've talked about you know trying to decouple things with interfaces and abstractions but i think one of the core things here is um when you're designing software trying to decouple concrete implementations from each other i think is really important so the example i like to give is it's a really simple one probably overused but like um something like a data layer you you know that your software wants to be able to uh persist data to some type of storage right um you might have picked like a mysql database maybe it's a sqlite database whatever it happens to be um but abstracting that away so that the people uh the developers calling that api that say like i just want to save information i want to persist this they don't know that it's uh mysql they don't know that it's sqlite unless they're like right into those libraries if they don't know then they don't make assumptions about um certain things and sort of like scatter that through your code base because the more scattered those dependencies start to get the more coupling you have across your code base and then it becomes significantly harder to have a fluid architecture um so that's i think one of the most critical parts of this is like is really just trying to decouple things as you're designing stuff if you have systems or that are already coupled um that's not the end of the world i wouldn't say oh no things are coupled time to throw it all away there's definitely strategies you can take to start decoupling as you're going again my opinion on that is by decoupling as you're going that's starting to introduce a fluid architecture right um taking some small steps to start saying you know what um we we're calling this concrete class here on this uh specific implementation let's let's wrap this in an interface let's put this behind an api um so the next person to come in here can start building on this then we can share as a team hey you're fixing bugs in this area no problem if you see this type of thing just start introducing um this new api that we're using right like you start introducing things as you go without saying everyone hit the brakes let's go back no one can develop anything else until we go address this like you can just do it um as you're going and i think that's really critical next part functional tests um i mentioned this one because a lot of what i like to talk about when it comes to testing just just by from my again my experience as a as a programmer i love writing unit tests because for me it gives me confidence over um like the granular steps in my lines of code in my classes my methods so when there is a problem i know exactly where it is however as you're starting to refactor things your like a unit test that's testing specific lines of code being executed become very coupled it's one of the things we just talked about not doing but they become very coupled to your implementation for a unit test in my opinion that's totally cool because the point is that you're proving it does exactly what you say and i think it's important to mix in things like functional tests that don't care about that and the reason why is if you only had unit tests and you start to refactor these systems because your unit tests become very coupled to the implementation you have to change all the unit tests right because you've changed the implementation have to change the unit test cool um but does it still work right and i think if you have something like functional tests that can cover larger parts of this system then you can say you know what we changed the implementation this functional test still says that it's doing um you know like a black box it's still doing what it says it should we don't know what's happening on the inside from the the perspective of the functional test but it looks like it's doing what it should it has the behavior we expect so it passes um i think it's important to mix these two things together um i think if you're only gonna have one type of test a functional test um you know it's the way to go it'll cover coded functional tests we'll cover like uh more of what you want um i do think that if you can design stuff to have uh the ability to be unit tested then that's great too because then you have really granular coverage over things that's why functional tests i think are important for this for your fluid architecture um i've talked about it already for like in the pros but like you definitely want to look at things as like a bite size or piecemeal um part of what you're doing so when you're thinking about a fluid architecture and you're like you know what how we've been you know interfacing with our database we all of our spots in the code are they know that it's a mysql database we're already i'm just making this up but like we're already talking about like switching over to um you know a certain type of uh nosql database and we have all this logic scattered so instead of saying everyone's i've already talked about this in different ways but instead of saying everyone's stopped like we can't land any more code until we go address this and put this behind a new api or whatever it happens to be instead of doing that just start saying okay the new pieces we're adding we're putting behind this api i think it's a really good sell to your product owners it lets you keep making progress and delivering value and i think it's just in general a good practice for agile software development to be able to have these small incremental deliverables i think a lot of the times some people they think about software deliverables as being like things that users can see um and this might be different in different software shops um i've seen this different in the same software shop but across different teams but having this perspective of like um we have a user story so the user should be able to see it so um if i'm going to add value in like we should have someone maybe like a tester or another programmer they can see the or the product owner we can demo it to them and they can see the end result i think sure um that's you know one way to look at it but i do think that there's absolutely space in agile software development to say like we're delivering functionality and call it a story maybe it's not a user story because the user won't see it or you have to change who the user is to i don't know like sometimes i feel like the agile stuff is really contrived but anyway being able to deliver a small piece of functionality covering it with tests um having someone review it and saying like this code will do what we expect i think it's a great way to start delivering smaller bits of value again without pausing everything and then just the last part i talked because i want to be able to address some of the cons here communicate and review i think it's it's critical um it's less important if your team is like one person so if it's just you working on stuff great um you don't really have to communicate with anyone you're probably not doing a code review uh a formal code review for yourself but as soon as you have one other person on the team um i think it's really important that you start talking about hey like this is the pattern we want to start doing um you know i've noticed i'm going to keep using the database example i've noticed that we keep addressing the the the storage mechanism is explicitly my sequel maybe that's not a good practice so going forward we should do this um then on the reviews um you know trying to call that kind of stuff out looking for it and then the flip side too is like when you're starting some work like you're picking up a new story on your team the sprint's just started and you're getting into it instead of just instead of just putting your head down and being like oh i know i know everything about the code base i'm just going to blindly start coding take take a couple minutes and just you know message a colleague like uh if everyone's uh working remotely it's you know you're not just turning to a colleague shoot them a message on slack or whatever um maybe call them up and just be like look i'm about to do this i just want to make sure this is the right pattern we said we're going to adopt and then get into it right catching it early before it gets to the review i think is super helpful it saves people time so um i wanted to give a couple of concrete examples um first one i'm gonna give and why uh i wanted to put this video together is something uh that recently happened and see if i can switch over the stream here to um show visual studio not that one no it might be gone okay well one sec let me again this is the the nature of doing stuff live um let me get some code up there we go um okay so when i was working through um this application i'm making with a couple of friends um we basically reached a point where we said cool like things are mostly um and then we started talking about some of the ui designs and we started questioning some things we were doing and this is kind of what i was getting at in my slides where i'm saying like things are going to evolve so we came up with some paradigms that kind of challenged what we were doing before and instead of me throwing away the whole app and going well it's good that we learned this let me start it from scratch what i started doing was porting things over to a new format so the thing i wanted to call out here was um i i have a bad habit of doing this to start and i it's just uh literally from years of doing it trying to break out of it but i generally make architectures and layers um and almost always get to a point where i'm like the layers don't make sense anymore so the pattern i had before was that um i would if i wanted to put a new screen in i would put a i'm already digging into multiple folders here i would go so i'm going to pick on this friends page i would go add this view so this is in xamarin so i'd have a wpf control and then i would go have a interface for it an interface so again another folder over here and then i would go into the view models and i would go into this friends folder and then i put all the view model stuff in and i might have a controller for it so there's a friends controller and you can as i'm clicking through this your your i don't know like you might already be like why is this scattered everywhere um so what i started to do is as we were revamping the um the ui for some of these uh these new paradigms was i said you know what like i'm i'm i'm sick of jumping around all these folders every time i want to make a new screen it has these similar components so i started migrating stuff into a screens folder so we have a login screen and it has the view pieces and right beside it it has the view model pieces um i have a common navigation thing that i generally use for these different screens there's a controller for it and then the nice thing is there's an uh an auto fact folder that has a module for doing all the dependency injection so the really cool thing is this layout lets me have like one folder that contains everything i need for doing login stuff so it's it's a nice little sort of package of of login like it's it's really convenient so my point with this and why this relates to you know this whole topic is that i was able to start porting the screens over to this paradigm and i still interleaved i have these other pages and stuff still mixed in so i have both the old way where i have pages with their interfaces scattered with these view models over here with the controllers folder over here i still have that paradigm in the code base but i've started introducing this new one and i'm doing that fluidly so every time i go to update one of these pages i port it over to the screen folder and convert it so i just do it as i go so when i started doing that i was like hey this might be a good thing to talk about um in a video um i've i've seen this come up literally um you know in professional environments where it's like well we feel that we can't go ahead and make this change because we can't have these things coexist and i just wanted to be able to talk about why i think you can um another example from my professional experience is i think a pretty cool one so this is a this example i just walked through here is a little bit more of like an organizational um sort of code organization um whereas this other one i want to talk about was actually a technology choice so um the app the application we were building uh at the time we started building out in wind forms um and we were you know like rapidly putting this application together um and it was really awesome we were able to use all the technology that we were used to using um part of what the the organization was using already um iterating fast and had something functional within no time and then what ended up happening was towards the end of that we started hearing well we think that we want to start like as an organization start shifting to wpf because winforms is getting a little dated wpf is sort of like um you know it's going to be replacing it um a lot more support looks way better like so there's a lot a lot of really good reasons to shift to it and we were kind of like well that sucks like we just build all this stuff in wind forms like what are we gonna do um so we came up with a pattern that essentially allowed us to to start interleaving we were building new screens we could start interleaving wpf with winforms and then every time we had to go back to a screen or we got a little bit of time that from our po we could say like you know we want to add another story to the spring just let us go back and update this page from winforms to wpf there's no issue at all and we were able to start um you know all the new screens being added were wpf all the old ones that we had to go refactor um because of bugs or new features and stuff we would just port them to the new format um and i think a really cool thing about that that i want to share is like i think i think it took about and it might to this day it might not be true but i think it took about four years for us to actually replace the final win form screen and i say that might not be true because there still might be win forms in there and the reason i think that's important is because if we would have taken um all of the time to go replace all of it um without uh you know um continuously delivering and having like a fluid architecture uh i'm pretty confident that that product would not have ended up where it is um we maybe would have missed the mark on you know a window of opportunity for shipping to customers getting their feedback um so in my opinion i think it was critical that we had a way that we could fluidly um keep adapting the architecture um so yeah i wanted to share that one because that's it's a real example of something we did how we did it that's a different story um but um the point is that we decided like when you're talking about software engineering there's always constraints right so one of the constraints we imposed was that we can't stop shipping the software right we made that a constraint once that's a constraint and we said we can't stop shipping the software and the other constraint is all the new stuff has to be wpf great like we play within those boundaries from there we come up with creative solutions for making that true um so i think that when people say well it can't be done it's like well you're just looking at the wrong constraints like we're if that is truly a constraint that you have to keep shipping software get creative right that's that's part of our job as software engineers so i think that's about it um just wanted to touch on pros and cons of doing fluid architecture again i think that it's i think it's critical in in software engineering organizations that you're able to do that um obviously the technology choice how you're shipping the customers and things like that will have an impact on on and sort of the age of your software will have an impact on uh your ability to do that uh i think it's something that's important to think about though um and yeah again i don't think that it it is free like there's obviously trade-offs to doing it things like communication and cognitive load but i do think that the benefits uh almost always outweigh the the drawbacks here so thanks for watching and hopefully that helped in some way