FORGET The Ninja Fixes - Tackle Refactoring And Tech Debt With Purpose

look I totally agree that it's really important to invest time paying down Tech de in our software applications but I do disagree with some of the mentality around how we approach refactoring and when we should be doing that I see a lot of people promoting that as you're coding things you should be refactoring as you go and while I'm not opposed to the idea of doing this I think I'm opposed to the implementation of how I see people suggesting this in this video I'm going to walk you through my approaches for addressing Tech debt and refactoring based on my experiences at a startup where unfortunately I was the contributor of a lot of the tech debt that existed just a quick note that if you're interested in learning more about refactoring in Tech debt pay close attention to the pin comments below and another reminder that if you're not already subscribed to my weekly software engineering newsletter you can find a link there as well all right so let's dive into the discussion so I mentioned that I'm not opposed to the idea of doing Tech de as you're going along with other feature development or bug fixes but what I am opposed to is doing it completely in line with that set of work and there's a few reasons for this and I want to start by backing up and explaining where I'm coming from I spent a large part of my career being a software engineer and an engineering manager at the same time so when it comes to things like what's a priority in terms of software development I often wear two hats when I'm thinking about this and as a programmer and someone that likes developing software I like thinking about refactoring I enjoy doing it I enjoy having clean code I understand the cost as a software developer of being in a place where it's you know brittle code hard to test lack of tests code's hard to read all of these things I understand that like investing time into that can really have a big impact on your ability to move forward and deliver in that space and then as a software engineering manager I also wear the hat of trying to think about the different priorities and trying to understand the business value so that we can always be focused on the highest priority things and if I go back to some of my earlier days when I was working at a startup and we were really trying to figure out ways that we could properly prioritize Tech we tried doing this thing where we had Tech de Sprints and the IDE idea here is that we all agreed that Tech de was important to address but we had so many other high priority things like bug fixes and feature development that we said look let's put our heads down get that work out of the way and then sort of as a reward or before we're planning for the next phase that we're going to start delivering things let's make sure that we have a full Sprint where we can go address all of this important Tech debt and that way the engineers are happy but also the product managers or product owners are happy as well but this was a really fast moving startup and as you might imagine we would get into our Tech debt Sprint and we'd have feature requests coming up or bug fixes and truly if I reflect on some of those instances they were high priority things in the field that we were working in if we had any type of error in the data that we were presenting that could be a huge problem for our customers and the side effect of this whole thing going on was that our Tech de Sprints were often very interrupted by other normal work that wasn't Tech Deb now when you have this kind of thing going on what ends up happening is that Engineers end up having their their trust eroded with the product owners and the people scheduling the work they're told yeah you're going to get a tech de Sprint but what they actually have happened is this situation where a lot of that Tech de they don't actually get to work on and this ends up leading them into another direction that is really similar to this boy scating approach where people are refactoring or addressing Tech dead as they go and the reason I want to focus on this part in contrast with what people are recommending online is because I think a lot of them are painting what's more of like a cheery picture of how this works and when they do it this way on the surface it seems really good so for example what a lot of people are suggesting online is that you're reading through this code and you're seeing stuff that you know could have better names it could have better structure to have more clarity you're going to be making it better for the next person coming along if you already know that and you're already there you might as well fix it up as you're going I've even seen some people saying like how could you break it just by refactoring it you might as well make the fixes and then you have that all along with the stuff that you're delivering there's this focus on being able to streamline that Tech Debt Pay down so much that you can refactor all of your work just as you're going and I just think that that's too idealistic so I want to walk you through some of the things that I was seeing in real life in real scenarios okay so in the situation where our Engineers are now no longer really believing that they're going to have time to address the tech debt they're taking matters into their own hands because we all did agree that paying down Tech debt is important we do need to have code that we can work in so that we can deliver things properly so people started to do this boy scating approach where they were making changes refactoring things cleaning things up and then committing that with their feature or bug fix but guess what a lot of that code wasn't tested or wasn't tested well so what would end up happening is either the feature or the fix that they were putting in that had to get reverted and along with it was the refactoring effort that they had alongside that or what was more common was that the thing that they were trying to refactor ended up breaking because there wasn't good test coverage in place now their new feature or that really important bug fix they had to deliver yep that's backed out as well because the whole commit has to get reverted so an easy little suggestion there is if you're going to do any refactoring in line I would highly recommend just separating out the commit from the work that you're doing then you're not coupling when you have to back changes out should something go wrong okay we also had some teams that were doing different types of estimation like story points or other things and when you have people that aren't totally alligned on what work has to get done because you have some people that are planning behind the scenes to go do refactoring work they know they're going to be cleaning things up it's really difficult to get alignment between different people on the amount of work involved now personally I'm not a huge fan of sitting down and spending a lot of time throwing out story points and trying to get the right estimate in place because I often feel the conversations are beneficial when you're talking about features but often times the amount of effort that goes into getting an accurate estimate is a total waste but for the teams that were doing this this ended up causing a lot of churn and a lot of time wasted because people can't really align on what needs to be delivered so just calling out that we had this separation and what's actually supposed to be delivered what's really a priority and the amount of work that was going to be going into particular features or bug fixes and the third thing I want to call out that is arguably one of the most impactful things when it's taken to the extreme case is this situation where people can't really prioritize properly the amount of effort that goes into sumary factoring now I know in an ideal case if you're working with engineers and their product owners what should be happening is that if some amount of refactoring is taking place or you're having these standup meetings and syncs to make sure that people are progressing you'd be able to catch something like a Rog refactor that's going on for too long I totally get it but I'm basing this information on real scenarios that happened and not just some hypothetical ideal situation now in some situations we'd have these more Junior Engineers that would be doing refactoring work alongside their feature development and sometimes as Junior Engineers they don't have a really good idea of prioritizing in time management so they would end up getting really stuck in going down rabbit holes trying to refactor things now if this kind of thing isn't caught in a standup or a syncup meeting it's really easy for this to go off the rails for that individual they might be spending a ton of time doing some type of cleanup or refactoring and the thing that they were assigned that should have been you know a small amount of effort is now just totally running over but it's not just limited to more Junior softw and in fact with the more senior ones I've seen this get significantly worse there are cases where more senior software Engineers have decided that they understand the priority of having to refactor something and it's the same type of situation where it's not necessarily caught in a standup or a sync meeting and they're kind of going down this path where they need to start refactoring and making big sweeping changes to the code base in order to support the refactor work that they deem is a priority now when this is finally caught on to what ends up happening is like look I've already spent so much time investing into this and we're really close we should just finish up this refactoring work and by the way you know that feature over here and this bug fix over here well they all really depend on this refactoring work so I should just move ahead and once I land this these other things can land or get scheduled as well but as you might expect things aren't always that simple because that refactoring work that gets pushed out yet another week there was something else that came up in the refactoring and we have to account for that too so yeah that bug fix and that feature yeah there's still waiting on this but the good news is there's yet another bug fix that once we land this refactoring it's totally going to be able to get Incorporated right into the code very easily after this and this pattern continues on now there are certainly ways that we can address this as it's ongoing but the whole point that I'm trying to make here is that I think when we jump into this behavior of let me just kind of ninja the fixes or the refactoring in let me boy scat some things if you're not setting some guidelines for that or you're just assuming ass uming that everyone has the same level of knowledge and experience to be able to do that effectively I think you get into some dangerous territory now I'm not a huge fan of necessarily like setting these hard and fast rules that every team has to take on because I've certainly worked with more senior software Engineers that are working in mature code bases that they spent a lot of time working in they probably can go handle some simple refactors alongside what they're doing and things more often than not will work out really well so it's not like a rule that I want to carve in stone but what I would suggest is that instead of trying to Boy Scout all of your changes in I would say try to get those prioritized alongside your features and your bug fixes and if you're hearing that and immediately wanting to resist it going Nick well that's never going to work the product owner just never going to schedule that we've seen it time and time again where we're saying hey we want to go refactor this hey there's Tech debt here and the product owner like too bad we got features and Bug fixes well this might be kind of tough to hear but I want you to think about about this the responsibility of the product owner is to help ensure that value is getting shipped to the customer and a lot of the times a product owner will not totally understand the value of the tech debt they will usually have a bias where as a product owner they're not going to be thinking that this Tech de here or over there should get prioritized over a critical bug fix or a feature that customers are asking for ultimately it's going to be up to you to help bridge that Gap now your product owner is interested in business value like I said their goal is to get value shipped to the customer so I would say more often than not if your product owner is not taking the time to schedule Tech debt it's because you're not able to clearly articulate what the value is of that Tech debt especially how it might align with the other work that's coming up so to give you an example consider an area of code that hadn't been touched in years you've gone looking through it and you said holy crap this code is disgusting and it looks like there aren't even really many tests here we really need to refactor this code to get it tested and make sure sure that it's more readable and that more people coming through here can easily start making changes without having to worry about it I mean that sounds like a really good opportunity for refactoring something if the code's that bad it probably needs some attention but being a good candidate for refactoring doesn't necessarily mean that it's a priority of work to do so why should your product owner go schedule that so for example if you didn't have any upcoming work that required changes in that area of the code is that really going to offer any benefit to your product owner if you spend time going to ref fact that likely not and I mean maybe you just had a lot of changes recently maybe in the last couple of Sprints you were making changes to this area of the code that hadn't been touched in ages before that and the side effect of that was that you guys were breaking things and you had bugs that were coming up and you had to you know spend extra time fixing these bugs and trying to play around with the code that might be a really good argument to your product owner to say look we had to spend so much extra unplanned time delivering changes there because it was was so brutal to work in and you know what you have two more features scheduled coming up that you really want us to land but we saw how costly that was if you give us a Sprint or whatever the time frame is we think that we could clean this up make sure we have better test coverage in place and that way when we go to add those new features in we're not risking going overboard with our timelines that might mean that we're pushing out the initial timeline a little bit but we have a lot more certainty in delivering the other work after because it will be tested and it will be a lot more clear for us to work through to me that might be a lot better of a sell to a product owner to get them to prioritize it but the reality is all that you're doing is enabling the product owner to have more information to make a better decision at the end of the day they might say look I still don't think that it's worth to invest the time into that and there's going to be situations where that happens there's going to be situations where a product owner is totally comfortable incurring yet more debt or taking on some of that risk but in my opinion the best thing you can do as a software engineer in these situations is make sure that you're providing as much information about the value and I mean the business value and how that's going to help deliver value to the customers when you're trying to articulate why you should be focused on Tech debt so to summarize my perspective on this I think that Tech debt is absolutely important to be paying down and that refactoring code is super important for us to be focused on personally I think that the way that we can accomplish this is by scheduling Tech debt alongside other features and bug fixes I think we have a responsibility as software Engineers to make sure that we're articulating the business value for why we should be spending time on it and it's not an easy thing to do we have to wear a different hat and think about our perspective as a product owner and how we would want to deliver value to the customers even though we're talking about something like refactoring code I think that boy scouting changes can work when you're trying to refactor things and clean up as you go I totally think it can work but I do think that people have this Vision that it's a lot more ideal than it is in reality so at the end of the day all I can encourage you to do is think about the pros and cons of these different approaches think about how your team can operate more effectively and ideally you're just focus on more collaboration when it comes to being able to focus on Tech dead so thanks so much for watching this video and if you're interested in hearing more about how my career ended up here and how I have this perspective on Tech de you can watch this video next thanks we'll see you next time