How To Approach Refactoring And Tech Debt – Dev Leader Weekly 19

Hey there!

I’ve been seeing a lot of discussion on social media about how to approach tech debt and refactoring, and I wanted to share my experiences with you. There are a lot of creators with large followings taking a stance that I think might be misleading more junior folks. So I felt like it was important to try and set the record straight on some of this stuff. This week’s exclusive article focuses on this topic as well as early access to a video I’ll release next week. But more importantly…

SOMETHING BIG is coming next week!

I’ve been working on a project for a little while that I’ve just wrapped up, and I’m so excited to be able to share it with all of you next week. Let’s just say this week’s newsletter topic is extremely timely. Please make sure you’re following me on LinkedIn and Twitter because I will be resharing the announcement once it’s made.

A quick reminder that you can refer your friends to sign up for my newsletter! You can get free membership access for a few months to my site to access the newsletter archive and my Behind The Screen weekly vlogs that I do. Check out the details at the end of this issue.

A Journey on Paying Down Tech Debt and Refactoring

I’ve Written a Ton of Legacy Code

Before I get into explaining my stance on tech debt and refactoring, I want to explain to you why I feel like I have a voice about this. Before my gig at Microsoft, I spent 8 years at a digital forensics software company. I started there when the company was very new as one of the original engineers and quickly took on a management role alongside my development.

The side effect of being there super early was that I was responsible for writing a ton of code at a time when the company was growing extremely rapidly. We had little to no processes in place and we were doing everything we could to ship value to customers as fast as we possibly could. Now let’s fast forward a few years. That same code I wrote in the early days of the company lasted long enough to become the legacy code that many of us had to go refactor.

And it’s worth noting that my role was a little bit unique — I managed software engineering teams but I was also still coding regularly (it’s amazing how much work you can get done when your only hobbies are work and the gym). This gave me a unique perspective that I understood the drive to want to clean up and maintain code properly, but I also was very focused on delivering business value.

In these next sections, I don’t want to make it sound like “this is the only way managing tech debt and refactoring works”. But I do want you to understand where I’m coming from when I explain these things. You can watch along here as well with this early access video:

Tech Debt Sprints

As I said, I get the desire to want to focus on tech debt and cleaning up code. We even decided we’d try out this thing where we convinced our product management team that we could move fast, but we’ll need to make up for it by having a tech debt sprint. This made the product folks happy because they’d get dedicated effort from the engineers for a period of time, and the engineers felt good because they’d get to hunker down and code to clean areas up that they wanted to.

But this seemed a lot better on paper than in reality.

We were a ridiculously fast-moving startup. Guess what happened when the tech debt sprint came around? You guessed it — any bug fixes and feature requests that could possibly randomize the team started to take priority of the tech debt cleanup. Engineers were losing time to the stuff they were told that they’d get some time to go clean up!

… But for good reason! These things coming up WERE a priority. We had critical bugs that were surfacing (we’re a digital forensics company, and something being incorrect in the data we produce can have SERIOUS implications). We had big features that needed attention to start working on.

It was inevitable that higher-priority things were going to come up. We truly did want to invest in tech debt, but we found we couldn’t pause the business to do it.

“Boy Scout” All of the Tech Debt & Refactoring

The side effect of the previous experiments being run was that some engineers started getting the feeling they’d never truly get time to work on tech debt. After all, if we carved out a sprint for it and they’d get taken off of it… What would ever get product folks to finally commit to doing it? They felt like it was a lost cause.

So the popular “boy scout” approach was taken more and more. And look, I want to be clear here — I love the concept of this but I want to show you why it’s not as cheery as you might expect in some software environments. Engineers started trying to perform their refactors alongside their bug fixes and feature deliverables instead. Again, this is something I actually DO like, but we need some boundaries which I’ll discuss later.

Now the result of this was pretty chaotic:

• We’d get bug fixes and features committed together with some refactoring to clean up debt. But either the bug fix (or feature) would introduce a new bug OR the refactor itself would break something. And big surprise — there weren’t tests in place to catch this stuff.
• We’d get programmers who would decide they could invest as much time as they want in their refactoring. If they didn’t have a project/product manager who could understand the technical details or other engineers to push back on them, it became a black hole. Deadlines would push out. Other teams would get COMPLETELY blocked on rogue refactoring.
• Some of the teams that were still doing story point estimations — good luck on getting anyone to agree on the amount of effort things take when you have people planning to sneak in tech debt fixes without articulating the complexity of that to the team in planning.

… But hey, they were “boy scouting” changes into the codebase!

How Do We Invest Into Both?

The theme continues here, but I’ll repeat it: it is very important to ensure you’re addressing paying down tech debt during your development. But clearly, we had some different systems that weren’t working. I feel like things started to change when we treated features, bug fixes, and tech debt as equal categories of things to at least consider working on, but required that we prioritized them against each other.

I think that this worked well because it brought us back to a common denominator: What’s the business impact of doing this work?

When working with your product owner who is going to be placing priorities on the work streams, this is what they care about. They’re going to have a bias towards customer-facing things (like fixing bugs and features), of course. But it became the job of the software engineers to articulate the business value of the tech debt.

You know that code that Nick wrote 7 years ago that really sucks? The whole module that would be hectic to try and write any tests on and is already lacking test coverage? You want it cleaned up because you don’t like the design patterns used there, the fact it’s not tested well (and hard to test), and Nick was even worse at naming variables back then. No problem! What’s that going to afford us?

Scenario 1:

• Well that code was written 7 years ago and last touched 4 years ago after a quick bug fix.
• We haven’t had customers reporting any problems with it. We don’t have conclusive proof, but it seems like it’s doing what it needs to.
• None of the upcoming work we have focuses on this area.
• The code is messy, largely untested, and it’s going to be a total pain in the butt to try and refactor it to make it testable.

What do we get out of improving this right now? It’s not reported to be causing issues for customers and we’re potentially destabilizing it because it’s messy and without test coverage. This probably has very little business justification at this point in time.

Scenario 2:

• Well that code was written 7 years ago and last over the last few sprints as we tried adding a new feature. It was challenging to do.
• We had customers reporting problems recently after we went in to touch it to introduce the new features.
• We had a few more bugs get introduced because the bug fixes we added ALSO caused regressions.
• We have another small feature coming up that’s going to need to touch code in that area.
• The code is messy, largely untested, and it’s going to be a total pain in the butt to try and refactor it to make it testable.

This has a much stronger case for business value. There’s evidence it’s brittle and has been causing issues recently. We’re about to schedule more work in there and likely run into the same types of problems. It could certainly be worth delaying a new feature to fix this up first… OR perhaps the combination of this tech debt cost and the feature we plan to add makes this LESS priority than some other feature.

Bridge Engineering and Product Owners Personas

Ultimately, I think the answer to paying down tech debt and refactoring code is not:

• Ninja the code refactoring in because nobody will ever give you a chance
• Never do any tech debt and let the code rot until you can just rewrite it anyway

Instead, I think you need to fix the problem of aligning engineering with product owners. Ultimately, the product owners are responsible for what’s getting focus for their product or service. Instead of engineers feeling like product owners are against them and hate tech debt, they need to collaborate with them. Find ways to demonstrate why delivering on tech debt will help with business value going forward.

I often acted as a partial product owner for my teams, but more often than not had to interact with a dedicated product owner. Not a single one said that they don’t value tech debt — in fact, they told me they DO want to schedule time for the engineers to work on it. The challenge was that they couldn’t possibly go find the areas themselves to address and they needed engineers to explain the value to them.

When both engineering and product owners aligned on similar values, it made these sorts of conversations significantly easier to navigate. That might mean that the big (or small) refactor you want to do just isn’t a priority right now — but at least you can understand better why compared to the other priorities.

Final Thoughts on Refactor-As-You-Go

Now, what I’ve seen online being pushed is something like: “You’re on your journey to be a senior dev. You know how senior devs approach refactoring? They refactor as they read the code”.

This advice could be pretty terrible for a junior developer who:

• Isn’t aware that sometimes even a refactor as simple as renaming a variable can break things (Hello, reflection!)
• Cannot properly prioritize or timebox their efforts (this was a small feature, but maybe I can refactor for an extra week or two)
• Isn’t familiar with the code (how much test coverage, how brittle, do you really need to be there?)
• Is already struggling with commit sizes that are too large (might as well bundle some unrelated refactoring in there too!)

The “senior” devs that can do this and do it well very likely have the experience in these areas to not run into such problems. OR… Maybe they do still run into these problems but they have broken collaboration with their product owners. I’m not against refactoring-as-you-go. But I think it’s not as trivial as some people make it. I think many could benefit from communicating such changes. I’m a strong believer that refactoring changes shouldn’t be bundled into other commits. And I think that unfortunately there are many situations where values between engineers and product owners are not aligned.

So you don’t need to replicate what I say, I just ask that you consider the facets of what I’ve discussed.

Quote of the Week:

Discussing tech debt seems to get both engineers and product owners into uncomfortable situations. We’ve all seen or heard of situations where the programmers want to address all of the tech debt but the product owners don’t want to schedule any of it. These two opposing forces can never seem to align when it comes to refactoring and cleaning up code, it seems.

But we shouldn’t run away from tech debt. There are ways that we can properly balance tech debt if we have both groups collaborate and work together on aligning what the overall priorities are. By taking this stance, tech debt can get stacked ranked with other feature/bug-fix work. If the business value is articulated clearly, it absolutely should get scheduled if it’s important.

I find one great driver for this is if you’re actively developing in an area or you’re about to be spending more time there. Being able to explain to your product owner that it’s going to be much more expensive to deliver changes into a particular area because of how the code is can be very powerful. Now, they might still be comfortable with living with the debt (or even incurring more). But if you can present them with such information and they understand the consequences of not addressing it now, then they can make informed decisions.

At the end of the day, our goal as software engineers is to solve problems to help deliver value to customers. Customers don’t care about our code. And realistically, our product owners only care about our code if it’s keeping us from delivering value. The code is not what customers are paying for.

Featured Content:

Quite a week getting through content! After wrapping up my big project as well, I feel like there’s been a bit of a weight lifted. More on that in this weekly vlog.

How Many Promos to Principal Level? – My Software Engineering Journey

This is the early access video from last week’s newsletter! If you haven’t checked it out yet, have a watch-through and learn about my career journey getting me to my role at Microsoft.

How To Implement The Observer Pattern In #CSharp

This video is a follow-up from the first in the series where I introduce another way to implement the observer pattern in C#.

Using System.Reactive (RX) For The Observer Pattern in CSharp

Did you miss this video this week? Well… you wouldn’t be the only one! I left this one unlisted and only realized while I was writing this newsletter that I forgot to schedule it! The focus of this video is yet another way we can implement the observer pattern in C#.

Remove Control Flag Refactoring – How to Simplify Logic

A refactoring technique to help get rid of control flags. Often control flags can lead to trickier code, especially in situations where they’re being assigned in multiple places or you have multiple control flags to deal with.

Exploring Examples Of The Mediator Pattern In C#

First, you might be asking, “Why so surprised?”. And while I don’t have a good answer, the reality is if it made you stop and pause then the thumbnail is doing half of its job. The other half of the job is right here when I tell you that this article builds on some thoughts from the observer pattern articles where we see a different way of communicating between modules with a decoupled approach.

What Is Refactoring And Understanding Why You Need It

Could you tell refactoring was top of mind this week? Just wait until you see what’s coming next week! This article is an intro to refactoring for folks who haven’t had to work through much of it yet.

How to Implement the Strategy Pattern in C# for Improved Code Flexibility

If you’re interested in learning more design patterns, then this is a great intro article on how to leverage the strategy pattern in C#.

How To Harness System.Reactive For The Observer Pattern

I linked the video above, but this article is a companion to that video. The RX package has so much cool stuff to try out and I really owe it more time… But here’s a good intro to it if you haven’t used it yet!

When To Refactor Code – How To Maximize Efficiency and Minimizing Tech Debt

And another article dedicated to refactoring. It’s a seriously important skill that investing in will pay dividends later. I’m sure there’s a clever joke to make about dividends and paying down tech debt but… Well… you get it.

Solution to Last Week’s Coding Challenge:

Did you try out last week’s coding challenge? You can check out an interactive solution here! This was more of a data structures and algorithms kind of question — so you weren’t necessarily building a full application.

This Week’s Coding Challenge:

Weekly Coding Challenge: Implement a Simple File Compression Tool in C#

Challenge Overview: This week, let’s step into the realm of file processing and algorithms by creating a basic file compression tool in C#. Your task is to develop a console application that can compress and decompress text files using a simple algorithm of your choice.

Problem Statement:

1. File Compression:
   • Write a method to compress a text file.
   • You may choose a simple compression algorithm, such as Run-Length Encoding (RLE) or any other algorithm you’re comfortable with. The goal is to reduce the size of the input file by encoding repetitive sequences in the file efficiently.
2. File Decompression:
   • Write a method to decompress the files you’ve compressed, restoring them to their original state.
3. Console Application:
   • Create a console application where users can input the path of a text file and choose to compress or decompress it.
   • The application should handle errors gracefully and provide meaningful messages to the user.

Example:

For a simple RLE compression algorithm, the string "aaabbcdddd" would be compressed to "3a2b1c4d".

Constraints:

• The focus is on text file compression. Consider how whitespace and special characters should be handled.
• The decompression algorithm should perfectly reconstruct the original file from a compressed file.
• Pay attention to memory usage, especially for large files.

Method Signatures:

public class FileCompressor
{
    public void Compress(string inputFile, string outputFile)
    {
        // Implement compression logic
    }

    public void Decompress(string inputFile, string outputFile)
    {
        // Implement decompression logic
    }
}

class Program
{
    static void Main(string[] args)
    {
        // Implement console interaction logic
    }
}

Tips:

• Start with writing a robust algorithm for compression and decompression before integrating it into the console application.
• Test your application with different types of text files to ensure reliability.
• Consider the trade-offs of your chosen compression algorithm in terms of efficiency and complexity.

This challenge will give you an opportunity to work with file I/O, explore basic compression techniques, and further develop your problem-solving skills. Enjoy coding!

As always, thanks so much for your support! I hope you enjoyed this issue, and I’ll see you next week.

​Nick “Dev Leader” Cosentino
​[email protected]​
​
Socials:
– Blog​
– Dev Leader YouTube​
– Follow on LinkedIn​
– Dev Leader Instagram​
​

P.S. If you enjoyed this newsletter, consider sharing it with your fellow developers. Let’s grow our community together!

Nick Cosentino Principal Software Engineering Manager

Principal Software Engineering Manager at Microsoft. Views are my own.

See Full Bio

ASP.NET C# DotNet .NET Software Engineering