Technical debt is not debt; it’s not even technical(markgreville.ie) |
Technical debt is not debt; it’s not even technical(markgreville.ie) |
and I say that as someone who doesn't much like analogies or metaphors for most code management issues as I think they often obfuscate the problems more than they illuminate.
Why do we create encapsulated build setups if everyone from the team knows how it install it on their machines anyway?
Why do we waste time on writing documentation?
And above all: if during a three-day hackathon we can create a game, why does it take months or years to create a publishable one?
getting it to walmart is where the chefs get bored and turn one amazing night into years of backlog refinement
I think the term “technical debt” is useful in the sense that it can do a good job of conveying the gist of the consequences of this mismatch that matter to people outside the development, without the above esoterics which I’d do a poor job at communicating (without confusing or alarming my target audience, anyway).
Its definition of "debt" is too strict. Sure if you take on a mortage or a bank loan, then principle, interest and term will be well defined and enforced. What about when you borrow twenty bucks from your friend or they help you move?
Arguing that technical debt is not technical because it has a broader impact doesn't really seem very important. Calling it "technical" because it originates in a technical part of the company and while building out tech makes perfect sense to me. The fact it can have a broader impact doesn't seem very important when naming it.
I really like this kind of epistemological analysis on tech terms, specially those that we use all the time and that we think we understand, but mainly for going deeper on a topic and analyze.
"what are the 2 hardest things in technology?"
"Cache invalidation, naming, and off by one errors"
Source code is the expression of knowledge about a problem that happens to be machine readable.
Test code is the expression of knowledge about how things can go wrong solving the problem, and it happens to be machine readable as well.
I agree that a sad aspect to this is the use of a money metaphor. Once the Silicon Valley types hear a finance term, they lock on to it as something to be optimized, and deployed as a tool. It leads people astray.
It rarely costs less to delay tech work. It often costs more. The "moral force" of the debt analogy is strong.
Tends to happen at the worst of times when you're least prepared.
TD is any code/architecture aspect to which PMNOPML applies.
As with gravity, it's going to suck whether one fancies the label or not.
"Technical debt" is good company here. Popular speakers, thought leaders and the like are rather keen to coin new terms but not to define and delineate them rigorously.
(Like for "microservices": Let two engineers count the number of microservices in a larger system. The will surely not arrive at the same number.)
It's remarkable how much substance there is to the technical debt metaphor.
1) There is good debt and bad debt. It's only good debt if it buys you an asset that creates a value stream.
2) There is compounding interest over time on technical debt.
3) You can metaphorically go bankrupt from technical debt. (IE total project failure and or abandonment)
The fundamental debt metaphor was about the need for ongoing re-factoring of past code and systems to match today's understanding, and the accumulating cost of failing to have the discipline and taking the time to do so.
Cunningham's specific quotes from the video [0] linked in the article: "...accumulate the learnings we did about the application over time by modifying the program to look as though we knew what we were doing all along, and to look as if it had been easy to do", and "If we fail to make our program align with what we then understood to be the proper way to think about our financial objects then we were going to continually stumble over that disagreement, and that would slow us down".
This problem is fundamental to software systems and how they change over time, and is not about making poor choices, tradeoffs, or generally poor engineering to ship code faster.
A system or feature is initially built based on the requirements that were known at that point in time. Requirements are frequently not static though: that original set of requirements may continue to evolve over time, either as we better understand them after implementation, or when requirements are added, refined, or removed by the stakeholders or organization.
Additionally, we can't view each feature or requirement in isolation over time: only when we see the overall set of features in a system will the important patterns start to emerge that inform how refactoring should occur. We can't know this early on, only over time as the system evolves.
I have witnessed this many times on software projects, both small and large. Requirements and features evolve over time, but the code base was never refactored to take into account the overall understanding as of each point in time. Rather, the changes are applied in isolation ('bolted on?') incrementally. Over time these numerous incremental changes result in a system that is fragile, increasingly difficult to understand and maintain, and generally veers toward spaghetti. It becomes harder and harder to make changes. That is the debt that was never repaid.
For my part, I use the term sparingly since I find I can easily use it as a catch-all thought-terminating cliche. That doesn’t mean it is useless. Just that it’s bad for me to use it.
People around me know to not let me get away with saying that.
What? The US consumer debt industry would beg to differ.
I was hoping that maybe they’d extend the concept to a liability or something. In general the idea is that you’re borrowing a resource from your future self to accelerate a business outcome now.
I've only ever seen these terms used in programming contexts
But the result is just that people outside technology think technology is clueless and "they have debt, whatever that means".
> Cunningham: I’m never in favor of writing code poorly, but I am in favor of writing code to reflect your current understanding of a problem, even if that understanding is partial.
Technical debt is not about technology, but about better understanding of functional domain, after which you should refactor your current implementation to better reflect reality in the sense that there is a more clean abstraction and accompanied code.
No abstraction will save you from this.
"We have technical debt to repay" = bad
"We outgrew the choices we made that allowed us to grow fast, and need to refresh our product to sustain the growth rate" = actually means what we want to say
Makes it clear that prioritizing new features will come at the expense of growth. Which is ultimately what the stakeholders really care about.
"Technical debt is the burden that implementation choices may place on future development."
Nothing in the article leads me to think that this is either incoherent or irrelevant.
With regard to its coherence, I say that it is empirically verified: it actually happens!
The claim that it is not technical because it has consequences beyond the technical is a non-sequitur. Just about everything we do that is 'technical' has implications and consequences that go beyond the technical: The mere fact that most of the technical things we choose to do are done for non-technical reasons is enough to establish that. The root cause of a problem and its broad consequences are separable concerns.
This formulation avoids the objection that its only debt if it is paid back in the same form as it taken out, but that was a tendentious claim anyway - it is completely reasonable to consider the situation where, say, I give a farmer cash in return for some produce at harvest time to be a debt. If we are going to be pedantic, the author might as well say that technical debt is not debt because it is not an exchange of money. Arguing over dictionary definitions is not going to lead to anything useful in this matter.
A little debt speeds development so long as it is paid back promptly with a rewrite. Objects make the cost of this transaction tolerable. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt.
>> Cunningham disliked the notion that technical debt signified “doing a poor job now and a better one later.” This was never his intention.
You can split hairs of weather code was poorly written vs written from an underdeveloped understanding of the problem, but the effect is the same. People have to deal with suboptimal design decisions repeatedly until they are corrected. To extend the debt/interest metaphore, these poor decisions also undergo compounding if not paid back.
I've always assumed engineers (myself included) write code to the best of their abilities to solve a given problem at the time. But we all know that, over time, features and such get added to the code base and it is stretched beyond what it was originally meant to do.
That the code base would should be refactored to better accommodate these new features that have been glommed on is the tech debt. There was nothing poor about the original implementation and the additional features were added with the understanding that the code could not be rewritten from scratch.
I may be splitting hairs here but I want to make the distinction that at no point was an engineer making poor choices.
I want to point this out not to sound defensive, but rather to make it clear that when I use the term "tech debt" I am not saying it in a castigating or "accusational" way. There is no shame in tech debt.
(BTW, other engineers I've worked with, will add a "BOGUS" comment next to any line of code they are in fact embarrassed by with some explanation like "using very small number here, render crashes if border is zero thickness.". Do a search for "BOGUS" in the sources to discover you BOGUS debt.)
Aha! But people do things just for technical reasons that don't affect the users in any way.
One could claim that just refactoring code without changing anything is a "purely technical" exercise.
What I've seen developers spend time on, that has absolutely no effect on end users in a large C++ codebase:
* Changing old C-style string handling to std::strings
* Change naming convention to uniform
* Random formatting that should be done automatically but someone decided to do by hand
* Sundry "modernizations" like replacing vanilla C++ loops with C++ range based for loops etc
Improved maintainability => lower innovation costs, and those savings can turn into profits and/or happier customers.
Improved security => reduced risk of exploit, and that improved posture can be turned into profits and/or happier customers.
I acknowledged that not all changes of the sort you list do yield actual improvement in the quality attributes.
The reason is still partly non technical; the goal is to make future changes less costly or less dangerous.
Nevertheless, people doing things for just technical reasons does sometimes create technical debt, as when doing so over-complicates the implementation to no observable end. So? If anything, these cases stand as counter-examples to the author's argument that technical debt is not technical.
True, provided we understand that those choices are not always conscious. I more commonly use the term looking back: some code technically works and at least minimally satisfies current use cases, but is hard to read and therefore hard to maintain or extend and bug-prone, so it needs to be refactored, improved, and/or modernized.
Before the term "technical debt" existed, it could be hard for developers to convince management of the need to clean up existing code. It sounded like just some thing that crazy programmers wanted to do - spend time and money for something that delivered nothing new. Now managers can understand the need to pay off "technical debt."
Interesting. I wonder if there is an analogy to the first draft of a book. Everything needs "editing" -- that is a given. Some writing only needs line editing; some needs a restructuring, a rewrite, etc.
> With regard to its coherence, I say that it is empirically verified: it actually happens!
Something happens, on this we can agree, but if you are making checkers and find out you need chess, it isn't that making chess is any harder than it was before you did anything -- so no, I would not agree that it placed any burden on future development.
No, I think the real issue is that you have already paid for checkers and so you feel unhappy that your time (and maybe money) was wasted on checkers when you simply can't sell checkers because all along you needed chess.
> The mere fact that most of the technical things we choose to do are done for non-technical reasons is enough to establish that. The root cause of a problem and its broad consequences are separable concerns.
The root cause is that you do not know what you want. Writing the crappiest version of checkers as quickly as possible is a great way to find out it is the wrong thing. Trying to write an "abstract board game" instead of just writing checkers so that the decision to choose checkers or chess could be delayed as long as possible is not: It assumes that it is mere time that informs us of what we want, rather than the feedback of seeing what we don't want.
> it is completely reasonable to consider the situation where, say, I give a farmer cash in return for some produce at harvest time to be a debt
We can fantasize about a great many things, including this, but software is not produce that is harvested and sold; you do not plant great software seeds to grow into great software trees. No, the debt is the cash that the farmer received, just like the debt is the cash you receive to write the software. The software itself is not debt and never was.
it can just be building a minimal chess quickly using methods/frameworks that are not easily extensible to future requirements related to chess.
the vast majority of software projects will have features/requirements that evolve after the initial design, it is a rabbit hole to try and guess all of those, but some things are just common sense but would add a little time to the first milestone. then you are in the position of tacking on things indefinitely making it a horrible tangled mess.
If you incur technical debt as a mismatch between your codebase and your requirements (owing to the evolution of requirements) initially you may just be able to work around the mismatch, but it starts to incur "extra" work - this has been called the "interest" in the extended metaphor. Everything you do that isn't addressing the mismatch keeps adding to the debt though, so eventually the "interest" becomes equal to your entire budget and no more forward progress can be made, and typically bankruptcy is called (give up on code base, propose rewrite to address the actual requirements) and the cycle starts again.
Instead what happens is they say "Well, build what you can for $1m", and you say "Ok, we can make 'a bridge' for that", and then either (a) your infantry can cross, but the tanks have to get diverted 20 miles out of the way, or (b) the tanks end up in the river along with the bridge. Since (b) is bad, you then have to spend a lot of time planning the routes for the tanks, and making sure the tanks have the right air cover, etc etc, i.e. doing more work. More likely, however, is that the manager, who is not a structural engineer, sees a perfectly good bridge and orders the tanks across anyway, causing the loss of the bridge, the tanks, and the war.
It's not debt. It's just (at best) an incomplete solution or (at worst) a bad solution that fails at the worst possible moment - e.g. database collapses during registration for the largest event of the year.
Ah, but surely, if you build the lightweight solution for $1m, and acknowledge the increased costs of managing the problems that it doesn't solve, then thats fine? Sure, but that's not technical debt either! That is scoping: we (engineers + business) identify a workable solution that provides some business value. And then we do that well. When Cunningham, for example, talks about what to do about debt, e.g. YAGNI, these are all good ideas: scoping. But the term "debt" is incorrect in this case. The term "debt" will only ever get a team in trouble as it indicates that the team is unwilling to confront the actual problem as engineering, but only as a broken metaphor.
Well, those are two possibilities, but a third is that you don't need it because of one of several possible reasons.
The debt metaphor still fails to hold up; the nice thing about accepting "technical debt" into your project is that you may never come upon a time where it needs to be paid off at all. Not all tech debt must be paid off.
Usually only the highest priority tech debt can actually be paid down, and it doesn't always come with an interest cost. But when it starts to "come due," if you're an engineer in the hot seat, you can often (usually) tell pretty quickly just how much the "interest payment" actually affects your ability to deliver future iterations.
Your "customers" or stakeholders can tell too, indirectly, when your estimates start outpacing their "very reasonable expectations."
Any choice you make now, may place a burden in some possible future.
> If we are going to be pedantic
The whole article was about going to be pedantic over dictionary definitions.
"Now, one major concern in academia is rigor. Academics like to get deep into a topic, examine the nuances, and bring clarity."
I suppose it's hard to argue against that words in "technical debt" were chosen poorly: it's not debt and it's not technical.
Maybe better definition is needed.
Being entirely-too pedantic about financial instruments: that’s a commodities future. Financially, it’s not a liability, it’s an asset: you’re owed something. It’s not really a good comparison to technical debt, which is at the very least clearly a financial liability of some kind — something that makes your company worse-off for having it.
I think the author of the article, meanwhile, was interpreting “debt” to specifically be shorthand for “a loan taken out against future earnings.” Loans a company takes out give them both a short-term asset (cash flow) and a matching long-term liability (loan repayment.) Which seems kinda-sorta like what “technical debt” is motioning at: you get short-term gains (working code) but a long-term liability (the need to rewrite said code when its lack of robustness or unmaintainibility causes it to break down in the face of scaling challenges.)
But this analogy is still pretty far off-the-mark, because:
1. You need to pay back a loan, or bad things happen to your company; but you won’t necessarily need to pay back technical debt — sometimes, despite your company scaling as a whole, the relevant software component just never needs to scale, and so it that particular technical landmine just never “goes off”;
2. loan repayments are predictable, and involve entering into contracts where you either conscientiously repay the debt month after month with low premiums, or you fail to make payments and the loan comes due. Technical debt is nothing like this; there’s no counterparty forcing IT companies into a default situation of paying down their tech-debt little-by-little. (Wouldn’t that be great?)
Thus, a loan against future earnings is likewise a bad analogy for what technical debt “is.”
Dredging around in finance for a more exact analogy for technical debt, a very close one might be writing and selling call options against your own company, where your share price hasn’t yet reached the strike price for these options. You would be raising short-term capital, but creating a future liability — in that when your share price hits the strike price (i.e. when your company grows), people would then have a growing incentive, commensurate with the growth in your share price over the strike price, to exercise their purchased call options by buying at the agreed strike price; and then some of those people — an unpredictable subset of them — would turn around and dump the resulting shares on the market at market value, lowering your share price.
IMHO, the analogy is pretty okay because both scenarios are the same sort of “setting up a bomb for yourself that may go off at an unspecified time into the future, but only contingent on your company being successful, where you’ll have more resources to deal with it” problem.
But on the other hand, you can’t really defuse having sold call options, in the way that you can “pay down” tech debt. (I guess you could try to buy back the call options — but the holders are under no obligation to sell them back to you. Whereas tech debt has no such willful resistance to being fixed.)
So even this analogy, though better, isn’t a perfect one.
Truly, though, although we can’t say we know what technical debt is specifically, we can probably be pretty explicit by just generalizing the term: all of what we call “technical debt” is in fact some kind of financial liability (in future engineer-hours needing to be spent.) So perhaps “technical liability” is, although vague, the best we can do.
But I think you're reading too much into the terminology. The concepts being discussed here really don't have anything to do with finance.
B) article points that you're accumulating debt to get speed to market or because of incomplete knowledge of the market at some point and uses this to say "it's not technical because it's business". But it is technical because, well, it is issues with your technology. That you borrowed on your tech stack for business reasons doesn't make it less technical
C) ironically I'm arguing on semantics because I find the title to be semantically wrong. In fact I largely agree with the point of view, tech debt is the result of a strategy, be it that you want speed or low cost. Sometimes it makes sense for the business, and as long as the strategy is conscious and explained, and that we're clear with the consequences, I'm fine with it. In my experience most of the time it's not an assumed strategy and we dont want to deal with the consequences.
Article says: "Debt repayment has three properties that are straightforward to grasp — principal amount, interest rate, and term [...] when comparing technical debt, there is no agreement on the principal, [...] there is no concept of an interest rate [...] , term length isn’t a fixed concept"
However, there are generally accepted meanings of the those terms for the technical debt:
- The "principal" is how many things is wrong with the code. It could be measured in features ("We need to implement unit tests and database layer to clear technical debt") or in time ("it will take 6 FTE-months to clear our technical debt")
- While "interest rate" is rarly used, "interest payment" is common -- this is extra time spent implementing the new feature. One can say: "This feature would take 1 day if we had database layer, but because of technical debt we own, it will take 4 days instead". In this case "3 days" is the "interest payment"
- Not all debt has fixed term. "revolving debt", like a credit card, has no term at all. Technical debt is like that as well.
(Of course getting the actual values for "pricipal" and "interest payment" is very hard and no two people are likely produce the same estimate for them. But even if don't know the values it does not change the fact the the terms are defined well -- so this is very much a "debt")
As for "not even technical" part: I am going to argue that everything indirectly affects competitiveness, costs, customer satisfaction etc.. A leaking roof will decrease morale, decrease development speed and can even kill the company. Judging by consequences does not really bring anything useful to the table, so if you want to qualify the term, let's use originating action. "Technical debt" if we don't have any tests. "Financial debt" if we took money from the bank. "Organizational debt" if we are not creating the positions we need, and so on...
It is literally a debt. I dont know what the article is trying to say, it sounds to me like something my boss would say after a 2 hour meeting in a delirious caffeinated state, but thats probably just because I am not familiar with the corporate use of the term "technical debt".
Technical debt is not corporate debt but it is debt. The metaphor succeeds mostly because it's accurate.
This really is NOT how you start an article. You only come across as needlessly assertive and arrogant.
> Nobody explained technical debt; we assumed it was a fundamental property of the work.
Total BS. Literally every manager I ever addressed in a sentence where "technical debt" was mentioned did question what exactly it is and why do we need to address and "repay" it. Where is the author of the article living? Not with us the programmers here on the ground, that's for sure.
> Now, one major concern in academia is rigor.
Ah, so now it's about academia and its definition. I'll cut him a deal. Bring your academics to my former customers and see if they can override management's idea of a technical debt. Succeed 5 times and then I'll bow to you and accept your definition. Until then you're just an annoyance like that guy on parties who is always going around telling people "well ACTUALLY you are using the wrong term".
Doesn't matter what the dictionary says, people. It matters what most of the people think a word means. It matters what most of the people do when faced with the word. Why is this so hard to accept for many?
--
No, I haven't read the entire article. It smells of intellectual elitism and arrogance a mile away. The author must work on his tone.
Technical debt, whatever your definition of it is, is still a natural property of tight schedules. That's it. We can all go home now.
I wouldn't expect an academic to understand realities of schedules and budgets though.
The leaning Millennium Tower in SF cost over >$100m dollars to fix. We call that a mistake or an oversight, not "engineering debt"
In software we'd have called it "technical debt", as if spending $600m to build a system that doesn't work and needs to be fixed at tremendous cost while wiping out all of the profits was somehow part of our business strategy all along.
1. Is "technical debt" debt: If it walks like a duck...
Not all debt is in the form of retail loans. Leaving a "IOU" note where someones lunch was, is a form of debt (a very dangerous one). Good luck trying to find out your interest rate in advance.
2. Is "technical debt" technical: If we follow the author's logic, I guess now financial debt is not financial. Financial debt will also affect:
- Competitiveness by slowing/speeding up new product development.
- Costs (short-term decrease/long-term increases in development cycles)
- Customer satisfaction
- Whether a company can survive
Because it is inextricably tied to career ambitions. There is no promotion benefit to just "paying back" a small debt by fixing some kluge, while there is enormous benefit to leading a rewrite.
Furthermore, once a rewrite is in play, the temptation to "just fix" a bunch of other things and add some new features that were hitherto impossible is irresistible. Perhaps the debt repayment gets added to Version N+1.
You can say "oh, it doesn't have to be that way" and point to some glorious time when it wasn't, but in my experience, at least, it always is.
Spoiler, we never have time.
"What are your thoughts on managing technical debt across the organization?".
His response:
"I don't know what this technical debt you refer to is". Then he proceeded to answer the question as if I had asked about financial debt. This is the same person that allegedly said "Coding isn't hard, it's just typing!" according to some of the graybeard sources that worked with him when he was still a technical contributor.
I agree with the premise of this article that top decision makers at a company need to be familiar with and actively monitor technical debt (or whatever you want to call it). It can have disastrous consequences for the company. I see it every day.
Debt has a negative connotation, where as it’s more like a mortgage.
I always wonder if there’s an equivalent term in manufacturing.
Debt in business is a well-known and appreciated way to get things moving now at some additional cost later.
This applies to both financial debt and technical debt. The difference between those, and hence the name, is how you incur and repay it, not how you make the decision. It's not "business debt" because you can't repay it with "business".
If I were to imagine "business debt", it'd be making decisions now that work well for the moment, but won't work well in the future and will actual retard the business for a while later. You could do it the "right way" now and have fewer customers up front, or you optimize for now and have more customers now, but make it harder to get customers later.
Likewise, financial debt gives you more money right now and costs you more money later. Technical debt gets technical stuff working right now, but takes more time from your tech team later when you need things to be different.
I consider it a coefficient of development friction that must be managed to be an appropriate level for your strategic (business and technical and other) goals.
You want it as low as possible, but bringing it that low costs development effort that could be used on other efforts. Too high and simple changes take days and steals from productive work.
Greenfield development has a near-zero coefficient, which is why it is so fun :)
Author: this has no clearly defined principal and interest, so "owing" is not a word you should use
Someone: ... Never mind
Sometimes startup founders have very little management experience and hire others who have very little experience, you could call that management debt.
Taking on debt of any kind should be done strategically with a clear awareness of the trade-offs associated with it.
There's nothing intrinsically bad about any of the types of debt.
I think the most useful mental model is more risk-oriented. Think of the business as a portfolio of risks. Adopting a zero technical debt strategy reduces the risk of unexpected technology failures, but increases the risk of competitors coming to market sooner, etc. Successful businesses are the ones that navigate and make choices that turn out in hindsight to have been smart about the allocation of that risk portfolio.
In my view the relevant case study for technical debt is that day years ago when Facebook's homepage malfunctioned and showed the PHP source. This revealed some pretty ugly code that had apparently been written by Zuckerberg. Facebook already had a valuation in the billions on the basis of code that would not pass code review at most startups.
If it were this easy you wouldn't see so many people taking on so many unnecessary debts. Technical or otherwise.
Chiefly the answer to the question "how will this amount change if the interest rates go from effectively zero to 6%, which is the 30-year average?" is not straightforward.
Perhaps technical liability would be more accurate, but I like the sound of technical debt.
There is always the temptation to say, 'We have made something. It works. Pure asset.' (E.g. concept of passive income.) But it is seldom true over the long term. The debt is almost always there, or will be as the rest of the world moves on and the existing asset needs to be developed or becomes obsolescent.
The trick for asset owners (asset defined as the right to future economic benefits) to profit from an asset is to spend some of the returns from the asset on paying others to maintain and refresh the asset. In this way, the asset owner enjoys the truly passive net benefit.
Infrastructure carries both the attributes of a static, tangible thing (a good) and a service.
I like that the coding world is so conscious of technical debt, and it makes sense: the sector is moving so fast, and the fixes can be applied with access to a skilled problem solver, a computer, access to software and time. The maintenance &/or switching costs are comparatively low in comparison to the benefits.
I think that providers of physical infrastructure could pay more attention. Maintenance contracts and dilapidation reserves do make (small) inroads into acknowledging the ultimate sclerosis that additional large infrastructure introduces to the built environment. But the time scales for its regeneration to a large extent alleviate developers from the obligation to apply a more overarching perspective of the debt it will place on future generations to improve on what has been laid down.
Rather than questioning the term, it should be rolled out more widely.
If you're an experienced (or even somewhat inexperienced) developer working on projects of even minor complexity or in a domain you don't quite understand, you know exactly what technical debt is, and probably have plenty of it.
Perhaps it's best for everybody involved if we define "hard technical debt," which meets the highest criteria.
1. The interest is in measurable engineer-time (not risk, not user-experience, not sales, not satisfaction). E.G. "Every time we deploy it takes the scripts 30 minutes to run, and I can't switch focus for those 30 minutes"
2. The capital is in measurable time. "It would take 15 work hours to fix that script, with planning, debugging, documentation, etc"
With those two, we can calculate the annual interest rate at 173% (at 1 deploy per week).
This would help separate people who love doing rewrites because it's fun to put their name on things from actual "this is wasting my time every day" concerns.
In the best case people embark on well-informed yet insane ”version 2.0” over-engineering binges. This can be nice, but is often a waste of time.
Quite often the ones who cried “muh technical debt” in every meeting simply did not understand the problem being solved by the debty code, and end up making something novel but equally problematic.
In the worst case, you have a combination of ignorance and best-practice-hysteria that gets applied to actually-fit-for-purpose code. Then the 200 lines of performant and correct code that takes two hours to fully understand, gets turned into 2000 lines of deeply nested class hierarchies mixed with immutable FP-wank that takes several days to get a grasp of and both fails to handle real-world data (“not in the specification!”) and tanks system performance (“premature optimizations!”).
It's called second system effect and it's relatively easy to avoid by methodically documenting original and mercilessly avoiding adding features.
And this includes design features that are not immediately shown to save a lot of time and code. Every such feature needs an advocate and also a prosecutor, and alternatives have to be briefly explored using a prototype.
The worst mess is actually trying to pay off the "debt" in installments by taking on new one. Then running out of time or shifting focus, ending up with double the problem and half the readability.
Seniors point out specific issues with specific consequences and lay out specific actions to resolve/mitigate. The term “technical debt” isn’t usually needed in that case.
Technical debt terminology is a fig leaf for “I dont understand and I wont put in the effort to learn”.
There is no need to do a 2.0, and the number of lines of code will often drop. Though that depends on how much edge case handling code was skipped in the debt version.
https://news.ycombinator.com/item?id=27720078
FTA:
Debt repayment has one vital characteristic: it is easy to understand. Debt repayment has three properties that are straightforward to grasp — principal amount, interest rate, and term (i.e., length of time to repay).
I think the author is making an argument that it isn't really debt based on a failure to understand debt.
Actual debt isn't really somehow magically simpler and more straight forward than technical debt. It's kind of a bet that taking this hit now will pay off enough that it will be worth it.
It's a context based decision that compares a hypothetical path over time and guesses what will get the greatest overall benefit.
There is a saying in the military: Sometimes, a 90 percent solution now is better than a 100 percent solution later.
Life is a series of bets always based on partial information where we try to navigate our way through this one-way time travel adventure where we all move inexorably into the future minute by minute without knowing exactly what will happen. Choose poorly and things can really go to hell. Choose wisely and you can see life get better.
There's always some luck or randomness involved. Whether it's a good decision or a bad one will depend in part on if the future takes the direction you think it will.
Actual financial debt is no less complicated than technical debt. Someone who is more programming literate than I am could probably come up with some equivalent mental model for how technical debt gets repaid: "There are three components to repaying technical debt..."
The real issue in both cases is the question "Is it good debt or bad?" In other words, (for technical debt) did this decision to do something quick and dirty provide more value than it will cost to clean up later?
For money, it's theoretically easy to put numbers to the road not taken. In practice, it's a lot harder.
It's relatively easy when it's a hypothetical model. Real life has other factors that get hard to fully account for, especially if you have to justify it to someone else whose views of what is pertinent may differ dramatically from yours.
I think the key here is indeed that technical debt should not be used as an excuse to write poor code.
But I think it may be hard to systematically tell apart code that was written intentionally fast and poorly, and code that just was written to explore the market.
It "is accepted" that the most of the cost a software comes from the maintenance period - adding bugfixes, new features, and just reading the code.
If the software can be moved from point A where these costs are high (including risk of things getting broken due to complex architecture) to a point B where they are lower, then I could say "technical debt is paid back".
Was the software in point A due to intentional shortcuts, or just because that's how we write software in an unfamiliar territory, is maybe besides the point in gauging the practicality of the term.
The "technical debt" gets anyway accumulated due to the empirical quality of most of software engineering process.
I think this is the most critical thing people should realize - don't take short cuts unless the house is burning down - but do try to explore the problem space fearlessly.
The exploration is due to incomplete understanding of the end user needs. And sometimes you can't really explore that space without delivering something to the user.
But!
I think the exploration phase does not necessarily need to incur technical debt if it's done using prototypes that won't end up in production (yeah yeah like that's ever happened but in theory...).
In this case the deliverable will be just used as a template for the actual product development, with the result that the product may be in a more mature state from the get go, and hence, have less debt from the start.
Should the exploration be done with prototypes or actual vertical slices of production software? I suppose this depends on the situation. Sometimes the problem is actually so hard that you need to deliver a build based on an actual production codebase no matter what.
It is important to occasionally have arguments to clarify semantics so we maintain the meaningfulness of our language. Otherwise, we lose the ability to communicate.
Some people believe that meaningful communication is unimportant. Those people are wrong.
Statistically and historically speaking, he's waging a losing battle and I find myself not interested about what he believes technical debt is.
Furthermore, what academia thinks of technical debt is seriously irrelevant. Academics don't write code for a living. Their take on this matter isn't super important.
Even if you don't touch the code at all, it loses its value.
- Developers forget why did they make particular choices, and what were the implications.
- Technologies do age, and there will be maintenance cost for updating that.
When people dont understand or agree with a model, then you get these conflicting perspectives on what constitutes correct or incorrect expression.
In some simple cases it’s clear that there is a problem, or debt, when there is a clear semantic violation, a code smell or similar. But often there are multiple ways of expressing the same.
Quantification of unnecessary complexity can only be done after you know what is necessary and how you express that.
Nit: Not the entire 3 days is interest in this analogy because you are also paying down principal.
For example: "Implementing the database layer properly would have taken 5 days; because we took shortcuts (for business reasons) it only took us 3 days. Now we have to implement feature X, and because of our previous shortcut feature X will take 4 days, so 3 + 4 days in total. But if we had implemented the database layer properly in the first place the new feature would have only taken 1 day, so 5 + 1 days in total."
Then in the three-day payment, two are towards principal (that's how much you "borrowed" previously), and one is towards interest.
I didn't read the GP as saying that they were paying down principle. They don't say what the hypothetical feature is, but my assumption was that it's a feature that they could implement without a database layer but that would be faster if they had a database layer. That is, I assumed that even after the 4 days of development for the new feature, they still won't have a db layer – which is why the extra three days is pure "interest".
This is why it is important to refactor dangerous system deficiencies sooner. If tech debt takes 2 weeks to clean, one can always squeeze that into the plans. If tech debt takes 1 year to clean, it is nigh impossible to persuade the business give that much time.
The project accrues technical debt.
And it owes it to the users.
Because, ultimately, that's who all these FTE hours (the currency borrowed!) are going to.
By taking a shortcut now, you make an implicit promise to the user to rewrite things later (on behalf of the project).
It's the user that suffers if the debt is not repaid (by experiencing bugs, missing features, slow releases, paying for higher headcount to fix bugs, etc).
Shortcuts don't become debt unless the user is affected. If it's stupid and it works, it's not stupid.
And it is the user that will dump your project if that debt is never repaid.
The debt analogy holds water.
Ideally, a dev team delivers feature value at a particular sustainable rate. Think of that as your product team’s ‘income’. Now, if the product team needs a feature early, they can have it if they take out a loan - they can have the feature built without sustainable integration into the codebase - but it will reduce their effective income because future feature delivery will be slowed by the friction of the incurred debt - the interest payments.
Don’t let the product owners make you feel like you have to deal with the burden of debt they took out! They owe you the time to clean up the mess they demanded to get their features faster.
Regardless, technical debt was always clearly a metaphor. I can imagine many other metaphors where one may owe something to oneself, eg, “sleep debt”.
Who cares if it conforms to the definition of the Bank of England?
My initial thought was that, as academics, they were going to be aloof and out of touch. However, as a boots on the ground dev who has worked from start up through multi-thousand employee public company, and who has held leadership roles, I agree with the article. Maybe if you read it, you would too.
I've given very compelling arguments to address tech debt that made a room of 20+ people in management smile and nod approvingly. Yet higher leadership still overruled me (and them).
If not, I am not convinced the article has anything to offer me in particular.
Software is very different to building the millennium tower - that's designed once, and when built won't be expected to substantially change or grow (at least from a structural perspective, you will change the interior). Building a skyscraper can take 5 years to complete, and when it's built it looks like the design from 5 years ago (with all the same load assumptions). With software the design is expected to change throughout the lifecycle, and should adapt to customer demand and scaling requirements, and you can't wait 5 years for perfect when 'good enough' will do and get you live.
Building complex solutions that you don't really need is also "bad engineering". Part of the complexity comes from the fact that the needs that software projects solve are not fixed, they change during the products life cycle.
Even architects use technical debt, it just comes in different ways. Instead of trying to plan what every room is for, architects leave empty sections, to be determined later. A lot of buildings have little spaces reconverted to closets just because there was leftover space there. That's technical debt, you know the "best approach" (according to waterfall engineering) would be to find out how things will work and then build it, but you'd rather write "IOU" on that section and move on to the important bits. You can then solve the kinks with plasterboard.
The best example of technical debt is code that doesn't scale. You know you should write your code to support 1M users, but you currently have ~100 users. Instead of spending a month serving 990k users you don't have, I would say it's good engineering to solve the problem for say 10k users if that means you'll spend a week, and have the rest of the time to solve other more pressing concerns.
I'm not sure I agree that an engineer experienced in building scalable systems will need 3 extra weeks to pick a schema and data access pattern that's scalable.
You don't have to actually _use_ spanner or cockroach or vitess in your initial prototype if you don't want to, but it seems entirely reasonable to have an _idea_ of what the story would look like to migrate to a horizontally-scalable approach if product gains traction so it doesn't end up being a multi-year project later.
For social media kind of traffic it became inadequate at some point and the debt had to be paid (and thanks to success was easy to pay).
To me a smart use of technical debt.
Certain qualities or features like horizontal scalability and security can be relatively inexpensive to design in at the beginning and immensely expensive to try to tack on later once a system is already in wide use.
I have worked in places as you describe, and tech debt is still a bad analogy. It isn't debt, but rather friction, which is a much better analogy.
SUMMARY: tech debt is just like financial debt, but with a clock whose speed varies with the rate of change of use cases and dependencies. Also, more subjective because instead of borrowing fungible money, you borrow extremely illiquid (and thus hard to objectively appraise) assets (like the notebook your great great great grandmother wrote her recipes in, or that Mario 64 cartridge we've been hearing about lately, or the mona lisa, or the last gas station in manhattan, or the vacant corner lot that's next to your house but can't be built so is of little use to anyone else, or the goodwill that the Disney name has -- all very real assets that could be conceivably borrowed through some kind of future contract, and many of them appear on real corporate ledgers and are reported on real SEC form 10Ks).
Financial debt doesn't "HAVE" to be paid off either, so that's no objection. A special purpose LLC subsidiary or something can go bankrupt. A security can drop to zero value and be written off. Just like a codebase can be abandoned, possibly in favor of a rewrite, possibly just entirely.
What's confusing you is that in finance, the clock marches on and interest accrues at a constant rate. But with tech debt, time only passes as fast as the software gets used, generating new use cases and usage experience that demand features and bugfixes. Software that doesn't get used doesn't pay interest because the clock is stopped. The more it gets used the faster the clock spins and the more interest collects. If it reaches an equilibrium where use cases and dependencies aren't really evolving anymore, like some sort of small core system library component like a heavily optimized sorting algorithm, time slows down again. Although the day a change comes -- eg a new weird CPU architecture that makes the old algorithm suboptimal -- time will pass again.
You do have to pay the tech debt, if the clock keeps moving. But dead men tell no tales and unused code does not "age"
The other confusing thing is financial debt is one dimensional. $1000 is $1000 is $1000. But there is no unit of exchange for tech debt. It's like bartering, like I have a monkey and a jar of pickles and who knows what each one is worth to anyone. A codebase well adapted to certain types of changes might be poorly adapted to a very different kind of change. Like maybe one codebase has hard to change business logic but easy to re-optimize the performance on a new architecture, while another one has mediocre performance on any but the chosen architecture but has very flexible business rules. So pricing is not well defined. My jar of pickles might be worth 2 cows to me, or 40 personyears of legal services to you. Who's to say? Without a way to trade, we cannot value tech debt except subjectively relative to our own use case.
Much like a joke, if you have to go to great lengths to explain an analogy, it is a bad analogy.
Tech debt is just a bad analogy, with a bit of use because friction is less understandable (but more accurate) to many financial and business people.
You should not build something you do not need if you know it is wrong and you will not need it.
That is just wrong.
Many engineering managers do this to “show progress” but this is because they are bad at their job.
On the other hand, if you need it so the UI team has a mock to work against (so that they can explore what interfaces they really need), it isn’t debt at all.
Futures contract, and then a loan taken out with the future in your possession as the collateral. Which is the thing any large low-margin operating company holding some futures would do by default (through the commercial paper market), because cashflow. (Or they’d just sell the future to the mutual fund they likely hold equity in. Same difference.)
> But I think you're reading too much into the terminology. The concepts being discussed here really don't have anything to do with finance.
The author of the article was also “reading too much into the terminology”, though. They were trying to pick apart the financial implications of technical debt as a type of loan — when it’s clearly not any type of loan. But the exercise they went through seemed fun, so I figured I’d try shoving the concept of tech-debt into some other pigeonholes, to see if it could actually fit one of them.
I encourage everyone with experience in both finance and tech to try this exercise; maybe there’s some exotic financial instrument that actually does exactly mirror the “mechanics” of keeping tech-debt around, such that examining the properties of that instrument could elucidate the previously-vague properties of technical debt (in the same way that the properties of mathematical objects can be elucidated by mapping them into different equivalent objects from other mathematical subfields.) Maybe we could even prove some things about technical debt. Wouldn’t that be neat?
Financial debt can be both good and bad. Good financial debt lets you take advantage of opportunities now that you wouldn't otherwise be able to afford, and you end up better off in the end. Bad financial debt leaves you worse off in the end. Taking on financial debt absolutely can be a poor choice.
I don't see why these same qualities can't show up in technical debt as well. So sometimes engineers might be making poor choices, the point is to evaluate whether the debt is taken on to exploit an opportunity, evaluate whether you'll be better off for taking on that debt, and to ensure you pay it back in a timely manner.
sometimes it is obvious to the engineer that this will cause problems in a few months, but the first deadline is in a few days, and that's all leadership cares about.
I dislike working with Bootstrap projects but I understand that "getting into debt" sometimes is unavoidable. And debt isn't necessarily a bad thing.
I've always assumed engineers (myself included) write code to the best of their abilities to solve a given problem at the time.
Another way of looking at technical debt, that I unfortunately must use regularly, is debt accrued by not updating systems. I work with hardware and software that is, in some cases, from the 80's. As these machines age, they become more and more likely to break. The longer we wait to replace them, the more value their initial investment is worth, but also the greater the chance that production will be halted for months while we try to design and implement a new system. So this is technical debt accrual as a function of time, not as a function of development speed.
Writing code that isn't great but works in order to ship a product in a timely manner incurs technical debt because you may have to do more work in the future to fix it once it is in use. But ideally your project is generating real income on that technical debt so it's probably better to have something to works right now vs something that's going to be perfect when it's done, eventually.
In your own personal project, I'd probably agree. But in my experience in a work project there can often be imposed deadlines or project changes that result in engineers writing code that they themselves know is suboptimal even at the time.
In other words, the sort of change that might include a comment like "I know this isn't the best way to do this but don't have time to do it the best way right now." It's not the best I can do overall, but it's the only thing I'm able to do for whatever reason. That's the kind of thing I'd call tech debt.
I never intended to put shame on tech debt. It's usually a now vs later (or never) tradeoff.
I initially agree strongly, but then I remember that labs maintain FORTRAN codebases for years. So I'm curious about their view of technical debt, but its definitely in a different context.
If you have a system that's been modified to handle every edge case encountered in the last decade, and it works, and other people rely on it to do their work, your perspective switches to maintaining the environment the system operates in.
You're looking at automation continuously built around processes that might predate electricity. The interfaces are ingrained into the organization. Other organizations rely on these interfaces and sometimes there are legal responsibilities.
It's a longer-term perspective. Tech debt isn't a crisis, it's an eventuality you try to encapsulate.
Plus, I'm not really sure good practices mattered to them even back then. Academics, whether they're programmers or not, chase different goals and have other incentives.
Like you, I'd love listening to one of them one day (if anyone speaks up). I'm sure there will be a few pearls of wisdom in such a shared experience.
That's not a technical choice since the person writing code wasn't aware of the options, so it's a business choice. How good of a business choice is it, though?
We all know at some level that a well-intentioned but inexperienced engineer can make a decision in an afternoon that will take weeks or years to clean up. I often see junior engineers spend several days over-engineering something that a senior engineer would have done simpler and better in a fraction of the time. I don't think it's a given that junior engineers provide better return on investment early on that a senior engineer.
Getting some worthwhile feedback and advice from a seasoned engineer doesn't need to cost a fortune in up-front cash, and personally I think it'll pay for itself. You either compensate them with stock or hire them as a temporary or part time consultant.
The problem is rather that the technical debt is often not repaid.
You can ship good code fast.
Shipping slow is not a guarantee of good code.
I think the pathology here is defending "sloppy" or "negligent" code by the need to ship fast, when in fact you probably could ship as fast with cleaner code, but the coder just could not care enough.
The problem with brittle code is it makes any change more difficult. The most obvious example is variable names without semantic meaning. The compiler doesn’t care if you’re using “asdf” but such choices collects its pound of flesh every time anyone messes with that section.
Another example of technical debt is when you write code without tests. You’ll probably finish much faster in the first week or two and then the manual testing will take more and more time. But how often did we see code bases without tests? All the time! And the managers don’t realize how much money they could save by focusing more on quality instead of features.
If you think you‘ll just implement this one thing and then never touch the code again, chances are high you‘ll just hack up the thing and copy paste some code everywhere until it kind of works. This is increasing technical debt, you take out a loan and someone in the future has to pay it back IF they want to make a change. But this is almost always the case, even if you don’t know it now.
My definition of technical debt I wrote down before reading the article: technical debt is the inability to make changes due to not making changes (on a conceptual layer) before.