What is being advocated is that businesses should only try and acquire good debt. When a better understanding of the problem is apparent after the development work has been done, that good debt may be paid off. The bad debt should never be acquired in the first place.

Thanks also, Justin, for pointing us (offline) to this explanation of why “A Mess is not a Technical Debt”.

I note that the comments in this article reference this YouTube clip in which Ward Cunningham discusses why he came up with the metaphor of “Technical Debt” in the first place

Makes it easier to understand than before.

One more example of how we all have much still to learn…

As we have been working through a business planning process for a spin-off business recently, something I said with reference to that debate is relevant here:

(in my view) a short, sharp process of business planning is invaluable - however a business planning document (on its own) has very low value

For more of my view, see my post “Why do we document”.

The conventional view of a spec as a bible, or as a record, does not figure for me. A stapled group of User Stories, and properly commented code, would be much more effective.

To avoid future confusion, I would much prefer we did not even use the term “spec”, because of the conventional view it conveys.

A design spec is an aid, not a holy book.

Also while I think it’s very important to take pointers from other disciplines like you’re saying, it’s also important to realise that software has it’s own unique qualities as well. I think the big difference for me comes because very often software can be changed without the cost of it being prohibitive. Or at least that’s the perception everyone has.

For example, you can’t go changing the way the foundation’s of the Empire State Building are designed when the top floor is in place (unless you’re spending a whole lot of money). Software tends to not be like that. The fact is, because it’s (perceived as) able to change, techniques that enable it to change more rapidly that minimise the cost make a lot more sense. While change still does cost (and too often people think it’s free) it is no where near as much if it’s done right.

On a related point, i.e. “design is not finished when a spec is written”. I’d be thinking about why you would even need a spec in the first place. If the design is never complete until the completion of the release then that spec and the time spent on it become a big waste of money. A big formal spec document will tend to get ignored when the project is in development anyway.

You make some good points:

1) re Technical Debt

With respect to Technical Debt, this is another case where we need to consciously avoid the thinking that is best summed up as “Tyranny of the OR” - from “Built to Last”.

I have posted previously that the ability to reject this will be one of the important soft-skills that our GM DDD WCW will bring to the company, and which we will then have to continue embedding as part of our culture.

Investing this time effectively will give us the platform for the next stages of growth.

This is along the lines of what you have noted - i.e. Technical Debt is neither good nor bad - it’s what we do with it that counts.

In my view, it’s just another investment in the ongoing growth of how we delight customers, and hence grow the company.
(a) If we invest well (as we have with NEM-Watch and NEM-Review, for instance) then our incurring of Technical Debt was massively worthwhile.
(b) In other products that have not performed so well, our investment (in all its forms) has not been as favourable. However, even this investment is not completely wasted, so long as we learn from it.

As you know, we have gained major benefit from the recurrent revenues generated by products developed in the first 4-5 years (thanks for that!), even though we incurred some Technical Debt to get to where we are now. Because we can afford to now, we have begun to pay off this technical debt, starting with a re-write of NEM-Review in the past year.

Hence, I agree with you that we should use caution in blanket statements like “technical debt is something that we wish to reduce as much as possible”, less they be misinterpreted as implying we need to avoid Technical Debt.

For us (as with everyone else) it comes down to a cost-benefit trade-off. On that note, our ability to - as a team - understand NPV is something we need to improve, moving forward (another thing our GM DDD WCW will help us with).

Similarly, new products we develop to serve an expanding range of clients won’t be perfect, either (internally, in terms of the way the code is put together - and externally, in terms of how many client needs we nail). Our challenge is to continue to iteratively improve - bootstrapping the company forward.

2) re Crossed Wires

You are correct in that Stephen and I had wires crossed. We clarified this offline.

1) Stephen was just making the point that the design is not finished when a spec is written (I agree - and see this as one of the fallacies of a Waterfall paradigm)

2) I was just making the point that this is not dissimilar to other disciplines, in many cases - and we need to learn from them (as they have been running for many years longer than the software engineering one, so have made plenty of mistakes we can learn from).

Note it was more of a discussion than an argument - as you will recall, I certainly welcome different ideas (so long as they are positively promoted, and so long as we collectively continue to learn something new every week, and not get stuck in inappropriate paradigms)!

Hope the snows are enjoyable!

Cheers

Paul

I’d be careful with how you think about technical debt. Maybe I’ve taken what you are saying the wrong way but I don’t think you should consider there being 2 different cases here (speed to market and long lived code) that need to be treated differently. It is not something that you either ignore or don’t ignore. You will always have some tech debt. The comparison with financial debt is based on the assumption that you would take on debt for short term gain with the intention of paying it off at some later point. If you never pay it off then things get worse and worse (much like financial debt).

The important thing is that you’re not trying and keep your debt at zero at anytime. There are always going to be things in your code base that you don’t like but it doesn’t mean that you should change them immediately when you notice them to keep your tech debt down. Keep track of them and pay them off by treating them like any other chunk of work (prioritise it relative to other upcoming things in terms of importance etc.).

Regarding your discussion about the document being a deliverable and what that means. It sounds a bit like you guys are discussing different kind of things so don’t get your wires crossed. I think that Stephen is talking about the act of actually writing the software (a developer concern) and Paul more about the entire process of delivering it (the concern of the whole team). The point is I think you might be arguing about slightly different things.

I am unsure of what you mean by this statement. I was trying to state that in software, the final design document is the source code itself. It is the document which describes to the “builders” (compiler) where to put “all their nuts and bolts” (machine or interpreter level instructions).

In the empire state building example, the final deliverable for the engineers was not the building itself, but the plans for the completed building. Just because the plans were delivered in parallel with construction doesn’t change the fact that the plans were the engineers’ deliverable.

Perhaps I am misunderstanding your point.

Sounds like an interesting presentation.

When I get some time, I would like to check out the materials referenced on the IBM site - though it is more important that those in our Division for DDD WCW (a.k.a. Products Division) do this. Certainly when our General Manager for DDD WCW is onboard, I would expect that they would be leading this kind of ongoing learning for our growing team.

I will contact Neal Ford directly, as I am unclear on whether something was the intent of his message, or whether it was just lost in translation. This has to do with the following:

1) With reference to Software Engineering:

You reference the comment “The design is not final until the last line of source code has been written”.

This is one of the points I was trying to make back in September 2009 as one of the key reasons we needed to be more agile (by adopting Agile) with our development.

At the time, I heard internally that “I still don’t see why we need to go Agile” which illustrates that I was not as clear as I needed to be - which is why I was more direct (in Nov 2009) ,
despite the fact that this confused a few people externally and needed clarification (Feb 2010).

Thankfully, we have come a long way since then, with everyone now “on the bus” in terms of accepting Agile as the way forward - though there is still much that we actually need to implement (and our GM will help us with this).

The context of your quote is the implicit assumption that:

(a) In “design”, there are still uncertainties (which might be in terms of technical uncertainties, or budget uncertainties, or in terms of what the customer wants - or, in our case, all three).

(b) After “design”, the implicit assumption is that these have been alleviated and the solution is known - how to do it, how much it will cost, and what the customer response will be. Hence all that is left to do is “crank the handle”.

From what I have seen (internally and externally), most software projects never get to the “after design” phase. However some do, in which case Waterfall processes might suit just fine for them.

2) With reference to other branches of Engineering:

You quote Neal Ford quoting Jack Reeves as stating that “every engineering discipline produced a document as a deliverable …” - with the implication being that the design (in other disciplines) as “done” (and all uncertainties addressed) when the document is drawn up, whereas it is not in software.

I am unsure of where in the chain of chinese whispers the communication became crossed, but in my mind this is misleading, and myopic.

In my mind, this line of argument is much too similar to the line I have heard in many different industries, companies and countries whereby “that’s fine for them, but we’re different, and [INSERT METHOD] won’t work here.

To be direct, there are more similarities between software development and other streams of engineering than there are differences - and, for us to achieve our vision in terms of 20x number of clients with only 4x number of staff, we will have to be very smart about “stealing ideas shamelessly” from whatever field we can find them.

(a) That’s the point I was trying to make back in September 2009, though unsuccessfully by comparing and contrasting the “design” stage from the “manufacture” stage.

(b) That’s also the point that Scott Berkun made in chapter 1 of “Making things Happen” - drawing on examples as diverse as restaurant kitchens and hospital emergency rooms. Incidentally, being able to drag in ideas from diverse fields is requires a combination of the five core skills facilitating disruptive innovation.

(c) Finally, this was one of the main messages in a Webinar presented by Mary Poppendick a on February 3rd through the IEEE on “Advanced Agile - Below the Low-Hanging Fruit”, which Adam sat in on and will post about later. In the presentation, Mary drew on the experience of the construction of the Empire State Building to highlight how:

i. In the same paradigm, the “design” of the building was not done until the building was completed; and
ii. How some engineering practice (used almost 100 years ago in that example, and now labelled “Lean”) are just as relevant to some software practices today.

Hope this makes sense.

We have come a long way in 8 months, but have a long way still to go…

Paul