Introduction

Here is a story which is very common in finance and most likely in other industries too. It goes like this:

1. A business need is identified.
2. New software is written to address the need.
3. The software is actively used and people are happy.
4. Suddenly, it is realised that the software can no longer be maintained.
5. Frustration builds up.
6. A decision is made to write a new software, surely a better one.
7. The process repeats, often resulting in both software systems running in parallel.

Does it sound familiar?

The value of re-writing

Believe it or not, plenty of people find this situation to be desirable. Some like to fiddle with code, while others like having large teams under them. The endless churn creates an impression that value is being created and adds more importance to the people doing this work.

The “added value” of re-writing can be “proved” in a formal way. Let’s suppose that the original software has provided benefits A, B, and C. The frustration experienced by its users is an additional concern D. Therefore, the result of a re-write would be software that addresses A, B, C and D. That’s plenty of boxes ticked, and it does look like a valuable result to a naive outside observer.

But surely there are some real benefits. After all, it is promised that the alternative would be faster, easier to use, and less expensive to run. Well… I would be delighted to see those but in 20 years or so of my practice I have seen none. Have you?

The cost of re-writing

Overall, this endless process of re-writing essentially the same thing — again and again — is costly. Not only time and money are spent on the re-writing, testing, and re-integration of software, but also the end users and support need to get themselves familiar with the new generation of the system.

Furthermore, the process of onboarding the new software is often accompanied by discovering various bugs and imperfections, which results in reduced productivity for months.

But there is more: the software engineers are spending their time re-writing the old code instead of working on matters which can be actually beneficial to the business. The real efficacy of engineers drops, although you won’t necessarily see it through metrics.

And let’s not forget that a re-write of a single software system often triggers a cascade of re-writes or at least changes for the related software systems. Of course, theoretically, the interfaces should stay intact. But in practice, they tend to change “for the better good”.

The causes of the cycle of wastefulness

Why do organisations end up in this situation?

There are many reasons for it, and I cannot possibly cover all of them, but here are a few:

1. The accumulation of technical debt results in the software being increasingly hard to maintain, up to a point where nobody would dare to try mending it;
2. Adding toxic changes which result in overcomplicated software;
3. A belief that new technology would suddenly improve everything so the existing software needs to be re-written to reap its benefits;
4. And last but not least, the loss of the engineering team, resulting in software nobody can hope to understand and maintain.

So let us consider these causes one by one.

Technical debt

Technical debt is a name for a situation where software — which had been tidy and maintainable initially — has lots of these negative qualities due to various alterations made without enough care, resulting in the code being an unsightly chimaera. From a practical point of view, this means that it has become hard to reason about the code, which means it is harder to make any more changes (including the ones which may tidy it up).

The problem of technical debt can be a result of the lack of experience of the engineers, management demanding new features (often ASAP) which go against the grain of what the software was meant to do and how it operates at the moment (see below), lack of discipline in the engineering team so that not enough time and effort is allocated for the work of maintaining the code in a healthy state, and counterproductive performance metrics which reward for the delivery of features while sacrificing long-term viability of the software.

Every single cause of uncontrolled accumulation of technical debt can be addressed by upskilling the workforce and/or by setting the priorities right.

Toxic changes

Almost every software system in the world is changing over time: deficiencies are found and mended, integrations with other systems are added, removed, or modified, the behaviour is tuned, and new features are added while old ones are sometimes removed.

However, not all changes are equally benign.

Let’s forget about software for a moment and consider an example featuring a car.

First, a car is made and serves its purpose. The owner notices that the steering is a bit stiff and that is mended. That is called “fixing a bug”. Then the owner decides that the car needs to be able to tow a caravan. No problem. A towbar is added to the car and is used to attach the caravan. This is called an “integration”. The owner had decided to replace the carpets in the car. That is called “cosmetic changes”. And then the owner decided that the car should have a VTOL capacity. And that, ladies and gentlemen, is called “insanity”.

Now let’s go back to software.

Just like in the example above, not every change is benign or even sane. However, where there is a [political] will, there is a way. And so, where the engineering team does not have a strong enough say to push back on the demands, embarks on a tough project of adding a VTOL capability to a car — I am sorry — building a state-of-the-art database as an addition to a market connector.

The solution here is better communication between software engineers and their customers, educating the customers on what is easy to implement and what is not, and considering the requirements in their entirety rather than individually.

The latter is important because when an individual requirement is considered, it may look as if the natural response is making or not making a single small toxic change, while when all the requirements are considered together, it may become obvious that they call for a separate system or some kind of re-arrangement, after which the requirements can be implemented while avoiding any toxic changes altogether.

Belief in a “silver bullet”

A belief in a silver bullet is sadly a common thing in software engineering teams, especially among less experienced individuals.

It appears that the majority of people believe that switching to a fresher version of a library or a completely new framework, adopting a new and popular approach in the way the software is written or the engineering process is organised is going to somehow improve everything, as if by magic. Suddenly, there would be so much fewer bugs, the code would look beautiful, stakeholders would be satisfied, and there would be no need for that boring job of code maintenance so engineers would move on to greenfield projects to write more wonderful code.

Well… no. It does not work this way.

Such changes might improve some aspects, but the improvements are likely going to be very small indeed. At the same time, much time and effort is going to be spent on figuring out less than obvious matters resulting from those changes. Oh, that new framework is nice for sure but you did not realise that it had plenty of quirks so several weeks were lost just to make sense of what’s going on. That new approach to software engineering is fantastic, but it actually does not fit your specific team and projects well. So by adopting it, instead of improving the efficiency of the team, you have sabotaged it. And so on.

A prudent approach is to recognise that there is no silver bullet and to resist unnecessary changes. It is not that new technologies cannot be useful, but the added value is often overestimated due to inevitable hype, while problematic points are swept under the rug and would remain invisible to you until you have made a switch, paid the cost, and have committed yourself to the new shiny thing.

Loss of engineering team

This is perhaps the biggest problem of all, and at the same time, it seems to be the least recognised one.

Here is how it works. Once the software is written and onboarded, suddenly the situation changes in many ways.

1. Software engineers are keen to move on to the next big thing, and they are not particularly interested in day-to-day maintenance work.
2. The stakeholders believe that the job is done as the software already exists and can be used. So they do not see much value in keeping the engineers around.
3. The work culture is such that creating new things is seen as more valuable than supporting the existing ones.
4. Furthermore, the support is seen as a cost, especially when it is support for software that has existed for a while and is perceived to be “old” and “obsolete”.

The result of that is that software engineers move on to other projects and sometimes to other organisations. That tends to be unnoticed at first. Some stay, but often they are less ambitious and less competent in the team.

Eventually, a realisation dawns upon everyone: not a single person from the team of the original engineers is left in the firm and so nobody really understands how that system works. There is no practical way to save the day and the system is declared to be “obsolete”. A full re-write is inevitable.

Some believe this kind of issue can be mitigated by writing plenty of documentation. Good luck with that. Documentation — no matter how well written — is dead without its authors. When you end up with a code — even a well-documented one — but its original author(s) are gone, you may be better off re-writing everything from scratch, possibly never looking at the original attempt.

As a side note, documentation can be useful but few people understand what needs to be put into documentation, so much of it turns into an irrelevant abandonware much earlier than the software it covers. That is a matter for another post, however.

Another way in which organisations are trying to deal with the problem is by ensuring that there is always more than one person responsible for — and familiar with — any given software system. This idea has some merit, of course. When the primary engineer(s) leaves or becomes unwell, you would be in much better shape if there is a living soul who has at least cursory familiarity with that software. However, cursory familiarity won’t cut it when ensuring the longevity of the software is needed — one needs a deep understanding of the software and its problem domain in order to identify which strategic (rather than tactical) changes are required.

It is a mistake to believe that software is an asset (more on that in a different post). Software is a liability. It is the useful features offered by your software and the understanding of the problem domain in the heads of your engineers that are assets. Realising this simple truth can save you from ending up in an ugly situation.

So do not lose your engineers if you want your software to survive and serve you well.

Conclusion

The cycle of wastefulness can be caused by many factors but not a single one is inevitable.

It is commercially valuable to fight against the trend of endless re-writing of the same software because keeping the old software healthy and well-maintained reduces the actual costs overall, keeps the efficacy and efficiency of the teams high, and as a side effect helps you to retain valuable talent.

Love “legacy” and those who understand it. It is a result of people thinking about the problem domain and the specific challenges of your business, and figuring out the best possible solutions to those challenges. As long as you care for your legacy, it may continue faithfully serving your needs.