Argh, we're drowning in technical debt!!!

Thanks AI! Now can you help us fix it?

Earlier this year, I wrote about vibe coding. This is the fun term for using code generated by AI models without doing any quality checks. Vibe coding remains a thing, but I get the impression that sensible developers, at least, have realised the virtue of doing sanity checking. You know — does that code look like it does something along the right lines, or is it going to send all my data to <insert enemy of the people here>?

But it turns out that a lot of the issues around AI-generated code are more subtle than this. It’s not so much that code does the wrong thing (though of course this can happen) but that it does the right thing in the wrong way. This is often not apparent from a brief code review. But over time, these niggles build up and contribute to technical debt — a general term for the effort and cost involved in fixing problems in code bases. Technical debt has always existed, and it’s the reason why good development practices involve regularly looking at code and fixing problems before they get out of hand. But it’s become much more of a problem with AI-generated code, because a lot of people don’t understand the code they’re working with, and it’s hard to fix things if you don’t realise they’re wrong.

It’s well known that AI-generated code is far from perfect, so it’s no surprise that there are issues. After all, it’s generated by sampling a model of code that was itself generated from code samples written by squillions of people. One that’s inevitably managed to capture all the historical mistakes regularly made by people. And the historical part is quite important here: human developers learn from past mistakes, and mostly avoid doing things which are known to cause issues. AI, on the other hand, is trained on lots of old code, and doesn’t know it’s wrong.

I touched on this a couple of months back in my post on security risks in AI-generated ML code, but the basic issue is that coding practice moves faster than AI training data. Generative AI models need a lot of code to train them, so it’s unavoidable that they’ll be learning from code written a long time ago. And consequently they’ll also generate code in styles that were used long ago. Whilst this is a particular problem for ML, where coding infrastructure moves really rapidly, it’s also a problem more generally.

And by styles, I include things like the use of particular libraries and function calls, some of which may be deprecated or considered risky due to known issues. A huge amount is known about this, thanks to initiatives like Common Vulnerabilities and Exposures, which map out all the things that regularly go wrong. Generative AI models also know about such things if you ask them, but this doesn’t mean they apply this knowledge to the code they generate. Which tells you a lot about the lack of joined-up thinking in current generative AI models.

The consequence of all this is that active code bases are now stocked with oodles of vulnerability-ridden code¹ which developers hardly understand and often feel little ownership of. Fixing all of this, assuming anybody cares to, is a substantial and potentially expensive challenge that could push maintenance budgets into the stratosphere. So, to answer my own question; yes, we are drowning in technical debt. And, as usual when AI gets us into a “situation”, the natural response of the tech community is to ask: can AI get us out of the hole it made?

The answer seems to be: potentially. There have at least been some productive steps in this direction. DeepMind’s new CodeMender tool, notably, has been quietly fixing security issues in open source projects; but so far we only have DeepMind’s word on how well it works². My own experience, vicariously through a PhD student who’s been doing experiments on this kind of thing, is that the ability of LLMs to identify vulnerabilities in code is still quite wanting. That is, they catch some, but they miss a lot. But maybe DeepMind has some secret spice that they’ve not told us about yet?

1. An interesting read here mentions how widespread these vulnerabilities are.

2. They have said they will publish technical papers at some point.