Will everything become a neural network?
Attitudes towards neural networks have changed a lot during my career, especially in the last few years. Twenty years ago, the thought of integrating one into a computer system was met with horror. People would say things like “How can we trust something that wasn’t designed by the mind of a human? Surely it will do things we won’t expect? Argh, run for the hills, the terminators are coming!!!!”
Fast forward to today, and the mindset has flipped. Now, you’re more likely to hear something like “Why not replace that existing piece of software, carefully written by a team of trained software developers, with a neural network?”
This has been visible for some time in efforts to make LLMs do maths: the kind of thing we can do very reliably using a calculator, or more advanced software like Mathematica. And to be fair, this made some sense before agentic AI came along, since people wanted LLMs to integrate mathematical reasoning with other tasks. But now you can (see Plumbing LLMs into the world) give an LLM access to a calculator, or software like Mathematica, and it can use it to do sums, solve equations, and myriad other things that make right-minded people scream in agony.
A more recent example of neural networks replacing mature software is DeepMind’s Genie 3. Check out Hannah Fry’s interview with the developers for a nice overview, but basically this is an LLM that can internally create and simulate world models. Up until now, world models have been the domain of dedicated simulation software, most visibly in the form of game engines such as Unity, where you configure a world model and the simulator then applies rules (typically based on conventional physics) to make it evolve over time in a meaningful and consistent way.
The difference with Genie 3 is that there’s no dedicated simulator that is explicitly programmed to follow physical rules. Rather, simulation is something that’s been implicitly learnt by the LLM. The benefits of this include a natural language interface (rather than code or configuration files), the simulator being fully tied into whatever else the LLM is doing, and — following on from this — the flexibility to change the simulation on the fly, according to the whims of the LLM.
The downside is that it only maintains consistency for several minutes at a time, and objects in the simulation might start to move unexpectedly or appear or disappear after that. Or physics as we know it might break down. So, a pretty major issue. Which begs the question: is it a sensible or necessary goal for the LLM to do everything? That is, we already have mature and reliable world simulators that don’t require world-draining quantities of computation to run, so can’t there be a half-way house where the LLM extends the abilities of existing simulators in some way whilst preserving their dependability?
I don’t really know the answer to that question. Maybe LLMs can competently replace human-developed software entirely. Maybe they can’t. Either way, it’s interesting how much the view has changed from let’s use human-written software as much as possible to let’s get the LLM to do everything. Maybe this just reflects current fads, AI obsession, or the increasing laziness of humanity. Or maybe it’s a sign of things to come.
Another perspective on this is whether we need everything to be 100% guaranteed reliable. It’s typically the view of traditional computer science that we do, but actually many things can be done quite adequately with lower levels of reliability. This view has been championed for quite some time in the field of approximate computing (see Wikipedia), which is all about trading off reliability against things like computational cost. We also see it in traditional machine learning, where it is normal for classifiers to get less than 100% accuracy, yet still be useful within many domains.
And largely this does come down to domain-specific concerns. For example, you wouldn’t want a nuclear control system to be trained using a less-than-perfect simulation. But actually you might be happy to train a general-purpose robot on approximate simulations. And this is a driving argument behind approaches like Genie 3: that we could train a robot within a large, and potentially infinite, number of simulations without someone having to design them all by hand. Yes, it may learn the occasional weird or suboptimal behaviour, but if the robot’s not going to tote weapons or hang around in a safety-critical context, that might not be a problem. After all, humans also display their fair share of weird and suboptimal behaviours.
Anyway, we’ve reached a point where it’s not hard to imagine neural networks entirely replacing conventional software. To be fair, I never found it that hard to imagine this, since my background is in bio-inspired computing, and I was playing around with neural networks and their ilk way before they became sexy. What has changed is that other computer scientists no longer look at me askance when I suggest such things. Though they might still shout “Argh, run for the hills, the terminators are coming!!!!” And maybe that’s the right response. Or maybe it’s just the start of something new?