One of the most impactful articles on software engineering that I’ve ever read was The Law of Leaky Abstractions at Joel on Software. Go read it if you haven’t. Joel points out how abstractions, such as TCP/IP, allow us to compartmentalize and ignore large parts of the software stacks we are building, but also that all abstractions leak.
I have assigned this article to students in my robotics class for years because in robotics, we have lots and lots (and lots) of abstractions, and also those abstractions leak like a sieve. The networking examples in Joel’s article are especially salient, because of how many networking problems we’ve had with how many robots at how many sites over years and years. Today in lab, we were unable to connect to our wifi network because there were too many personal hotspots crowding up the wifi spectrum, and we fixed it by moving closer to the lab’s access point. The abstraction is that wireless networking Just Works everywhere inside the lab; the reality is that when the spectrum is congested, the abstraction leaks, and you have to move yourself and your robot closer to the access point to make it work.
I sometimes joke that my lab sprouts home wifi routers, almost like mushrooms. Not because my students are bad or breaking the rules, but because the network abstraction of our enterprise wifi leaks, and they need to fix it, and they need total control over the network to do it. I like to say that robotics is the decathlon of computer science: to build a robot you need systems and theory and networking and algorithms and computer vision and then you have to do robotics on top of it all. And every one of those events is an abstraction that leaks.
AI coding (as distinguished from robotics foundation models) does not make these problems go away. It helps and it hurts. Finding and tracking down weird bugs when the abstraction leaks is one of the strengths of coding agents. I have been accelerated many times because I can use Claude to quickly find a problem, and it’s better than me at holding all the moving pieces in its head. But also, even more than before, it’s easy to go down the rabbit hole of the wrong answer: there are more possible rabbit holes as we stand up larger and larger systems. And we understand each one less.
Here’s an example. Once I was using Claude to write a Scratch program. (Yes, that Scratch.) I asked it to program a neural network with back propagation, and it happily did so. But we hit a roadblock. I asked it to add a way to verify the network worked by adding the ability to classify two-dimensional points trained on an artificial dataset created by two Gaussians. It said it did it, but I couldn’t see it. I reported the bug and it fixed it. No dice. It tried again. It said it had reproduced the bug and fixed it. We went through like five iterations and after each one it said it fixed the bug, but it still didn’t work. Finally I noticed that somewhere in there, Claude had renamed the file containing the text of the program, and I was uploading an old version from before I’d asked for the feature. The whole time I was confidently reporting the bug, and Claude believed me and confidently fixed it, but it wasn’t real — we were running open loop because I was looking at the old version of the file.
The leak wasn’t in the Scratch code at all. It was in the loop between me and Claude. This example itself is much simpler than a real robotic system, but it has the same flavor of many disparate systems working together, where it is easy to suspect the bug in the wrong part of the code.
With AI coding agents, we need to pay attention to specification, bug reports, and verification. Doing those well requires a deep understanding of the bits and bytes out of which we build our systems. But the agents also take away the day-to-day work that used to build that understanding.
Joel saw this coming in 2002. He said: “Code generation tools which pretend to abstract out something, like all abstractions, leak, and the only way to deal with the leaks competently is to learn about how the abstractions work and what they are abstracting. So the abstractions save us time working, but they don’t save us time learning.” Coding agents are the wizziest code-generation tool yet. They save us enormous amounts of time working, but they don’t save us the time it takes to learn.
Joal again: “The Law of Leaky Abstractions is dragging us down.”