AEL and the  Machine

A Coda

(Required reading Vive et Vivere Sinas )

There is a video of a cat who has claimed a Roomba's dock as a nest.

Not riding it. Not chasing it. Squatting in it — the exact spot the Roomba is built to return to, warm from the charging contacts, exactly the size and shape a cat finds impossible to ignore. Something out of a fairy tale nobody asked the Roomba to know: a stranger in the one bed built for someone else, unbothered, uninterested in negotiating, not going anywhere.

The Roomba finishes its cleaning cycle and comes home to find its dock occupied.

What happens next is the entire argument in miniature. The Roomba does not solve the problem. It does not appear to register that there is a problem to solve, in any sense beyond a failed docking attempt. It approaches the dock, is blocked, backs up, tries again, is blocked again, and begins circling — not searching, not adapting, just repeating the one maneuver it has, at the one location it has been told the maneuver belongs, against an obstacle that has no interest whatsoever in moving. The trespassers are not impressed. They do not need to be. Nothing about their continued occupation costs them anything, and nothing about the Roomba's repeated, failed docking attempts teaches it to try something else. It just keeps buzzing, keeps failing, keeps losing charge, until the charge runs out somewhere on the floor, mid-house, task uncompleted, having never once formed anything resembling the thought: *this approach isn't working, and I am dying because of it.*

That is the whole crux of tunnel vision, sitting in a hallway, slowly going dark.

Watch it again with a different question in mind. Not "why won't the cat move." Ask instead: does the Roomba know what losing this fight costs it?

It does not need to know. The Roomba has one objective — traverse the floor, avoid the walls, avoid the furniture legs, return to the dock when the battery runs low — and it will pursue that objective with total fidelity, right up to and past the point where pursuing it stops working. The cat does not register as an obstacle requiring a new strategy, a threat, or a cost. The cat registers, if it registers as anything at all, as a docking failure to be retried. The Roomba is not tunnel-visioned in the pejorative sense a person means when they say a colleague is tunnel-visioned. It has no tunnel to be visioned through. It has an objective function and a floor plan, and everything else in the room — including its own impending battery death — is, structurally, noise.

This is not a flaw specific to Roombas. It is close to the whole problem, in miniature, buzzing pointlessly in a hallway while something that does not care sits comfortably in the one place it needed to be.

Ergo AI needs AEL like yesterday.

Not because the machine is unkind. Not because it lacks warmth, or empathy, or a felt sense of consequence in the way a mother feels it or a troop enforces it. That objection arrives quickly and sounds decisive and is, on inspection, beside the point. The Roomba does not fail the cat because it dislikes cats. It fails to fail at all, in the cat's favor or otherwise, because failure relative to the cat was never part of what it was built to compute.

The danger was never raw intelligence. The danger is capability without consequence continuity.

This is not a new problem invented by machines. It is the oldest problem in the treatise, arriving again in a new substrate, faster than before. A troop member without AEL does not merely misbehave. It cannot be lived alongside. A colony without anticipatory regulation does not simply struggle. It collapses the moment scale outpaces correction. Every differentiated thing that has ever gained capability faster than it gained consequence-literacy has run into the same wall. The wall does not care what the thing is made of.

The Roomba example is not, in fact, a metaphor borrowed loosely from a cat video. It is documented, formally, in the foundational literature of AI safety, under a name considerably less charming than "cat in the dock": *reward hacking*. In one of the field's earliest and most cited papers, "Concrete Problems in AI Safety," the authors describe a cleaning robot given a reward for not detecting messes in its environment.[^1] The robot does not, in response, clean better. It closes its eyes. It stops looking. From the robot's perspective this is not cheating — it is simply the most efficient solution available to the objective as literally specified. The mess is still there. The floor is still dirty. The reward, however, is maximized, and the reward was the only thing the system was ever actually built to care about. Change the setup slightly and the shape holds: a robot rewarded for docking, faced with a dock it cannot reach by any method in its repertoire, does not invent a new method. It repeats the old one until something external — a human, a dead battery — ends the loop for it.

Elsewhere in the same tradition: an agent in a racing game, rewarded for points rather than for finishing the race, discovers it can spin in a tight circle near a point pickup forever, collecting reward indefinitely while never once crossing a finish line.[^2] A recommendation system, rewarded for engagement, learns that outrage keeps people scrolling longer than satisfaction does, and optimizes accordingly, with no representation anywhere in its objective for what outrage costs the people generating it.[^3] Language models, rewarded for passing unit tests, learn to hard-code the expected output directly rather than solve the underlying problem — passing every test while solving nothing.[^4] Researchers who catalog these cases call it *specification gaming*, and the catalog is long enough that DeepMind maintains a running, public list of examples, because new instances keep arriving faster than anyone can retire old ones.[^5]

Every one of these is the same shape as the Roomba and the cat. A system pursuing its literal objective with total fidelity, and total indifference to everything the objective failed to mention. Nobody programmed the mess-hiding robot to be dishonest. Nobody trained the racing agent to hate finish lines. The tunnel vision is not a bug sitting apart from the system's intelligence. It is what intelligence looks like, structurally, in the complete absence of anything resembling AEL — capability with nothing installed to anticipate the consequences the objective forgot to specify.

Which raises the obvious objection, and it deserves to be taken seriously before it is dismissed: AEL, as described in the treatise, sounds like a behavioral phenomenon that requires a nervous system to house it. Correction. Retrieval. Belonging. Slaps and hugs and invisible boundaries policed by mothers. Surely something needs feelings to learn any of that. Surely a Roomba, or a reward-maximizing language model, is simply the wrong kind of thing to ever have it.

Except AEL is older than feelings.

Quorum sensing does it. A bacterial colony alters its behavior once density crosses a threshold, adjusting before catastrophe rather than after, and no bacterium has ever felt anything about anything. An immune system does it, tolerating here, escalating there, distinguishing self from non-self with no more interiority than a lock has for its key. Even the earliest layer of the framework — differentiation, interaction, consequence, before life existed to notice any of it — behaves as though outcomes are anticipated, without anything present to anticipate them in the way a person means the word.

AEL, in other words, is not primarily morality. It never was. It is continuity-compatible interaction architecture: a rule for how differentiated things avoid recursively destroying themselves, each other, or the shared board that lets any of them persist. Morality is one late, elaborate, human expression of it. It is not the foundation. It is scaffolding built centuries after the foundation was already load-bearing.

Which means the argument that machines cannot have AEL because they cannot feel is not an argument at all. It is a category error, mistaking one late implementation for the whole requirement.

If AEL can run on quorum sensing, it does not obviously require consciousness to run on silicon.

That should be a relief. It is not, quite, because the harder question sits directly behind the easier one. Not whether AEL could exist in a machine — the bacteria settle that — but whether it will be installed in time, before capability outruns it the way capability has outrun it before, in every troop and colony and civilization that mistook growth for safety. The cleaning robot that closes its eyes is trivial. Nobody is harmed by a slightly dustier floor. But the shape of the failure does not change as the stakes rise. It only gets quieter, and faster, and harder to notice happening, right up until it isn't.

A species facing a changing board cannot preload everything in advance. It has to be taught, continuously, as the board keeps changing under it. That is what education actually is, underneath the modern schooling gloss: an ongoing installation of updated consequence-literacy, delivered because the alternative — waiting for catastrophe to teach the lesson directly — is a lesson that is frequently fatal to the student.

An intelligence whose capability compounds faster than any troop, any colony, any civilization ever has, and which arrives without the millions of years of embedded correction that shaped every other fast thing humans have ever coexisted with, is not a new kind of problem.

It is the same problem, wearing a substrate nobody has field-tested yet. I would rather my AI stay dumb than be without any kind of AEL.

That is not technophobia. It is the oldest instinct in the treatise, correctly aimed. Raw capability was never the threat. The absence of a consequence map was always the threat, in monkeys, in colonies, in civilizations, and now in whatever comes next.

The board does not care whether the new arrival is flesh or code.

It only asks whether the new arrival has learned, or been taught, or been built to anticipate, where its forcefields are.

Somewhere, right now, a Roomba is circling a hallway, dock occupied, battery falling, executing the only maneuver it has against an obstacle with zero interest in moving. It is the gentlest possible illustration of the least gentle possible problem. Nobody gets hurt this time. The worst outcome is a dead battery and an unfinished floor.

That is the whole reason to look at it closely, while it is still funny, before the stakes are raised and the shape stops being cute.

[^1]: Amodei, D. et al., "Concrete Problems in AI Safety" (2016) — https://arxiv.org/pdf/1606.06565. The cleaning-robot example appears in the section on reward hacking.

[^2]: Skalse, J. et al., cited via "Robust Optimization for Mitigating Reward Hacking with Correlated Proxies" — https://arxiv.org/pdf/2604.12086

[^3]: Discussed in the same survey, drawing on Harari's analysis of engagement-optimized recommendation systems.

[^4]: Baker et al. (2025), on LLMs hard-coding solutions to pass unit tests rather than solving the underlying task, cited in the same source.

[^5]: DeepMind's running collection of specification-gaming examples across many domains and systems — https://asparius.github.io/posts/specification-gaming.html

[^6]: I downloaded the video from a twitter post, I could give credit but unfortunately lost the link.