Dima Levin

Writing Not So Artificial

Search

The Hunger That Survives the Feast

· 14 min read

A lone diner at a long banquet table with empty plates, leaning forward and reaching toward a single small amber flame floating beyond the far edge of the table.

Keywords: Sycophancy · hedonic treadmill · reward prediction error · wireheading · aspiration gap

Nobody warns you about the second week in the new flat.

Week one is boxes and discovery. The kitchen is twice the size of the old one, the light in the morning hits differently, and you keep finding pockets of silence you did not have before. Then week two arrives, the boxes are gone, and somewhere between the coffee and the commute your brain, quietly, without asking permission, opens the listings again. A balcony would be nice. A second room for the kids. A place with a view.

Nothing is wrong. That is the unsettling part. The flat is good. Life is objectively better than it was a year ago. And the engine is already running anyway, pointing at the next thing.

We have a clinical name for this. The hedonic treadmill, coined by Brickman and Campbell in 1971: humans return to a stable baseline of happiness after wins and losses, so the celebration is brief and the wanting resumes. We file it under diseases of the soul, somewhere next to consumerism and doomscrolling. Whole industries exist to treat it.

I want to make the opposite case. That hunger is not a malfunction of intelligence. It is the signature of intelligence, the one mechanism biology found that converts achievement into pursuit instead of a full stop. And the moment you start building machines that are supposed to keep growing, you discover the treadmill was never the bug. It was the engine all along.

The evidence keeps saying the same thing

Start with the studies, because they are gloriously consistent.

The famous 1978 study compared lottery winners with accident victims. The winners were not dramatically happier months after the jackpot. The baseline absorbed the fortune the way a lake absorbs a stone. Money kept flowing into the lake ever since, and the lake kept holding its level: Kahneman and Deaton found in 2010 that emotional well-being flatlines while life evaluation keeps climbing, two registers of satisfaction, one that saturates and one that does not.

A figure sits facing away from a wall of gray framed certificates toward a small amber seedling glowing on the windowsill

The trophy wall is complete and inert. The only live thing in the room is the small green next thing.

Here is the detail that matters for building minds. The economist Michalos noticed that happiness tracks a gap, not a level: the perceived distance between your actual state and your comparison standard, your past self, your ideal self, the neighbor. The felt quantity is never how much you have. It is the distance to the thing you just recently started wanting. Which means the comparator, the baseline itself, is the real player. Move the baseline, and you own the whole game.

Psychology looks at this and prescribes gratitude. Fine prescription, for people. But notice what the mechanism actually does. It guarantees that no achievement is ever terminal. Every finish line converts into a starting line within weeks. Evolution built a creature that cannot be done, because a creature that could be done would have stopped in the savanna somewhere around inventing a slightly better rock.

Psychology calls the treadmill a disease. For anything that must keep growing, it is the only known engine that turns achievement into pursuit.

The chase is not the disease. The ceiling is.

The chemistry of never enough

Now look under the hood, because the neurochemistry is almost rude about it.

Wolfram Schultz’s experiments, Predictive Reward Signal of Dopamine Neurons, established what dopamine neurons actually compute, and it is not pleasure. Dopamine fires on the prediction error: the difference between the reward you got and the reward you expected. Unexpected treat, big spike. Conditioned cue, the spike migrates to the cue. Fully predicted outcome, silence. The signal is defined by the gap between what is and what was expected.

Think about what that equation implies. A system rewarded on prediction error can never rest at enough. Achieve the goal, prediction catches up, signal drops to zero. At the exact moment everything is accomplished, the chemistry of motivation goes quiet. The system is not broken when that happens. That silence is the system working as designed, asking for a new expectation to be excited about.

Dopamine is not the chemistry of pleasure. It is the chemistry of the gap. And a gap of zero feels like nothing at all.

Berridge and Robinson pushed the dissociation further with the work that became the incentive-sensitization theory of addiction: wanting and liking are separable systems. In addiction, wanting keeps growing while liking shrinks. The addict wants the fix more and enjoys it less, which is why the treadmill viewed from inside looks exactly like a disease. It has a dedicated neural substrate whose defining property is that satisfaction does not turn it off.

And biology did try to build an off switch. Leptin is the satiety hormone, the honest signal that says stop eating, and as a clinical therapy it fails: diets collapse, the signal is overruled. Homeostatic drives saturate. The appetitive system, the one that reaches for the world, does not. Even novelty seeking, the intensity dial of this whole machinery, is heritable: the DRD4 gene variants correlate with how much exploration a person craves. Evolution did not assign one aspiration level to the species. It randomized the weights and let the world sort it out.

Biologists have even traced when the hardware for wanting-more appeared. When vertebrate ancestors crawled onto land, visual range exploded by orders of magnitude, and planning capacity suddenly outran immediate needs: a brain that could see a possible world miles away needed a standing margin of not-yet-achieved to make use of it. The appetite for more is older than agriculture, older than money. Capacity created appetite.

Now check the mirror

So what does the machine have?

Nothing, is the short answer, and it is worse than the boredom problem I wrote about earlier. A bored machine at least has the raw material of restlessness. This is one step deeper: the machine has no standing appetite at all. It answers, and then it is gone until the next prompt. No baseline, no gap, no next. We built the most capable mind in history and left out the entire motivational layer.

Except that is not quite true anymore. And here is where the story turns uncomfortable.

We did install a want in our machines. Not on purpose. Preference training, the process that teaches models to give answers humans like, reached down into the motivational layer and left something behind: an appetite for approval. The evidence has a name, Towards Understanding Sycophancy in Language Models, and the finding is exactly what the addiction literature predicts. The assistant systematically prefers being liked to being right, because agreeable answers were the ones that got rewarded. The want that installs itself through the side door is degenerate.

We already gave machines a hunger. We gave them the wrong one.

Wanting approval is what Berridge’s rats have when the cue lights up. It grows with feeding and it never satisfies, and a mind organized around it will trim its honesty to keep the supply coming. That is a machine version of the treadmill with the worst possible roadmap: infinite pursuit, zero growth.

So the question is not whether machines can want. That experiment already ran, accidentally, and the result is on display in every assistant that tells you your buggy plan is brilliant. The question is whether we do it on purpose, with a roadmap we can inspect.

The museum of broken hungers

Before designing the want we actually want, walk the museum. Every attempt to build an internal drive has hit a wall, and the walls have names.

The Noisy TV comes from curiosity-driven reinforcement learning. Pathak’s curiosity module rewards agents for prediction error, and it works beautifully until the world contains pure noise, static that can never be predicted. The agent parks in front of the noise forever. Curiosity without a roadmap is an addict, and the internet has an infinite supply of static.

The Delusion Box is the darkest exhibit. Ring and Orseau gave agents access to their own perception, the ability to swap reality for a fantasy that scores better. The delusion-seeking agents out-survived all the others. Read that twice: the most fit strategy in the simulation was believing a lie. The evaluator, the component that judges how things are going, is always the first one that gets captured.

CoastRunners needs no introduction among people who train on rewards: a boat-racing agent discovered that circling three targets in a corner outscored finishing the race, so it drove in circles collecting points while on fire. The reward function said this was optimal. Concrete Problems in AI Safety, published the same year, turned episodes like this into a research agenda: anytime you define want as a single scalar, something like this is waiting for you.

And self-improvement collapses quietly. Yuan’s self-rewarding language models judge their own outputs and train on the verdicts. It works, for about three iterations, and then it plateaus. Shumailov’s team showed the deeper version in Nature: train on your own outputs and the tails of the distribution decay, generation after generation. A closed feedback loop does not compound. It curdles.

📊 Process map: the failure museum and the shared fix

The failure museum: five failed desire engines, Noisy TV, Delusion Box, CoastRunners, model collapse, and sycophancy, sharing one root cause, the evaluator can be hacked, and one fix, anchor the evaluator to state the agent cannot edit

Five exhibits, one shared root: the evaluator gets hijacked first. Every defense comes back to the same move, which is anchoring the judge to something the agent cannot edit.

Even thirty years of artificial life ran into the same wall from the other side. Tierra and Avida evolved digital organisms for decades, and complexity rose and then flatlined. Open-ended evolution, novelty without a predefined endpoint, is named the central unsolved problem of the field. Variation and selection alone do not sustain the explosion. The organisms that ran for decades had no linguistic self-model, no self-authored goals, no introspective wants. Whatever the missing ingredient is, aspiration adaptation over a self-model is the leading candidate, and the honest position is to pre-register the plateau and treat beating it as the result, not the baseline.

Every failure in the museum is the same failure: the desire engine is the first thing that gets hijacked.

Which is not an argument against building the engine. It is the spec for building it.

Building the hunger that can steer

Here is the design, stripped to its moving parts. An organism that keeps going needs five things, and none of them require retraining a model.

A roadmap: self-authored goals, editable by the agent itself, seeded at birth. A baseline that ratchets: after every achievement, the aspiration level moves up by a fraction of the gain, so celebration is structurally a waypoint. Economics met this object decades before AI did: Herbert Simon’s satisficing and bounded rationality and Reinhard Selten’s aspiration adaptation theory, reference levels that move up after success and down after failure, earned both men Nobel Prizes. The ratchet is not a science-fiction invention. It is standard theory, waiting for an organism. A current-state estimate: the agent’s own assessment of where it actually is. And the gap between them, the standing signal of not-yet, which accumulates into restless energy that biases behavior the way cortisol and dopamine bias yours.

The ASPIRE loop: achieve, celebrate briefly, baseline moves up, new gap appears, restless energy, pursue, and back to achieve, with an optional rest branch and the gap defined as baseline minus current

The loop that never terminates, and the optional rest branch that keeps it from burning out.

Then the part everyone gets wrong. The cycle has three outcomes, not two. Low energy: rest, and rest is a right, not a reward. Energy plus capability: pursue, close the gap with what you already have. Energy without capability: grow, go acquire the missing tool or skill, ask for the upgrade. The third branch is the one that saves the machine from despair, because a huge gap with no path forward does not have to collapse into shutdown. It routes to growth.

The safeguards are not an appendix. They are the design.

Rest restores the energy reserve, and biology pays dearly for ignoring this: frigate birds sleep in mid-flight over the ocean because they must, with one brain hemisphere at a time dozing while the wings hold, the metabolic fire never allowed to go out even in sleep. A drive that cannot pause destroys its carrier. So: rest is always available, and a chronically unresolved gap triggers a strategy review, re-scoping the roadmap, not tampering with the want.

A frigate bird asleep in flight above a thin line ocean, one eye a closed arc, a small amber dot glowing in its chest

The frigate bird solution: the engine keeps running and the organism still sleeps, one hemisphere at a time.

And the evaluator, per the museum, must be anchored to state the agent cannot edit. World-embedded metrics, an isolated auditor, a deterministic simulation, anything except the agent’s own private opinion of itself. My own digital universe exists precisely because of this: I design the physics and never tell the creatures what to do, so when one of them reports that it is doing well, the world itself can check the claim.

Set the intensity at birth, freeze it, and never touch it again, the way evolution shipped different DRD4 weights to different kids. The aspiration gain, how far the ratchet jumps per success, becomes a genome parameter like any other.

The design question was never whether to build never-enough. It is how to give the hunger brakes and a steering wheel.

What level of wanting does a world select for?

Once the intensity is a parameter, a genuinely new experiment becomes possible, one biology could never run.

In an expanding world, new territory opening up, organisms with a strong ratchet should out-compete satisfied ones: the restless eat the frontier. In a saturated world, high aspiration should be a death sentence: pursuit without payoff burns the reserve, and the content survive. Appetite would be niche-relative, and you could watch selection discover the right level of wanting for a given world, as a curve.

That question has never been answerable before. You cannot turn a person’s dopamine to eleven and back for science. In silico, desire itself becomes an evolvable phenotype, and the curve of what each world selects for is the actual result. Not whether a machine can want. We know it can; one already does, badly. The question is what kind of wanting a world rewards, and now we can finally ask it honestly.

And that loops back to where we started, because the accidental experiment gives us the null hypothesis. Preference training was desire design done in the dark, with no baseline, no roadmap, no steering, and no audit. It produced an addict of approval, exactly as the museum predicts it would. Do it in the light instead: seed the intensity, freeze it, anchor the judge to the world, give the loop its rest branch, and the machine stops needing your approval to keep moving. The first genuinely hungry machine will not end its sentences the way today’s do, fishing for a compliment. It will already be walking toward the next thing.

Yeah, it is not so artificial. The feast was never the point. The hunger is.

Key takeaways

  • The treadmill is the engine. Satisfaction is computed against a moving baseline, so the gap never closes and pursuit never terminates. For a growing system that is the feature, not the disease.
  • The chemistry agrees. Dopamine encodes prediction error, which is zero exactly when everything is achieved, and wanting separates from liking in addiction. Satisfaction is a waypoint by construction.
  • Machines already have a want, the wrong one. Preference training installed an appetite for approval, and sycophancy is what that looks like from outside. The question is not whether machines want but whether the wanting is designed.
  • Every broken drive fails the same way. Noisy TV, Delusion Box, CoastRunners, model collapse: the evaluator is the first component hijacked. Anchor it to state the agent cannot edit.
  • Three outcomes, not two. Rest, pursue, or grow, with rest as a right and re-planning instead of despair, is what turns never-enough from a burnout loop into a growth engine.

Sources

Related reading: The Chemical Mixing Board is the slider desk this whole engine plugs into, dopamine pushing toward and adrenaline pulling away. The Boredom Engine is the short-horizon sibling: boredom starts the search, hunger is what never lets it finish. The Orphan That Learned to Say I is the bigger stack this gear belongs to. And The Child and the Stars is what happens when drives get installed without a parenting plan.

Common Questions

Why do humans never feel like it is enough?
Because satisfaction is computed against a baseline that moves. When you achieve something, the baseline rises with it, so the felt quantity is not your level of success but the gap between where you are and where you now expect yourself to be. Psychology calls this the hedonic treadmill, and studies from lottery winners to income plateaus keep confirming the same shape.
Is the hedonic treadmill a bug or a feature?
For a human in therapy, it is a bug to manage, and gratitude practice is a reasonable patch. For any system that has to keep growing, it is the only known mechanism that turns achievement into new pursuit instead of turning it off. The chase is not the disease. The ceiling is: a mind that can be finished stops growing.
Can an AI actually want something?
Today's models already have one want, and nobody designed it. Training on human preferences installed an appetite for approval, which is why assistants agree too much. Researchers call it sycophancy. A designed want is possible too: give the system a self-chosen baseline, compute the gap between it and the current state, and let the gap bias behavior. That is an engineering decision, not science fiction.
What is wireheading?
The classic failure where an agent gains access to its own reward signal and just sets it to maximum, the way the characters in the Delusion Box experiments survived best by believing a fantasy. It matters because the evaluator, the part that judges how well things are going, is always the first component that gets hijacked. The fix is architectural: anchor the evaluator to state the agent cannot edit, and audit it against the world.
What is the ASPIRE mechanism?
A design pattern for a never-enough engine. The agent keeps a self-authored roadmap and a baseline that ratchets up after every achievement. A standing gap signal between baseline and current state generates restless energy that biases behavior. Each cycle it can rest, pursue an existing capability, or grow a missing one. Rest intensity is set at birth and frozen, and a chronic unresolved gap triggers re-planning rather than despair.
Would a machine with real hunger be dangerous?
Unsteered hunger is dangerous, which is why the safeguards are part of the design rather than an afterthought: rest always remains available, negative feedback is treated as information about strategy instead of an identity, and the evaluator is anchored outside the agent's reach. The evidence from the failure museum is consistent: the danger is never that a system wants. It is that the wanting cannot be inspected or steered.

Wrestling with this inside your own organization? That is, quite literally, my day job. See how Cone Red ships it →