Writing Not So Artificial

The Science Behind the Mirror

· 3 min read

A hand mirror whose reflection is built from many small citation marks and footnote numbers, one highlighted in amber.

This series is sci-pop, not a research paper. But the claims are not invented. Here is where they come from.

On attention and context windows

Claim: language models have finite attention windows, similar to human working-memory limits.

  • Kahneman (2011), Thinking, Fast and Slow: attention established as a scarce cognitive resource.
  • Vaswani et al. (2017), the transformer architecture: attention mechanisms explicitly modeled on selective attention.
  • Context-window research at Anthropic and OpenAI: diminishing returns past a certain token count, mirroring human cognitive-load limits.

On confabulation and hallucination

Claim: models confabulate for the same reason humans do, gap-filling under uncertainty.

  • Michael Levin, diverse-intelligence work: all cognitive systems construct plausible narratives from incomplete information.
  • Hallucination research (Zhang et al., 2023; Anthropic interpretability): hallucinations rise with context dispersion and uncertainty.
  • Human memory research (Loftus, Schacter): memory is reconstructive, not reproductive. We fill gaps with plausible fabrications.

On vectors and meaning

Claim: models operate on semantic vectors that encode relationships, not just word proximity.

On multi-signal motivation

Claim: complex behavior needs multiple coordinating signals, not single-objective optimization.

  • Neuromodulation research (dopamine, serotonin, acetylcholine systems): the brain uses many parallel reward and attention signals.
  • RLHF limitations (Anthropic, OpenAI): single-signal optimization produces shallow alignment.
  • Constitutional AI (Anthropic, Bai et al. 2022): multi-principle alignment produces more robust behavior.

On agent societies

Claim: multi-agent coordination follows patterns from human organizational theory.

The mirror is still forming. The science is still moving. But the direction is clear. We are not building artificial minds. We are building digital partners.


Related reading: The Vector turns the vectors entry in this annex back into the essay it came from: how meaning is stored, and why text can be deeper than video.

Sources

Wrestling with this inside your own organization? That is, quite literally, my day job. See how Cone Red ships it →