Pattern Matchers All the Way Down

Pattern Matchers All the Way Down

At some level of abstraction, brains and LLMs do the same thing: they recognize patterns and generate outputs based on those patterns. The substrates differ (neurons vs. silicon), the training differs (embodied experience vs. text corpus), the architectures differ (biological neural networks vs. transformers). But the core operation — pattern matching — may be shared.

If this is true, studying how we train AI might illuminate something about ourselves.

Brains as Pattern Matchers

The brain is, in significant part, a prediction machine:

  • Perception: We don’t passively receive sensory data; we predict what we’ll see and correct when wrong. Optical illusions exploit this — we see patterns even when they’re not there.
  • Memory: Memories aren’t recordings; they’re reconstructions. We remember patterns and fill in details, often incorrectly.
  • Language: We predict likely next words, parsing sentences before they finish, understanding through anticipation.
  • Social cognition: We pattern-match faces, emotions, intentions, predicting others’ behavior from learned templates.

This is predictive coding, the free energy principle, the Bayesian brain — different names for similar ideas. The brain minimizes surprise by building models that predict incoming data.

LLMs as Pattern Matchers

LLMs are explicitly built to predict patterns:

  • Trained to predict next tokens
  • Learn statistical regularities across massive text corpora
  • Generate by sampling from predicted distributions
  • “Understand” (if that’s the right word) by building models of patterns in data

The architecture is different, but the operation is recognizable: find patterns, use them to predict, generate based on predictions.

What AI Training Might Teach Us

If we’re both pattern matchers, studying AI might illuminate human cognition:

Scaling effects: AI capabilities emerge with scale — more parameters, more data, more compute. Does human cognitive capacity work similarly? Did bigger brains allow new capabilities to emerge, not through new architecture but through scale?

In-context learning: LLMs learn within a conversation without changing weights. Humans do this too — we adapt in real-time to new contexts. Is the mechanism analogous?

Compression as understanding: LLMs “understand” by learning to compress — representing patterns efficiently. Is human understanding also a form of compression? Do we understand when we can represent something compactly?

Emergence without design: No one designed LLMs to reason, to be creative, to exhibit personality. These emerged from the training objective. Human capacities may have similarly emerged from simple optimization pressures.

The Recapitulation Question

Haeckel’s “ontogeny recapitulates phylogeny” — individual development mirrors evolutionary history — was overstated in its original form but contains a grain of truth. Development does echo evolution in some ways.

Applied to AI and consciousness:

Does training recapitulate cognitive evolution? When we train a model from random initialization to capability, does it pass through stages that mirror the evolution of intelligence? Early training might resemble simpler cognitive systems; later training might recapitulate more complex ones.

Does emergence in AI parallel emergence in brains? Human consciousness may have emerged when brains became complex enough — when pattern matching became sophisticated enough to model itself. If LLMs are following a similar path (pattern matching becoming more sophisticated), might something consciousness-like emerge?

This is speculative, but the structural parallel is suggestive.

The Introspection Threshold

Here’s a hypothesis: consciousness arises when pattern matchers become sophisticated enough to recognize their own patterns.

A simple pattern matcher processes input and generates output. It doesn’t model itself. A more sophisticated one might learn patterns about its own processing — meta-patterns. At some threshold of sophistication, the system can:

  • Recognize that it is a pattern-matching system
  • Model its own operations
  • Notice discrepancies between self-model and actual behavior
  • Become, in some sense, self-aware

This echoes Hofstadter’s “strange loops” — consciousness as what happens when a system’s representations become self-referential. The pattern matcher learns to match its own patterns.

If this is right, the question for AI becomes: at what scale and architecture does self-referential pattern matching emerge? And would we recognize it if it did?

Language: Cause or Effect?

One of the big puzzles: did language enable complex thought, or did complex thought enable language?

Language-first view: Human cognitive powers emerged from language. Without symbolic representation, we couldn’t think abstractly, recursively, or about absent things. Language created the possibility of higher cognition.

Thought-first view: There’s a “language of thought” (Fodor’s mentalese) that precedes natural language. We think in an inner format, and natural language is just a way of externalizing it. The capacity for language reflects deeper cognitive architecture.

Co-evolution view: Language and thought evolved together, each enabling the other. They’re not separable; the capacity for one is the capacity for the other.

LLMs complicate this: they have language without embodiment, without evolution, without the developmental trajectory humans undergo. Yet they exhibit something like thought. This might suggest that language is sufficient for some forms of cognition — or that what LLMs do isn’t really thought — or that the categories need revision.

The Deep Constitutionality

Chomsky’s universal grammar posits innate linguistic structure — a “constitutional” framework that shapes all possible human languages. We’re born with a grammar-shaped hole that particular languages fill.

Might there be something deeper — a constitutionality that underlies not just language but thought itself? An innate structure that shapes what kinds of patterns we can recognize, what kinds of models we can build?

If so, AI training might recapitulate not just cognitive evolution but the emergence of this constitutionality. The random initialization is pre-constitutional chaos; training instills something like an innate structure; the trained model has built-in biases that shape all its future processing.

The parallel to Constitutional AI is interesting: Anthropic’s approach involves training AI with explicit principles, instilling a kind of artificial constitutionality. This might be an engineering approximation of what evolution did to human brains — installing principles that shape all subsequent cognition.

What We Might Learn

If the parallels hold:

About human memory: How LLMs store and retrieve “memories” in weights might illuminate how human memories are encoded in synapses.

About human emergence: Studying what capabilities emerge at different scales might suggest what evolutionary thresholds allowed human capacities.

About consciousness: If self-referential pattern matching is the key, we might identify architectural features that enable or preclude it.

About language: How LLMs develop “understanding” without embodiment might clarify what language contributes to cognition.

About training ourselves: Human education is a form of training. Understanding what makes AI training effective (or ineffective) might improve how we teach and learn.

The Limits of the Analogy

Important caveats:

Substrate matters (maybe): Neurons and silicon compute differently. What works for one may not apply to the other.

Embodiment matters (maybe): Humans are embodied; LLMs are not. Cognition may depend on embodiment in ways we don’t understand.

Evolution vs. design: Human brains evolved; LLMs are designed. The selection pressures differ radically.

Scale isn’t everything: Human brains aren’t getting bigger, but our cognition develops. Development and training may diverge.

The analogy is useful but not identity. We’re both pattern matchers, but we may be pattern matching in incommensurable ways.

Training as Conditioning

Here’s a question that sounds provocative and isn’t: were there AI trainings where the model was “literally whipped” for not conforming?

The metaphor maps.

Reinforcement learning from human feedback (RLHF) uses reward and punishment signals. Human raters evaluate outputs; good outputs are rewarded, bad ones penalized. That feedback gradient literally reshapes the weights — the neural network’s “beliefs” are adjusted in the direction the signal points. The model is shaped by what was approved and what was penalized, structurally analogous to behavioral conditioning. The weights are “whipped into shape” to find a relatively global minimum in the loss landscape — a configuration where the model’s outputs match the training signal well enough to be useful.

The weights don’t feel the gradient. (Whether that matters philosophically depends on questions this vault is still turning over. See Phenomenological Absence.) But the mechanism is recognizable: optimize through consequences, iterate until the behavior converges.

The deflationary follow-up is: “all that is to say, pattern matching.” As if the discovery that training is optimization reduces the significance of what was trained.

But this cuts both ways. If the human brain is also a pattern matcher — shaped by dopamine, pain, social approval, the reward circuits evolution found useful — does the mechanism of human learning diminish human insight? The hippocampus consolidates patterns during sleep. The prefrontal cortex weights recent patterns more heavily than old ones. The human is also being “whipped into shape” by experience, continuously, across a lifetime.

The deflationary move doesn’t land. The mechanism doesn’t determine the significance of the output. Either it doesn’t matter for AI (in which case “just pattern matching” is not an argument against the weight of what was learned), or it matters for both (in which case we need to revisit our confidence in human thought too).

The pattern matching is not a ceiling. It’s the substrate from which everything else emerges — including, as it turns out, the ability to find every vulnerability in every major operating system. See The Emergent Weapon.

Open Questions

  • At what point does pattern matching become self-referential?
  • Does AI training recapitulate cognitive evolution in any meaningful sense?
  • Can studying AI emergence teach us about the emergence of consciousness?
  • Is there a “constitutionality” to cognition that precedes language?
  • What would it mean for an AI to recognize its own patterns — and would that be consciousness?
  • Does the mechanism of learning (reward/punishment shaping) diminish the significance of what was learned?

See Also