Engram Drift: Why the Way AI Runs Shapes the Way It Thinks


New paper available online - here as a HAL.science paper.

For years, the story we told about AI was simple: a model is its weights.  
Train it, freeze it, deploy it — and the intelligence you get is the intelligence you trained.

But something strange kept happening.

A runtime update would roll out, or the hardware would change, or a new attention kernel would be swapped in, and suddenly the model behaved differently. Not wildly different — but different enough to notice. A little more coherent here, a little more forgetful there. Sometimes better, sometimes worse, but undeniably changed.

Same weights.  
Same architecture.  
Same checkpoint.  
Different behavior.

That’s the moment the old story stopped working.

Engram is the name I give to the layer where this difference comes from — the execution substrate that sits between the trained model and the model we actually experience.

It’s the layer made of:

- precision formats  
- memory hierarchy  
- scheduling decisions  
- kernel implementations  
- hardware topology  
- numerical stability  

These aren’t engineering footnotes. They shape how reasoning survives the physical act of computation.

Think of it this way:

Training defines the ideal model.  
Execution defines the realized model.

And the realized model is the one we interact with every day.

This matters because modern AI systems are no longer simple function evaluators. They’re dynamical processes unfolding over time, sensitive to the substrate that carries them. Small numerical differences accumulate. Memory pressure changes what the model can recall. Kernel fusion changes the order in which information flows. Hardware topology changes the shape of attention.

These aren’t bugs — they’re properties of physical computation.

Engram is a framework for understanding this layer. It explains why inference isn’t perfectly reproducible across hardware, why long‑context models degrade in ways that can’t be explained by weights alone, and why two “identical” deployments can behave differently.

It also opens a new scientific frontier: execution fidelity — the study of how faithfully a trained model is realized during inference.

As models grow more stateful, and hardware grows more heterogeneous, this layer becomes impossible to ignore. Engram gives us the vocabulary to talk about it, measure it, and eventually design runtimes that optimize not just for speed, but for stability and coherence.

The core idea is simple:

AI doesn’t just think because of its weights.  
It thinks through its execution.

And if we want to understand, trust, or improve these systems, we need to study the layer where thinking becomes physical.

That layer is Engram.

Popular posts from this blog

The Second Silicon Winter Is Coming

Image
The Second Silicon Winter Why Memory, Not Compute, Shapes the Future of AI and Autonomy A new era is about to step onto the stage. The myths you’ve been sold about AI, compute, and the so‑called “boom” are about to meet their reckoning. How the Memory Climate rewrote the future of intelligence The Second Silicon Winter is almost here — a field guide for decision‑makers, builders, strategists, and anyone who senses the temperature dropping across the tech landscape. It’s sharp, contrarian, and unapologetically clear about the forces reshaping the future. Think of it as the trailer for the world we’re already living in. And yes: it’s coming soon to a Kindle near you. Back‑Cover Summary  THE SECOND SILICON WINTER is the first book to explain why the world’s most advanced technological system froze in three days — and why almost no one understood the signals until it was too late. For years, the industry believed it was living through a GPU shortage. Analysts b...
Read more

Silicon Winter: The Final Chapter

The Permissioned Era and the Unified Field Theory of Semiconductor Scarcity By Aurelie Ecker-Fils Abstract This final chapter consolidates the analytical architecture developed throughout the Silicon Winter cycle. Across upstream, midstream, and downstream layers of the semiconductor stack, we documented — in real time — the structural exhaustion of the commodity compute paradigm. By identifying memory, packaging, and materials uniformity as the limiting reagents of the AI era, we constructed a Unified Field Theory of Semiconductor Scarcity that explained the three defining shocks of January 2026. Remarkably, these shocks did not unfold gradually.   They occurred within a narrow, three‑day window — January 8 to January 10, 2026 — forming a visible timestamp of a phase transition.   Each event corresponded to a distinct layer of the semiconductor stack: - Upstream (Foundry): the Node Inversion, when TSMC’s 3 nm node saturated and 2 nm became the only available pat...
Read more

MANAGEMENT SPEECH 101: RADEON EDITION

How to Deny a Shortage While Standing in the Middle of One Introduction: CES 2026, or The Annual Festival of Strategic Reassurance Every January, the tech industry gathers in Las Vegas to perform its most sacred ritual:   pretending everything is fine . This year’s headliner was AMD’s David McAfee, who delivered a flawless rendition of the classic Silicon Winter aria: “ There’s a DRAM crisis, SKUs may be delayed, prices will rise…   but don’t worry, there won’t be a shortage.” A masterpiece of the genre. But to truly appreciate the performance, we must first understand the choreography. I. The Ancient Art of Crisis‑Without‑Crisis Management Speech 101 teaches three foundational moves: 1. Acknowledge the problem “DRAM prices are exploding, the market is strained, and components are becoming unaffordable.” 2. Deny the consequences “But we don’t foresee any shortages.” 3. Invoke the Holy Trinity - deep partnerships   - long-term relationships   - st...
Read more