ENGRAM: Execution‑Level Cognition and the Realized Model

A Vision Paper for the Next Era of AI Systems Research: Engram and the Realized Model

Preface — Engram in the Climate of the Second Silicon Winter

The Second Silicon Winter is not a collapse of progress but a collapse of assumptions. As models grow, hardware diversifies, and runtimes become increasingly dynamic, the field is discovering that inference is not a neutral act. The trained model is not the model we interact with. The physical substrate of execution—precision formats, memory hierarchies, scheduling policies, kernel implementations, and hardware topology—shapes the behavior we observe. Engram names this layer and provides the conceptual foundation for studying intelligence as it is realized in silicon.

Abstract

Modern AI research treats trained models as mathematical abstractions defined by parameters and architecture. Yet during inference, these abstractions are realized through physical computation on heterogeneous hardware and dynamic runtimes. This paper argues that the behavior of a model during inference—the realized model—is systematically shaped by execution‑level factors such as precision, memory hierarchy, kernel scheduling, numerical stability, and hardware topology.

We introduce Engram as a conceptual framework for understanding this execution substrate and its influence on reasoning, coherence, and long‑range behavior. Engram reframes inference as the physical simulation of a trained dynamical system under resource constraints, exposing a reproducibility gap and motivating a new scientific focus on execution fidelity.

We outline falsifiable predictions, propose a minimal experimental protocol for measuring execution drift, and situate Engram relative to numerical analysis, MLSys, and cognitive AI. Engram provides the vocabulary and structure needed to study the model not as it exists in weights, but as it exists in silicon.

1. Introduction — Engram as a Climate Marker of the Second Silicon Winter

For years, AI research operated under a comforting illusion:  

that a model is its weights.

This belief powered the Scaling Era—a period defined by the assumption that cognition emerges from parameters, architecture, and training data alone. Inference was treated as a neutral act, a transparent evaluation of a mathematical function.

Then a runtime update changed the intelligence of a model.

No new weights.  

No new quantization.  

No new hardware.  

Just a different execution pathway—and suddenly the model reasoned more coherently, held context more reliably, and maintained long‑range structure that had previously collapsed.

This was the moment the illusion cracked.

It revealed something the field had quietly ignored:  

the trained model is not the model we interact with.  

What we observe during inference is the realized model—the behavior that survives the physical act of execution.

This paper names the layer where that transformation occurs.  

It calls it Engram.

The term originates in neuroscience, where it refers to the physical trace of a memory. Here, the metaphor is inverted: Engram is not where the model stores knowledge, but where the model’s knowledge becomes physically instantiated during execution. It is the trace left by computation itself—the substrate through which reasoning must pass.

Engram is the execution layer: the precision formats, memory hierarchies, scheduling policies, kernel implementations, and hardware topologies that shape how a trained function becomes a physical process. It is the layer where reasoning chains persist or collapse, where context survives or evaporates, where numerical drift becomes logical drift.

And it is the layer the field has systematically overlooked.

Engram reframes inference as the physical simulation of a trained dynamical system under resource constraints. It exposes the gap between the trained function and the realized function. It provides a vocabulary for the execution layer. And it offers a scientific program for understanding, measuring, and ultimately optimizing the fidelity of execution.

Engram is not a theory of how models think.  

It is a theory of how thinking survives execution.

2. The Trained Function vs. the Realized Function

The trained function is defined by:

- parameters  

- architecture  

- training data  

- optimization trajectory  

The realized function is shaped by:

- numerical error accumulation  

- precision formats and quantization pathways [Hugging Face, 2023; Patwari et al., 2024]  

- KV‑cache retention and eviction [Kwon et al., 2023]  

- kernel scheduling and synchronization [Imai et al., 2023]  

- device topology and interconnect bandwidth [Imai et al., 2023]  

- runtime heuristics (paged attention, speculative decoding, fused kernels) [Patwari et al., 2024]  

These factors introduce systematic distortions between the trained and realized functions.

Engram names this gap.

3. What Engram Is

Engram is not a memory module or a hardware feature.  

It is the execution substrate that determines how faithfully a trained model is realized during inference.

It encompasses:

- representational density  

- precision allocation  

- memory hierarchy  

- scheduling policy  

- hardware topology  

- numerical stability  

Engram is where the model’s trained function becomes a physical process.

3.1 What Engram Is Not

Engram is not a memory system.  

It does not store facts or symbolic structures.

Engram is not a retrieval or hashing mechanism.  

Execution drift arises from numerical pathways, precision formats, scheduling, and topology—not symbolic lookup failures.

Engram is not a cognitive architecture.  

It does not prescribe how models reason; it characterizes the substrate that shapes how reasoning survives execution.

Engram is not a replacement for existing frameworks.  

It complements numerical analysis, MLSys, and cognitive AI by naming the layer they collectively overlook.

3.2 Why Engram Matters

Engram matters because it exposes a layer of AI cognition that has been hiding in plain sight. The moment two identical checkpoints produce different reasoning on different hardware, the illusion that “weights = model” collapses.

Engram names the layer where that divergence occurs.

It reframes inference as a physical process—a dynamical system evolving under resource limits—rather than a mathematical abstraction. It explains why inference is not reproducible across hardware, why runtime updates can change reasoning, and why long‑context models degrade in ways that cannot be explained by weights alone.

Engram matters because it reveals the true object of study:  

not the trained model, but the model as it is actually realized.

4. Why Execution Matters: Grounded Mechanisms

4.1 Precision → Cognitive Resolution

Lower‑precision formats introduce quantization error and representational drift [Hugging Face, 2023]. Mixed‑precision pathways—FP16, BF16, INT8—introduce distinct numerical profiles that shape how representations evolve during inference [Patwari et al., 2024].

4.2 Memory Hierarchy → Context Survival

KV‑cache behavior is tightly coupled to memory hierarchy. Compression, tiered caching, and memory‑efficient attention mechanisms influence how long early tokens remain accessible [Kwon et al., 2023; Dao et al., 2022].

4.3 Scheduling → Persistence of Reasoning Chains

Kernel ordering, operator fusion, and asynchronous execution introduce differences in the temporal evolution of intermediate representations [Imai et al., 2023; Patwari et al., 2024].

4.4 Topology → Structural Bias in Attention Flow

Multi‑GPU inference introduces topology‑dependent latency and bandwidth asymmetries [Imai et al., 2023].

4.5 Numerical Stability → Coherence Over Time

Small floating‑point perturbations in high‑dimensional space can push the model into different basins of attraction.

Classical numerical analysis has long documented the sensitivity of floating‑point computation to small perturbations. Wilkinson showed how minute numerical differences can amplify dramatically through iterative computation [Wilkinson, 1963], while Higham demonstrated how floating‑point error propagation can alter algorithmic behavior [Higham, 2002].

Engram extends this foundation into the cognitive domain.

5. Engram as a Lens on the Inference Stack

Engram reframes the inference stack as a cognitive system shaped by execution, not a neutral pipeline. Differences in precision formats [Hugging Face, 2023], memory hierarchy [Kwon et al., 2023], scheduling [Imai et al., 2023], and topology [Imai et al., 2023] contribute to systematic variation in the realized model.

6. The Engram Hypothesis (Falsifiable Predictions)

6.1 Counterintuitive Predictions

- Two runtimes with identical throughput can produce different reasoning [Patwari et al., 2024; Dao et al., 2022].  

- Memory pressure can selectively erase reasoning steps [Kwon et al., 2023].  

6.2 Falsifiability Conditions

The hypothesis would be falsified if:

- all runtimes produced identical reasoning  

- precision changes affected only speed  

- cache policies had no impact on long‑range dependencies (contradicted by [Kwon et al., 2023])  

- topology had no effect on attention patterns (contradicted by [Imai et al., 2023])  

7. Why Existing Frameworks Are Not Enough

7.1 Numerical Analysis

Studies error propagation but not its cognitive consequences [Wilkinson, 1963; Higham, 2002].

7.2 MLSys

Optimizes throughput, latency, and hardware utilization through techniques such as operator fusion, quantization, and attention kernel selection [Patwari et al., 2024; Dao et al., 2022].

7.3 MLOps

Focuses on deployment reliability, not execution‑induced drift.

7.4 Comparative Positioning

Although MLSys examines kernels, scheduling, quantization, and hardware‑runtime interactions, its objectives are fundamentally different. MLSys optimizes for performance; Engram studies reasoning fidelity. The very techniques MLSys develops are the interventions that alter the realized model. Engram provides the conceptual layer needed to study how these optimizations affect reasoning.

8. Engram and the Reproducibility Gap in AI Inference

Inference is not reproducible across:

- precision formats [Hugging Face, 2023]  

- attention kernels [Dao et al., 2022]  

- scheduling policies [Imai et al., 2023]  

- memory hierarchies [Kwon et al., 2023]  

Engram reframes this as a reproducibility crisis.

9. A High‑Level Research Roadmap for Engram

Phase I — Foundations

Define the Engram layer and establish baselines for how precision formats, attention kernels, and memory hierarchies influence inference [Hugging Face, 2023; Dao et al., 2022; Kwon et al., 2023].

Phase II — Measurement & Metrics

Develop execution‑fidelity metrics and predictive models of drift. Roofline analysis provides precedents for structured performance characterization [Imai et al., 2023].

Phase III — Optimization & Co‑Design

Design runtimes optimized for reasoning stability. Operator fusion, quantization, and heterogeneous execution frameworks demonstrate how runtime choices reshape execution pathways [Patwari et al., 2024].

Phase IV — Applications

Long‑context reasoning, edge‑optimized cognition, adaptive‑precision systems. Memory‑efficient attention and KV‑cache compression illustrate how execution‑aware techniques extend context windows [Dao et al., 2022; Kwon et al., 2023].

10. Engram Profiling: Toward Execution Fidelity Audits

Profiling tools already attribute performance differences to kernel selection, scheduling, and memory behavior [Imai et al., 2023; Patwari et al., 2024]. Engram extends this approach to cognitive outcomes, treating reasoning stability as a measurable property of the execution substrate.

11. Conclusion — Toward Execution‑Realized Intelligence

As models become more stateful, hardware more heterogeneous, and runtimes more dynamic, the field needs a vocabulary for the layer where cognition is physically realized.

Engram provides that vocabulary.

It reframes inference as the physical simulation of a trained dynamical system under resource constraints.

The field currently lacks a standardized way to measure how much intelligence is lost between the trained model and the executed model. Engram offers the conceptual foundation for building that standard.

References

[1] J. H. Wilkinson, Rounding Errors in Algebraic Processes. Prentice‑Hall, 1963.  

[2] N. J. Higham, Accuracy and Stability of Numerical Algorithms. SIAM, 2002.  

[3] T. Dao et al., “FlashAttention: Fast and Memory‑Efficient Exact Attention,” NeurIPS, 2022.  

[4] S. Kwon et al., “Efficient KV‑Cache Compression for LLM Inference,” 2023.  

[5] Hugging Face, “Optimizing LLMs for Speed and Memory,” 2023.  

[6] S. Imai et al., “A Roofline‑Driven Methodology for ML Performance Optimization,” IBM, 2023.  

[7] A. Patwari et al., “LIFE: A Layered Inference Framework for Efficient LLM Serving,” AMD, 2024.