☰ sui.gn / fundamentals

The Term

Contextual Momentum: the tendency of a generated representation to preserve its own continuity and coherence while its correspondence with the described state decays.
“Coherence is a relationship between statements. Correctness is a relationship between a statement and the state it describes.” — suiGn

A distinction as old as the coherence and correspondence theories of truth.

This is a descriptive term, not a discovery of a new property of transformers. It names a pattern that is already visible: once a context has committed to a picture of the world, each new generation tends to extend that picture rather than check it. The text stays consistent with itself. It does not stay consistent with the world.

It is related to the observation in Zhang et al. (2023), How Language Model Hallucinations Can Snowball, that models tend to commit to an early mistake and then generate further claims to justify it. Contextual Momentum describes a broader case.

No Initial Hallucination Required

Snowballing begins with an error. Contextual Momentum does not need one. A representation can begin perfectly correct — an accurate observation, faithfully recorded — and still go wrong, because the world moves and the representation does not. The observation does not become false by being misread. It becomes false by becoming old.

Nothing in the text marks that moment. A stale fact reads exactly like a fresh one. The context carries it forward with the same confidence it had when it was true.

Coherence Is Not Correctness

A representation can maximize coherence while correctness quietly falls away, and from inside the representation the two are indistinguishable.

This is why a representation cannot be its own evidence. Asking the context whether it is still accurate returns more context — generated by the same process, anchored to the same picture. The check and the claim live in the same place. It is the same shape described in me.explain(): a generated explanation is not evidence about the computation; the computation's own record is.

Describing Authority Is Not Having It

A context can describe authority. It cannot manufacture it. Text that says "the battery is sufficient" or "this action is permitted" is a description of a state, not the state, and not permission. If generated text can grant itself the authority to act, then whatever drifted into the context has that authority too.

The resolution is a separation of roles: a model proposes, a runtime evaluates. The proposal may be fluent, coherent and wrong. The evaluation reads the current state and either agrees or does not. This is the same line drawn in the Byzantine pages — trust what can be checked outside the participant making the claim — and in the Axiom of Distinction: a description and the thing described are not the same object, and a system has to keep them apart. Or, as the site puts it, the code is the ultimate truth.

A Small Illustration

A robot's battery was observed at 90. A context records it and reasons forward: the robot can proceed. Later the battery is at 10. The context, unless it is told, still says 90, and still says the robot can proceed — coherently. 

me.robot["="]("canProceed", "battery >= minimumBattery")

battery: 90  →  canProceed: true
battery: 10  →  canProceed: false

In .me (verified with this.me 4.2.1), canProceed is not remembered; it is derived from the current state, and recomputes when the state changes. The generated representation can keep its momentum. The derived value does not have any. See Equations for how derivations are declared, and me.explain() for how their record is returned.

What Is Not Shown Yet

The illustration shows that the runtime value changes when the state changes. It does not yet show, end to end, a model proposing an action from stale context and the runtime rejecting it. That experiment is pending, and this page does not claim its result.

Related material: the LLMS demo and the .me repository.

Technical article on Hashnode