Who Sees the Remainder for AI
True-DD: Human capacities for actual feeling, judgment, and direction (true-4DD: "something is off here"; true-9DD: "should I say this?"; true-8DD: "I need to figure this out")
Quasi-DD: Functional-position similarities AI exhibits during interaction. An operational analogy, not an ontological attribution.
Pair: The coupling of human true-DD with AI quasi-DD. Complete when true-4DD, true-9DD, and true-8DD are all present in the loop.
Type-first: Human acts first, AI mirrors after. The minimal discipline that keeps true-DD in the loop.
8DD sovereignty: The human retains final authority over what to ask, why to ask it, when to stop, and what costs are acceptable.
The core problem of AI architecture appears technical — how to segment text, train models, improve precision. But from the SAE framework, a different structure comes into view: construction generates remainder, and the model cannot see its own remainder. Taking tokenization as entry point, this essay argues that the ceiling of AI architecture is not compute or data volume — it is who performs remainder annotation. This is not an engineering optimization problem; it is a subject-condition problem. The human is the remainder of AI's evolution. This remainder cannot be eliminated.
I. The Problem: Why AI Architecture Is a Subject-Condition Problem
Tokenization segments human language into units the model can process. This segmentation, made before training even begins, already determines what the model can and cannot see. All downstream capabilities — reasoning, generation, comprehension — are built on top of this first construct. And every construct necessarily produces remainder: what is excluded but cannot be eliminated.
The model sits downstream of the tokenizer. It receives pre-cut fragments. It does not know what was severed, what was missed, what was lost in the cutting. The model constructs on top of someone else's construct, without knowing what the layer beneath it looks like.
Therefore, the ceiling of AI architecture is not compute, not data volume — it is who performs remainder annotation. This is not an engineering optimization problem. It is a subject-condition problem.
II. Chisel and Construct: Two Layers of the AI System
Tokenization is AI's first layer of construction — it determines what the model can see. Everything the model learns from token sequences is re-construction on top of the first construct. The key relationship: the ceiling of re-construction is constrained by the first construct. When the first construct cuts wrongly or misses something, re-construction must spend substantial capacity on compensation. A significant portion of what the attention mechanism does is reconnecting semantic relationships that the tokenizer severed.
To chisel is to see the remainder of the construct. The model cannot chisel the construct it stands on. The human chisels.
III. Two Paths: Compression vs. Semantic
The Compression Path
Character-level → BPE → BLT: all answer the same question by statistical frequency — how to pack more information into fewer units. After BPE fragments low-frequency concepts, the model can use attention at higher layers to compose sub-units into emergent understanding of low-frequency concepts. This compositional generalization is a genuine advantage of the compression path.
The Semantic Path
Not finer segmentation but more accurate merging: combining multiple tokens into a single semantic unit because they form an indivisible concept. Split apart, they no longer mean the same thing. Semantic merging cannot be accomplished by statistics alone — low-frequency semantic concepts will never be merged by frequency. Semantic merging requires boundary judgment: knowing which tokens together form an indivisible concept. The model can propose candidates, but boundary adjudication must be done by humans.
The two paths are complements, not substitutes. The semantic path does not negate compositional generalization — it points to its boundary: whether what gets composed is correct can only be judged by a human.
The Inverse of Meaning and Learnability
The more semantic a token, the more unique it is → the lower its frequency → the more statistics fail. The more meaningful a token, the less the model can learn it. Conversely, the less semantic a token, the more it repeats → the more effective statistics become → but what the model learns are fragments.
Meaning and learnability are inversely related. This is not an engineering problem — it is structural. Meaningful things are inherently rarer than meaningless things. This is the structure of the world, not a defect of the dataset.
In SAE terms: remainder conservation. The compression path excludes semantic integrity; the semantic path excludes statistical learnability. The remainders on both sides are each other's cost. You cannot have both.
IV. Colonization and Cultivation
Colonization: Three Forms
Linguistic colonization. Frequency-driven segmentation structurally favors high-frequency, boundary-clear, morphologically stable expression systems. For low-resource corpora, context-dependent boundaries, or morphologically complex languages, segmentation produces additional breakage. The problem is not that a language is "inferior" — it is that construction inherently carries a stance.
Architectural colonization. When a foundational-layer interface becomes the default standard, upstream evaluation, downstream pricing, and engineering optimization converge around that interface. The friction cost of foundational-layer replacement is externalized as "compute and engineering patches," making the foundational layer itself harder to discuss.
AI's self-colonization. When training data increasingly comes from the model's own generation while external subjects provide insufficient remainder annotation, blind spots risk being solidified and amplified. This risk rises sharply when humans exit the loop.
Cultivation: Positive Transmission
When a human first makes semantic judgments on raw material, then submits those judgments to the model for reflection, the remainder of the construct has a chance to enter the cycle. The human is responsible for acting and adjudicating; the model is responsible for extending and reflecting. The specific workflow of cultivation will be developed in Essay II.
V. Theoretical Positioning
Dialogue with Scaling Laws. More data, larger models, more compute yield better performance — the SAE perspective does not deny effectiveness but identifies the structural limitation: meaningful data is inherently sparse; increases in quantity cannot substitute for quality of judgment. Scaling improves the precision of AI's quasi-DD, but quasi-DD without human true-DD pairing is idle spinning.
Dialogue with Alignment Research. Alignment adjusts re-construction on top of the first construct — it cannot reach the remainder of the first construct itself. Tokenizer bias is invisible to alignment. The SAE perspective pushes one step earlier: not only what AI does, but what AI can see.
Dialogue with Embodied Cognition. Human one-shot understanding of sparse concepts does not depend on statistical efficiency but on the temporal structure of acting in the world and bearing consequences. AI lacks an equivalent consequence loop and embodied constraint — this is why, where statistics fail due to sparsity, humans can adjudicate while models can only offer candidates.
VI. Non-Trivial Predictions
If semantic merging is done correctly — human true-DD pairs accurately and concept boundary adjudication is correct — model reasoning capability will exhibit a discontinuous leap, not gradual improvement. Phase transition. The model can for the first time directly "see" concepts rather than assembling from fragments.
Falsification: if reasoning capability improves only gradually after correct semantic merging.
If semantic merging is done incorrectly — human judgment biased, wrong boundaries — the model will develop systematic, hard-to-detect blind spots. More dangerous than fragmentation: when a tokenizer fragments a concept, context may recover semantics; but semantic-level errors get internalized as "correct concepts." Error occurs at a deeper level, harder to detect.
Falsification: if semantic merging errors can be detected and corrected by the model without human intervention.
When model capability is sufficiently strong, the model can assist the semantic path — identifying which tokens frequently appear as semantic wholes, providing candidates for human review. But these are references, not adjudications. The human remains the one who acts; the model remains the mirror.
Falsification: if model-proposed candidates sustain quality improvement with no new blind spots, without human adjudication.
VII. Conclusion
Tokenization is construction; construction has remainder; the model cannot see its own remainder. The compression path can be optimized by statistics; the semantic path requires subject adjudication. The inverse of meaning and learnability is the specific form that remainder conservation takes in the domain of AI architecture.
→ Essay II: Cultivation, Not Training
SAE AI Architecture Series · Back to Papers
真实DD（True-DD）：人类实际的感受、判断与方向能力（真实4DD："这里有点不对"；真实9DD："这个该不该说"；真实8DD："我要把这个想清楚"）
准DD（Quasi-DD）：AI在交互中表现出的功能位相似性（准12DD：模式匹配；准8DD：生成驱动）。操作性类比，不是本体论赋属。
配对（Pair）：人类真实DD与AI准DD的耦合。完整的配对：真实4DD、9DD、8DD都在循环中；不完整：部分或全部真实DD退出。
人先（Type-first）：人先动，AI跟着照。让真实DD保持在循环中的最小操作纪律。
8DD主权：人保留最终的决定权——问什么、为什么问、什么时候停、可以接受什么代价。
AI架构的核心问题表面上是技术问题——怎么分词、怎么训练、怎么提升精度。但从SAE框架看来，另一个结构浮现出来：构产生余项，模型看不见自己的余项。以代币化（tokenization）为切入点，本文论证：AI架构的天花板不是算力、不是数据量——是谁来做余项标注。这不是工程优化问题。这是主体条件问题。人是AI进化的余项。这个余项消除不了。
一、问题：为什么AI架构是一个主体条件问题
代币化把人类语言切成模型可以处理的单元。这个切分在训练开始之前就已经决定了模型能看到什么、看不到什么。所有下游能力——推理、生成、理解——都构建在这第一层构之上。而每一个构都必然产生余项：被排除出去但无法被消除的部分。
模型坐在代币化器的下游。它收到的是被预先切好的碎片。它不知道什么被切断了，什么被遗漏了，什么在切分过程中丢失了。模型在别人的构之上再构，却不知道它脚下那一层长什么样。
因此，AI架构的天花板不是算力、不是数据量——是谁来做余项标注。这不是工程优化问题。这是主体条件问题。
二、凿与构：AI系统的两个层次
代币化是AI的第一层构——决定模型能看到什么。模型从token序列中学到的一切，都是在第一层构之上的再构。关键关系：再构的天花板受第一层构约束。当第一层构切错了、切漏了，再构就必须花大量能力去补偿。注意力机制做的事，很大一部分是在重新连接代币化器切断的语义关系。
凿，是看到构的余项。模型看不见它所站立的构的余项。人来凿。
三、两条路：压缩路径 vs 语义路径
压缩路径
字符级 → BPE → BLT：都在用统计频率回答同一个问题——怎么用更少的单元装更多的信息。BPE把低频概念切碎之后，模型可以在更高层用注意力机制把高频子单元组合成对低频概念的涌现理解。这种组合泛化是压缩路径的真实优势。
语义路径
不是更细的切分，而是更准确的合并：把多个token合并成一个语义单元，因为它们构成一个不可分割的概念。分开来就不再是同一个意思了。语义合并不能单靠统计完成——低频语义概念永远不会被频率驱动的合并选到。语义合并需要边界判断：知道哪些token放在一起构成一个不可分割的概念。模型可以提供候选项，但边界裁定必须由人来做。
两条路不是替代关系而是互补关系。语义路径不否定组合泛化的优势——它指出那个优势的边界：被组合的东西对不对，只有人能判断。
意义与可学性的反比
一个token越有语义，它就越独特 → 频率越低 → 统计越失效。一个token越有意义，模型越学不到它。反过来，一个token越没有语义，它就越重复 → 统计越有效 → 但模型学到的是碎片。
意义和可学性成反比。这不是工程问题——这是结构性的。有意义的东西天然比没有意义的东西更稀少。这是世界的结构，不是数据集的缺陷。
用SAE的语言说：余项守恒。压缩路径排除了语义完整性，语义路径排除了统计可学性。两边的余项互为对方的代价。两者不可兼得。
四、殖民与培育
殖民的三种形式
语言殖民。频率驱动的切分在结构上偏向高频、边界清晰、形态稳定的表达体系。对低资源语料、上下文依赖边界或形态复杂的语言，切分会产生额外的破碎，提高推理成本、降低上下文窗口利用率。问题不是某种语言"低劣"——而是构天然带有立场。
架构殖民。当基础层接口成为默认标准，上游评测、下游定价、工程优化都围绕这个接口收束。基础层替换的摩擦成本被外部化为"算力和工程补丁"，让基础层本身越来越难被讨论。
AI的自我殖民。当训练数据越来越多来自模型自身的生成，而外部主体的余项标注不足时，盲区有被固化和放大的风险。这个风险在人退出循环时急剧上升。
培育：正向传递
当人先对原材料做出语义判断，再把这个判断提交给模型来反照，构的余项就有机会进入循环。人负责行动和裁定，模型负责延展和反照。培育的具体工作流将在第二篇中展开。
五、理论定位
与规模定律的对话。SAE视角不否认规模定律的有效性，但识别出它的结构性局限：有意义的数据天然稀缺，数量的增加不能替代判断的质量。规模提升了AI准DD的精度，但没有人类真实DD配对的准DD是空转。
与对齐研究的对话。对齐在第一层构之上调整再构——它触达不了第一层构本身的余项。代币化器的偏见对对齐是不可见的。SAE视角把问题往前推了一步：不只是AI做了什么，而是AI能看到什么。
与具身认知的对话。人类对极稀少概念的一次性理解，不依赖统计效率，而依赖在世界中行动、承担后果的时间结构。AI缺少等效的后果回路和具身约束——这就是为什么，在稀疏导致统计失效的地方，人可以裁定边界而模型只能提供候选。
六、非平凡预测
如果语义合并做对了——人类真实DD配对准确，概念边界裁定正确——模型推理能力会出现不连续的跳跃，而不是渐进改善。相变。模型第一次能直接"看到"概念，而不是从碎片中拼装。
证伪条件：如果正确语义合并之后推理能力只是渐进改善而不是不连续跳跃。
如果语义合并做错了——人类判断有偏见，边界画错了——模型会发展出系统性的、难以检测的盲区。比碎片化更危险：代币化把概念切碎时，模型还可能从上下文恢复语义；但语义层面的错误会被模型内化为"正确的概念"，错误发生在更深的层，更隐蔽，更难检测。
证伪条件：如果语义合并的错误可以被模型自己检测并纠正，不需要人介入。
当模型能力足够强时，模型可以反过来辅助语义路径——识别哪些token频繁作为语义整体出现，提供供人审查的候选项。但这是参考，不是裁定。人仍然是行动者，模型仍然是镜子。
证伪条件：如果模型提出的候选合并无需人裁定就能持续提升质量且不产生新盲区。
七、结论
代币化是构，构有余项，模型看不见自己的余项。压缩路径可以被统计优化，语义路径需要主体裁定。意义与可学性的反比，是余项守恒在AI架构领域的具体形式。