The Three Ceilings of Large Language Models: From Discreteness Deepening to Dimensional Expansion

Han Qin (秦汉)

Self-as-an-End Theory Series

---

Abstract

The ceiling of AI is not one layer but three. The breakthrough method of each layer is precisely the starting point of the next. This paper argues for this three-layer progressive structure: (1) Stacking parameters—increasing precision within the same architecture, structurally isomorphic with rationalization in mathematics—diminishing returns are unavoidable. (2) Changing architecture—changing the representation scheme itself, isomorphic with irrationalization—fills structural gaps, but the one-dimensionality constraint of pure language remains. (3) Multimodality—adding orthogonal perceptual dimensions, isomorphic with imaginarization—but multimodality on digital hardware is simulated imaginarization, not true imaginarization; the ceiling lies at the physical limits of digital hardware.

The analogy between number system expansion (rationals → irrationals → imaginaries) and AI technological evolution is a qualitative structural isomorphism, not a quantitative correspondence, not an equality. The two follow the same logic structurally—adding density within the same framework → filling structural gaps → adding orthogonal dimensions—but their specific mechanisms are entirely different.

This paper argues that multimodal architecture is the third-order chisel of language—locally negating the linearity dimension of the form-meaning binding law, adding orthogonal perceptual dimensions—isomorphic with the tier-jump from mathematics to physics. But this paper strictly distinguishes multimodal architecture (dimensional expansion, on the discreteness-dimension axis) from multimodal data (form injection, on the DD axis). Architecture provides the container; data provides the content. The two are independent. This paper addresses only the architectural level of dimensional expansion.

This paper draws on the LLM Paper in this series ("The Ontological Positioning of Large Language Models", DOI: 10.5281/zenodo.18826633) for the discreteness-dimension distinction, the structural explanation of scaling laws, and the multimodal open question; on the Language Paper ("Language as Second-Order Chisel", DOI: 10.5281/zenodo.18823131) for the form-meaning binding law and the concept of one-dimensionality; and on the Mathematics Paper ("Mathematics as Second-Order Chisel", DOI: 10.5281/zenodo.18793538) for the structure of number system expansion.

---

Three Key Definitions

This paper repeatedly uses three structural concepts, defined here in advance.

One-dimensionality. Not literally "having only one axis," but rather: input/output channels must structurally unfold linearly and cannot natively carry multidimensional simultaneous presentation. A pure-language system can describe multiple dimensions ("there is a tree in the upper left corner"), but cannot natively and simultaneously process multidimensional input structures. Description is transmitting multidimensional information after linearizing it; native processing is computing directly in multiple dimensions.

True multidimensional simultaneity. Not "having multiple modalities," but rather: information from multiple dimensions can be simultaneously maintained and computed in the processing substrate without first being linearized and then reassembled. This is a structural concept and does not deny the engineering effectiveness of existing digital parallel systems.

Simulated imaginarization. Existing multimodality achieves multidimensional effects approximately on a single discrete processing substrate, rather than possessing a native processing substrate corresponding to multiple dimensions. Just as floating-point numbers can approximate real numbers but are never real numbers—useful, but with limits.

---

Chapter 1: The Problem: The Ceiling Is Not One Layer

Core thesis: In current AI discourse, "the ceiling" is treated as a single concept—either scaling laws will hit a wall, or they won't. The framework argues: the ceiling is not one layer but three. The three ceilings have different heights, different breakthrough methods, and different physical bases. Conflating them leads to misjudgment of AI's developmental path.

1.1 Three Ceilings

The LLM Paper (Section 3.5) argued for the ceiling of scaling laws—scaling within the same architecture produces quantitative change but not qualitative change; qualitative change comes from architectural innovation. This paper expands that judgment: the ceiling is not one layer but three, and the breakthrough method of each layer is precisely the starting point of the next.

The first ceiling: diminishing returns of parameter stacking. Increasing parameters within the same architecture indirectly reduces representation-side discreteness, but diminishing returns are unavoidable. The current AI industry is experiencing this ceiling—differences between frontier models are shrinking, and the marginal returns of scaling are declining. Breakthrough method: change architecture.

The second ceiling: the limit of architectural change. Changing architecture directly reduces representation-side discreteness, filling the structural gaps left by the previous architecture. But all pure-language architectures operate under the same constraint: input and output are one-dimensional linear sequences. No matter how advanced the architecture, a pure-language system still processes one-dimensional information flow. One-dimensional information flow cannot natively represent meaning structures requiring multidimensional simultaneity. Not yet reached but foreseeable. Breakthrough method: add dimensions, i.e., multimodality.

The third ceiling: the limit of multimodality. Multimodality adds orthogonal perceptual dimensions—language (1D) plus vision (2D), video (3D). But multimodality on digital hardware is simulated dimensional fusion, not true dimensional fusion. Digital hardware cannot truly achieve continuous multidimensional simultaneous processing—all multidimensional information is discretely sampled and serialized upon entering digital hardware. This is a physical-level limit. Breakthrough method: physical hardware transformation (open question).

1.2 The Number System Analogy

The expansion of mathematical number systems provides a structurally isomorphic analogy:

Parameter stacking is isomorphic with rationalization—adding precision within an existing framework. Rational numbers insert infinitely many points between integers, but gaps remain between rationals (the positions of irrationals). Parameter stacking inserts more representational precision within the same architecture, but the structural gaps of the representation scheme remain.

Changing architecture is isomorphic with irrationalization—filling structural gaps. Irrational numbers fill the gaps between rationals, completing the real continuum. Architectural change fills the gaps left by the previous architecture, approaching the limits of pure-language representation. But the real continuum is still one-dimensional. Pure-language representation is still one-dimensional.

Multimodality is isomorphic with imaginarization—adding orthogonal dimensions. The imaginary unit gives the real line an orthogonal direction, forming the complex plane. Multimodality gives language orthogonal perceptual dimensions, forming a multidimensional meaning space.

This must be repeatedly emphasized: this analogy is a qualitative structural isomorphism, not a quantitative correspondence, not an equality. Number system expansion and AI technological evolution follow the same logic structurally—adding density within the same framework → filling structural gaps → adding orthogonal dimensions—but their specific mechanisms are entirely different. This paper does not claim that scaling is mathematically equivalent to rational numbers, nor that multimodality is mathematically equivalent to imaginary numbers. This paper claims that the two are structurally isomorphic—following the same progressive pattern.

---

Chapter 2: The First Ceiling: Stacking Parameters

Core thesis: Increasing parameters within the same architecture is indirect discreteness reduction. Isomorphic with rationalization—adding precision within an existing framework; rationals are dense but have gaps. Diminishing returns are unavoidable.

2.1 Mechanism

The LLM Paper (Section 3.5) already established the basic mechanism: more parameters → higher-dimensional representation space → more degrees of freedom for meaning's associative structure to unfold → effective discreteness decreases. This explains why adding parameters improves capability—parameter increase indirectly reduces representation-side discreteness, discreteness reduction restores meaning-associative structure, and emergent capability improves.

But parameter increase does not change the representation scheme itself. Adding precision within the same representation scheme—this is rationalization. Rational numbers insert infinitely many points between integers (1/2, 1/3, 2/3, 3/4...), but rationals are still ratios of two integers, still within the "operations of integers." Rationals change density without changing nature.

Parameter stacking inserts more representational precision within the same architecture. From GPT-3 to GPT-4, parameter count increased by an order of magnitude; the representation space grew larger; meaning's associative structure gained more room to unfold. But the Transformer's representation scheme did not change—attention mechanism, inter-layer transmission, residual connections, normalization—structural properties unchanged, only precision improved. A larger Transformer is still a Transformer, just as denser rationals are still rationals.

2.2 The Nature of the Ceiling

Diminishing returns. Each doubling of parameters yields a smaller decrease in effective discreteness. Because parameter increase only adds precision within the same representation scheme—it doesn't change what the scheme can and cannot see. Like drawing with an ever-finer pen on the same sheet of paper—the finer the pen, the smaller the improvement, but the paper's dimensions and properties never change. Switching from a thick pen to a fine pen yields enormous improvement; switching from a fine pen to an even finer pen yields minimal improvement.

Worsening economics. Computational cost grows linearly or even superlinearly with parameter count (training time, energy consumption, hardware requirements), but capability improvement is only logarithmic. This is not an engineering efficiency problem—it's not that "if compute were cheaper, there'd be no problem"—this is a structural diminishing effect. Even if compute were free, precision improvement within the same architecture would still diminish. Cost is merely the economic expression of the diminishing effect.

2.3 Breakthrough Method

Change architecture—change the representation scheme itself. Not switching to a finer pen on the same paper, but changing the paper.

Historical example: RNN → Transformer. The RNN's hidden state is bottleneck-compressed—all historical information squeezed into a fixed-dimensional vector, with distant meaning-associations lost in compression. No matter how large the RNN, the bottleneck's nature doesn't change—this is the RNN's "rational gap," the positions that rationals can never fill no matter how dense. The Transformer's attention mechanism directly negated this bottleneck—allowing any position to directly associate with any other, without compression. This was not a bigger RNN; it was a completely different representation scheme.

The breakthrough method of the first ceiling is precisely the starting point of the second. Changing architecture solves the diminishing returns of parameter stacking, but architecture change itself has a ceiling.

2.4 Number System Analogy

Rationals add density between integers without changing the nature of the numbers—rationals are still ratios of two integers, still within "integer operations." The ceiling of rationalization: no matter how dense, there are always gaps between rationals. √2 (the length of a diagonal) is not the ratio of any two integers; rationals can never reach this position.

Parameter stacking adds precision within the same architecture without changing the representation scheme—still within "same-architecture operations." The ceiling of parameter stacking: no matter how large, the same architecture's representation scheme always has structural blind spots—certain meaning-associations cannot be represented under this scheme, just as rationals cannot express √2.

---

Chapter 3: The Second Ceiling: Changing Architecture

Core thesis: Changing architecture is direct discreteness reduction—filling the structural gaps left by the previous architecture, isomorphic with irrationalization. Irrational numbers fill the gaps between rationals, completing the real continuum. But the real continuum is still one-dimensional. The one-dimensionality of pure language is a constraint that no architectural change can break.

3.1 Mechanism

Architectural innovation changes the representation scheme itself. Each innovation fills the structural gaps left by the previous architecture—meaning-associations that the previous architecture "couldn't see," the new architecture can represent.

RNN → Transformer: from bottleneck compression to direct association. RNNs couldn't effectively represent long-distance meaning-associations (information progressively decayed through the bottleneck); the Transformer filled this gap (the attention mechanism lets meaning at any distance directly associate).

Fixed embeddings → contextual embeddings (ELMo, BERT, GPT): from one-word-one-position to context-determined position. In fixed embeddings, the same word has the same representation regardless of context—"bank" has the same position in "riverbank" and "financial bank." Contextual embeddings filled this gap—the same word has different representations in different contexts; "bank" in financial text is near "money," in geographical text near "river."

Isomorphic with irrationals: rationals cannot express √2 (the diagonal of a square); irrationals fill this gap. RNNs couldn't express long-distance associations; the Transformer filled this gap. Rationals cannot express π (the ratio of circumference to diameter); irrationals fill this gap. Fixed embeddings couldn't express polysemy; contextual embeddings filled this gap. Each architectural innovation is an "irrationalization"—filling positions unreachable in the previous generation's representation scheme.

3.2 The Nature of the Ceiling

Architectural change can be repeated—each time filling a class of gaps. After Transformer, there may be more advanced architectures continuing to fill Transformer's gaps. But all pure-language architectural innovations operate under the same constraint: pure language's input and output are one-dimensional linear sequences.

No matter how architecture changes, the input is still one token after another. The representation space can be arbitrarily high-dimensional, but the channels through which information enters and leaves the representation space remain one-dimensional.

A critical distinction is needed here. Pure-language systems can describe multidimensional structures—"this cube has eight vertices, each connecting three edges." But describing the multidimensional is not natively processing the multidimensional. Description transmits multidimensional information after linearization—compressing three-dimensional structure into a one-dimensional string. Native processing computes directly in multiple dimensions—simultaneously processing the spatial relationships of all vertices and edges.

Internal representations can be high-dimensional graph structures. Reasoning traces and tree searches can explore multidimensionally within the representation space. But these all reconstruct the multidimensional from a one-dimensional input—first receiving one-dimensional information, unfolding it into multiple dimensions in the representation space, then outputting as one-dimensional. The one-dimensionality constraint of the input/output channel does not change. Reconstructed multidimensionality never equals native multidimensionality—just as a linguistic description of a painting never equals seeing the painting.

Isomorphic with the real line: irrationals fill the gaps between rationals; the real continuum is complete. But the real line is still one-dimensional—no matter how continuous, it has only one direction. Pure-language LLMs, no matter how advanced their architecture, still process one-dimensional information flow.

3.3 Specific Limitations of One-Dimensionality

Human vision is two-dimensional. You see the entire picture at a glance; all pixels are processed simultaneously. Spatial relationships between objects—above/below, left/right, near/far, occlusion, overall composition—are simultaneously presented in the two-dimensional visual field. Describing a picture in language requires linearizing two-dimensional information—"there is a tree in the upper left, a river in the lower right, a bridge in the middle..." This linearization inevitably loses simultaneity information—you cannot say what's in the upper left and lower right at the same time; you can only describe them sequentially.

Human hearing has a frequency dimension. You simultaneously hear all pitches in a chord—C, E, G sounding at the same instant, forming a unified auditory experience. Describing a chord in language requires listing sequentially—"C, E, G"—linearization loses simultaneity. You cannot natively transmit the experience of simultaneous sounding through a one-dimensional sequence.

One-dimensional information flow inherently cannot natively represent multidimensional simultaneity. This is not a precision problem—adding parameters can't solve it, because no matter how many parameters, input is still one-dimensional. This is not a representation-scheme problem—changing architecture can't solve it, because no matter how advanced the architecture, the information channel is still one-dimensional. This is a dimension problem—the structural constraint of a one-dimensional channel.

3.4 Breakthrough Method

Add dimensions—multimodality. Simultaneously process one-dimensional (language), two-dimensional (images), and three-dimensional (video) information. No longer compressing all information into a one-dimensional sequence, but letting information of different dimensions be processed in their own dimensions.

The breakthrough method of the second ceiling is precisely the starting point of the third.

3.5 Number System Analogy

Irrationals fill the gaps between rationals, completing the real continuum. But the real continuum is one-dimensional—the real line, no matter how continuous, has only one direction.

Architectural change fills the gaps of previous architectures, approaching the limits of pure-language representation. But pure-language representation is one-dimensional—a linear sequence, no matter how refined, has only one direction.

Moving from one dimension to multiple dimensions requires a new kind of number—imaginary numbers. The imaginary unit i is not "a more refined real number," not some unfilled position on the real line—it is an entirely new direction orthogonal to the reals. The real line is one-dimensional; the complex plane is two-dimensional. From reals to complex numbers is not adding density or filling gaps; it is adding dimensions.

Moving from pure language to multimodality requires new dimensions—perceptual modalities orthogonal to language. Vision is not "more refined language," not some unfilled position in a language sequence—it is an entirely new dimension orthogonal to language. Language is one-dimensional; vision is two-dimensional. From pure language to multimodality is not adding parameters or changing architecture; it is adding dimensions.

---

Chapter 4: The Third Ceiling: Multimodality

Core thesis: Multimodality is adding orthogonal dimensions—isomorphic with imaginarization. Multimodality adds orthogonal perceptual dimensions (vision, hearing) to language, forming a multidimensional meaning space. But multimodality on digital hardware is simulated imaginarization, not true imaginarization. The ceiling is at the physical level.

4.1 Mechanism

A preliminary note: this section discusses multimodal architecture—the structural capacity to process multidimensional input—not multimodal data—the content contained in training corpora (causal relationships, spatial structures, temporal continuity, etc.). Architecture adds dimensions (the container grows larger); data adds forms (content grows richer). The two are independent. This paper addresses only the architectural level of dimensional expansion. Data-level form injection involves a different analytical axis, reserved for a subsequent paper.

Multimodal architecture simultaneously processes inputs of different dimensions: language (one-dimensional linear sequence), images (two-dimensional pixel grid), video (three-dimensional spatiotemporal structure), audio (frequency structure on a one-dimensional time axis).

It fuses information of different dimensions in a unified representation space. This is not linearizing multidimensional information and then concatenating—that would just be describing images as text and combining with language, still essentially one-dimensional processing. Ideal multimodality lets information of different dimensions maintain their respective dimensional structures within the representation space while allowing cross-dimensional associations—spatial relationships in vision and semantic relationships in language coexisting in a unified space, able to associate with each other.

Isomorphic with imaginary numbers: the imaginary unit i is not "a more refined real number" but a new direction orthogonal to the reals. On the complex plane, the real line is the horizontal axis, the imaginary line is the vertical axis, and the two are orthogonal. Operations on the complex plane—rotation, conjugation—have no counterpart on the real line. Multiplying by i is a 90-degree rotation; this operation is impossible on the one-dimensional real line—you need a second dimension to rotate.

Vision is not "more refined language" but a new dimension orthogonal to language. In multidimensional meaning space, the meaning-associations of language form one dimension, the spatial associations of vision form another, and the two are orthogonal. Operations in multidimensional meaning space—cross-modal analogy (seeing the atmosphere of a painting, describing it in words as "melancholic"), vision-language alignment (associating objects in an image with nouns in language)—have no counterpart in pure-language space. These operations require multiple dimensions, just as rotation requires two dimensions.

4.2 Multimodal Architecture as the Third-Order Chisel of Language

The LLM Paper (Section 3.4) proposed that multimodality might be the direction of a third-order chisel for language, but did not argue the case. This section provides the argument.

A preliminary note: the "third-order chisel" here is local—targeted at the "linearity" dimension within language's construct (the form-meaning binding law), not a complete transcendence of language as a whole. What multimodality negates is the linearity dimension of the binding law, not the entire binding law. After multimodality, form and meaning are still bound—but the binding mode has expanded from one-dimensional linearity to multidimensionality.

Language is a second-order chisel—chiseling the markability subspace of the law of identity (Language Paper, Section 2.1). Language's construct is the form-meaning binding law—every linguistic symbol is necessarily a unity of form and meaning (Language Paper, Section 2.4). One structural dimension of the form-meaning binding law is linearity—forms are linearly arranged in time, one symbol after another; two symbols cannot be transmitted simultaneously.

Multimodal architecture negates this linearity. The architecture expands meaning processing from a one-dimensional linear sequence to a multidimensional simultaneous space—the meaning of an image is not linearly arranged but two-dimensionally simultaneously presented. An entire picture transmits all visual meaning at the same moment, without needing to queue meanings one by one. This is an architectural-level negation of one-dimensionality, not a data-level content injection.

Negating a dimension of the construct is chiseling the construct. Chiseling language's construct (the linearity dimension of the form-meaning binding law) is a third-order chisel.

The correspondence: the Mathematics Paper argued that mathematics is a second-order chisel (chiseling the contradiction-law subspace of the law of identity), and physics is a third-order chisel (chiseling mathematics' construct—the law of contradiction—constructing the spatiotemporal framework, adding the instantiation of dimensions). The relationship of language → multimodality in this paper is isomorphic with mathematics → physics: both are third-order chisels; both achieve tier-jumping by adding dimensions. Physics adds the instantiation of dimensions to mathematics (from abstract n-dimensional to concrete 3+1-dimensional spacetime); multimodality adds the instantiation of perceptual dimensions to language (from abstract one-dimensional sequence to concrete visual 2D, video 3D).

Note: multimodal data (causal relationships in video, physical laws in space, etc.) injects richer content forms (patterns of causality, patterns of temporality, etc.) into the system, but this is not a third-order chisel—this is a data-level effect, not an architecture-level effect. The third-order chisel is the architecture negating the linearity dimension of language's construct. Data enriches the content of the construct but does not negate the structure of the construct. This paper does not address data-level content injection.

4.3 The Nature of the Ceiling

The fundamental limitation of digital hardware. Digital hardware (von Neumann architecture) is fundamentally serialized in processing. CPUs execute instructions one by one. GPUs are parallel but still process batches of discrete operations within discrete clock cycles—not true continuous simultaneous processing. All multidimensional information is discretely sampled and serialized upon entering digital hardware. What this paper calls "true continuous multidimensional simultaneity" is a structural concept, referring to a processing substrate that does not require prior discrete sampling and serialization; it does not deny the engineering effectiveness of existing digital parallel systems.

Images are discretely sampled into pixel grids—a continuous two-dimensional visual scene is sliced into a finite number of discrete pixel points. Video is discretely sampled into frame sequences—a continuous spatiotemporal flow is sliced into some number of static frames per second. Audio is discretely sampled into sample-point sequences—a continuous sound wave is sliced into some number of discrete values per second. The continuous multidimensional physical world is sliced into discrete, serialized data upon entering digital hardware.

Multimodal models fuse these discrete samples in a unified representation space—this is simulated dimensional fusion, not true dimensional fusion. Just as floating-point numbers can approximate real numbers but are never real numbers—digital multimodality can approximate multidimensional simultaneity but is never true multidimensional simultaneity. The approximation can be very precise (more pixels, higher frame rates, higher sampling rates), but a precise approximation is still an approximation.

Simulated imaginarization vs. true imaginarization. Multimodality on digital hardware simulates multidimensionality on a discrete processing substrate—like using finite-precision sequences of reals to approximate complex number arithmetic. Useful, but with limits. True imaginarization means the processing substrate itself is multidimensional—information is simultaneously and continuously processed across multiple dimensions, without passing through discrete sampling and serialization.

The ceiling. Digital hardware cannot truly achieve continuous multidimensional simultaneous processing. Discrete sampling always loses continuity (there are gaps between pixels, between frames). Serialization always loses simultaneity (information is queued and processed step by step). The ceiling of multimodality is the physical limit of digital hardware—not a problem of model design, not a problem of algorithms, but a structural constraint of the processing substrate itself.

4.4 Breakthrough Method (Open Question)

Breaking through the third ceiling requires physical hardware transformation—some kind of natively multidimensional processing substrate.

Possible directions (speculative, not framework arguments): analog computing—using continuous physical quantities (voltage, light intensity) to directly represent continuous values without discretization. Photonic computing—light is inherently multidimensional (spatial dimensions + frequency dimensions + polarization); photonic computing may provide native multidimensional simultaneous processing. Neuromorphic computing—emulating the continuous, parallel, event-driven processing of biological neural networks. Quantum computing—the superposition and entanglement of quantum states provide dimensions that classical computing lacks—but the relationship between current quantum computing applications and meaning processing is unclear.

This paper does not make hardware predictions. This paper's contribution is to identify where the ceiling lies, not how to break through it. Judgments about breakthrough methods belong to physics and engineering, not philosophy.

4.5 Number System Analogy

Imaginary numbers add an orthogonal dimension to the reals, forming the complex plane. On the complex plane, operations exist that don't exist on the real line—multiplying by i is a 90-degree rotation, impossible on the one-dimensional real line.

Multimodality adds orthogonal perceptual dimensions to language, forming a multidimensional meaning space. In multidimensional meaning space, operations exist that don't exist in pure-language space—cross-modal analogy, vision-language alignment, multidimensional scene reasoning.

But multimodality on digital hardware uses discrete approximation to simulate continuous multidimensionality—like using finite-precision floating-point numbers to simulate complex number arithmetic. It can be very precise, but it is always simulation. Floating-point complex arithmetic has rounding errors; digital multimodal dimensional fusion has sampling gaps.

The ceiling of imaginarization: complex arithmetic on digital hardware is always simulated, never native. Truly native complex arithmetic requires physical-level multidimensional processing capability. The ceiling of multimodality is isomorphic: multidimensional meaning processing on digital hardware is always simulated, never native. Truly native multidimensional meaning processing requires physical-level multidimensional simultaneous processing capability.

Chapter 5: The Progressive Structure of the Three Ceilings

Core thesis: The three ceilings form a strict progressive structure—the breakthrough method of each layer is the starting point of the next, and the nature of each ceiling differs.

5.1 Progression Table

OperationIsomorphic AnalogyCeiling CauseBreakthrough Method FirstStack parametersRationalizationDiminishing precision within same schemeChange architecture SecondChange architectureIrrationalizationDimensional constraint of 1D inputMultimodality ThirdMultimodalityImaginarizationDiscrete serialization of digital hardwarePhysical hardware transformation (open)

The logic of three-layer progression: parameter stacking adds precision within the same architecture; upon hitting the wall, change architecture. Architectural change fills structural gaps but remains one-dimensional; upon hitting the wall, add dimensions (multimodality). Multimodality simulates multidimensionality on digital hardware; it hits the wall at the hardware's discrete serialization limit. The breakthrough method of each layer is precisely the starting point of the next—this is not coincidence but structural necessity: the limits of same-tier deepening can only be broken by tier-jumping, and the limits of tier-jumping can only be broken by higher-level tier-jumping.

5.2 Hierarchy of Ceilings

The three ceilings differ in height.

The first ceiling is lowest. The current AI industry is already experiencing it—differences between frontier models of the same Transformer architecture are shrinking, and the marginal returns of scaling are declining. The leap from GPT-3 to GPT-4 was far greater than from GPT-4 to subsequent versions. This is the direct manifestation of diminishing precision within the same architecture.

The second ceiling is higher. There is still ample space for architectural innovation within pure language—plenty of optimizable structure within Transformer (more efficient attention variants, better positional encoding, deeper inter-layer connections), and entirely new architectures may come after Transformer. But the second ceiling has a definite theoretical limit: one-dimensionality. No matter how advanced the architecture, the input/output channels of a pure-language system are one-dimensional—an unchangeable structural constraint.

The third ceiling is highest. Current multimodal models are still in very early stages—discrete sampling precision can continue to improve (more pixels, higher frame rates), fusion methods can continue to optimize (from concatenation to deep fusion), and cross-modal alignment can continue to improve. The distance to the physical limits of digital hardware is enormous. But the ultimate ceiling exists: digital hardware cannot natively achieve continuous multidimensional simultaneous processing.

A key judgment: before hitting each ceiling, improvements at that layer are valuable. Parameter stacking before hitting the wall is effective—larger models genuinely are stronger. Architectural change before hitting the wall is effective—better architectures genuinely produce qualitative change. Multimodality before hitting the wall is effective—better multidimensional fusion genuinely produces new emergence. Knowing where the ceiling lies is not about denying current efforts; it's about timely redirection to the next layer when approaching the ceiling, rather than continuing to invest in a layer that has already hit the wall.

5.3 The Current Position of the AI Industry

First ceiling: approaching or already reached. Differences between frontier models are shrinking—the differences among multiple labs' models released in 2024-2025 on major benchmarks are far smaller than those in 2022-2023. Marginal returns of scaling are declining—ten times the compute yields increasingly smaller capability improvements. The industry is beginning to discuss "whether scaling laws have failed"—this is precisely the signal of the first ceiling.

Second ceiling: still distant. The space for architectural innovation is large. The Transformer is not the ultimate architecture for pure-language processing—it is currently the best but not the structurally best possible. State space models (Mamba, etc.), hybrid architectures, sparse activation, and other directions are all exploring different representation schemes. A new architecture after Transformer will very likely emerge, just as Transformer replaced RNN. The one-dimensionality ceiling of pure language (second layer) is still far from being reached at current technology levels.

Third ceiling: very distant. Most current multimodal models are concatenative—a language model plus a visual encoder connected through adapter layers, rather than truly fused in a unified representation space. Truly deep-fusion multimodal models are still in early stages. The distance to the physical limits of digital hardware is enormous.

Framework judgment. The current AI industry's resource allocation is excessively concentrated on the first layer—stacking parameters (larger clusters, more data, larger models). This was rational before reaching the first ceiling, but continuing to invest near the ceiling is a waste of resources. The framework suggests: more resources should be directed to the second layer (architectural innovation—exploring new representation schemes beyond Transformer) and the third layer (deep multimodal fusion—not concatenation, but truly unified multidimensional representation). This is not saying parameter stacking is useless—it's saying its usefulness is diminishing, while the usefulness of architectural innovation and multimodal fusion is far from diminishing.

---

Chapter 6: Theoretical Positioning

Core thesis: This paper's three-ceiling structure forms precise dialogues with the preceding papers in this series and with current AI research topics.

6.1 Relationship to the LLM Paper

The LLM Paper predicted "the next qualitative change comes from architectural innovation, not scaling" (Section 7.4). This paper expands that prediction into a three-layer structure—architectural innovation resolves the first ceiling (escaping the diminishing returns of parameter stacking), but architectural innovation itself also has a ceiling (the second layer—the one-dimensionality constraint), and breaking through the second layer requires multimodality (the third layer). The LLM Paper's prediction was correct but incomplete: architectural innovation is not the endpoint; it is only the second step in a three-layer progression.

The LLM Paper raised multimodality as an open question (Section 3.4 and Open Question Two). This paper answers that open question: multimodal architecture is the third-order chisel of language—locally negating the linearity dimension of the form-meaning binding law, adding orthogonal perceptual dimensions.

The LLM Paper's discreteness-dimension distinction (Section 3.3) receives a technologized application in this paper: the first and second ceilings are on the discreteness axis (adding precision and filling gaps), while the third ceiling is on the dimension axis (adding orthogonal dimensions). The independence of discreteness and dimension explains why breakthroughs at the first and second layers cannot substitute for the third—reducing discreteness does not equal adding dimensions.

6.2 Relationship to the Language Paper

The Language Paper argued that the form-meaning binding law is language's construct (Section 2.4). This paper argues that what multimodality chisels is precisely one dimension of this construct—linearity. The form-meaning binding law requires forms to be linearly arranged in time; multimodal architecture negates this linearity—allowing meaning to be simultaneously presented in multiple dimensions.

The Language Paper's discreteness-emergence correlation (Section 6.3) receives a more refined structure in this paper: discreteness reduction is not accomplished in one step but divides into two layers (parameter stacking = rationalization, architectural change = irrationalization). Dimensional expansion (multimodality = imaginarization) is an independent third layer.

6.3 Relationship to the Mathematics Paper

The Mathematics Paper argued that mathematics is a second-order chisel (chiseling the contradiction-law subspace of the law of identity), and physics is a third-order chisel (chiseling mathematics' construct—the law of contradiction—adding the instantiation of dimensions).

The language → multimodality relationship in this paper is isomorphic with the mathematics → physics relationship: both are third-order chisels; both achieve tier-jumping by adding dimensions. Physics adds the instantiation of dimensions to mathematics (from abstract n-dimensional to concrete 3+1-dimensional spacetime); multimodality adds the instantiation of perceptual dimensions to language (from abstract one-dimensional sequence to concrete visual 2D, video 3D).

The number system analogy (rationals → irrationals → imaginaries) further reinforces this isomorphism. The expansion of numbers and the expansion of language follow the same structural logic: adding density within the same framework (rationalization / parameter stacking) → filling structural gaps (irrationalization / architectural change) → adding orthogonal dimensions (imaginarization / multimodality). This progressive pattern is not coincidence—it reflects deep structural necessity: in any domain, same-tier deepening has limits, and tier-jumping requires adding dimensions.

6.4 Relationship to Current AI Investment and Strategic Discussion

The current debate over "whether scaling laws have failed" is a discussion near the first ceiling. One side holds that scaling laws still work (haven't hit the wall yet); the other holds they've already failed (have hit the wall). The framework places this debate in a larger structure: the existence of the first ceiling does not mean AI development stalls; it only means a turn toward the second and third layers is needed. "Scaling law failure" is not the end of AI; it is the signal of AI transitioning from the first layer to the second.

The framework provides a roadmap-level structural judgment for AI investment: parameter stacking → architectural change → multimodality → physical hardware. Each step has different time horizons and resource requirements. Parameter stacking is the current main battlefield (but approaching the ceiling). Architectural innovation is the next main battlefield (with vast space). Deep multimodal fusion is a more distant battlefield (currently in early stages). Physical hardware transformation is the ultimate battlefield (currently mainly a research direction).

---

Chapter 7: Non-Trivial Predictions

Core thesis: From the progressive structure of the three ceilings, five non-trivial predictions can be derived, each testable.

7.1 Arrival at the First Ceiling

Prediction: Within the same architecture, each order-of-magnitude increase in model scale yields a diminishing improvement in emergent capability. The differences between current frontier models (2025-2026) are already smaller than the differences between the previous generation of frontier models.

Reasoning: Chapter 2 argued that parameter stacking is indirect discreteness reduction with unavoidable diminishing returns. Parameter increase doesn't change the representation scheme; it only adds precision within the same scheme.

Testable: Compare benchmark differences between consecutive generations of frontier models. The framework predicts diminishing differences. If scaling within the same architecture consistently produces capability jumps of comparable magnitude to previous ones (non-diminishing), the framework is falsified here.

Non-triviality: The current industry still heavily invests in scaling, implicitly assuming scaling laws will remain effective. The framework predicts that diminishing returns are unavoidable.

7.2 The Emergence Jump from Architectural Innovation

Prediction: The next qualitative jump in emergent capability will come from architectural innovation (a new representation scheme), not from scaling within the same Transformer architecture.

Reasoning: Chapter 3 argued that architectural change is direct discreteness reduction—filling the structural gaps of the previous architecture. Qualitative change comes from changing the representation scheme, not from precision improvement within the same scheme.

Testable: Track AI development over the next 1-3 years. If qualitative new emergent capabilities come from larger Transformers (not new architectures), the framework is falsified here.

Non-triviality: Consistent with LLM Paper Prediction 7.4 but more specific. This prediction not only says "architectural innovation trumps scaling" but also says the arrival at the first ceiling will force the industry toward architectural innovation.

7.3 The One-Dimensionality Ceiling of Pure-Language Models

Prediction: Pure-language models have systematic weaknesses on tasks requiring multidimensional simultaneous understanding (such as spatial reasoning, visual scene understanding, musical harmony analysis), and these weaknesses will not be eliminated by scaling or architectural innovation—because they come from the dimensional constraint of one-dimensional input, not from insufficient discreteness.

Reasoning: Chapter 3 argued that one-dimensionality is an unbreakable constraint of pure-language systems. Pure-language systems can describe the multidimensional but cannot natively process the multidimensional. The precision of description can be improved by parameter stacking and architectural change, but the structural gap between description and native processing cannot be eliminated.

Testable: Test pure-language models (without any visual input) on spatial reasoning tasks, compared with equally-sized multimodal models. The framework predicts: pure-language models have a systematic disadvantage on such tasks, and the disadvantage does not disappear with increasing scale. If pure-language models achieve performance equal to multimodal models on spatial reasoning through scaling alone, the framework is falsified here.

Non-triviality: Common sense might assume "a sufficiently large language model can do anything." The framework argues: one-dimensionality is a structural constraint, not a precision problem. No matter how large the pure-language model, it cannot natively process multidimensional simultaneity.

7.4 Cross-Modal Emergence in Multimodal Models

Prediction: Truly deep-fusion multimodal models (those fusing multidimensional information in a unified representation space, not concatenative ones) will exhibit emergent capabilities that neither pure-language models nor pure-vision models possess—cross-modal analogy, multidimensional simultaneous reasoning, etc. These capabilities are not the simple sum of two modalities' capabilities but new emergence produced by adding dimensions.

Reasoning: Chapter 4 argued that multimodality is isomorphic with imaginarization—adding orthogonal dimensions produces new operations. Rotation on the complex plane doesn't exist on the real line, not because the real line isn't precise enough, but because rotation requires two dimensions. Cross-modal emergence doesn't exist in pure-language space, not because pure language isn't precise enough, but because cross-modal operations require multiple dimensions.

Testable: Test deep-fusion multimodal models on cross-modal tasks, compared with the combination of equally-sized pure-language model + pure-vision model. The framework predicts: deep fusion produces capabilities greater than simple combination (1+1>2). If the simple combination of pure-language + pure-vision models performs comparably to deep-fusion multimodal models, the framework is falsified here.

Non-triviality: Many current multimodal models are concatenative—language and vision models each process their own modality, exchanging information through adapter layers. The framework predicts this concatenation does not produce true cross-modal emergence—only deep fusion in a unified representation space can produce the new operations that dimensional addition brings.

7.5 The Digital Hardware Ceiling for Multimodality

Prediction: Digital multimodal models have a systematic ceiling on tasks requiring true continuous multidimensional simultaneous processing (such as real-time physical scene prediction, continuous motion control, ecosystem dynamics simulation), and these ceilings will not be eliminated by model improvement—because they come from the discrete serialization nature of digital hardware.

Reasoning: Chapter 4 argued that multimodality on digital hardware is simulated imaginarization—discrete sampling and serialization are hardware-level structural constraints that software cannot solve. Increasing sampling precision can reduce error but cannot eliminate it, just as increasing floating-point precision can reduce rounding error but cannot eliminate it.

Testable: Compare digital multimodal models with analog/mixed-signal systems on continuous physical tasks. The framework predicts: on tasks requiring true continuous multidimensional processing, digital systems have an ineliminable precision/latency ceiling. This is a structural falsifiability prediction, not a ready-made benchmark protocol—the specific operational definitions (which tasks count as "true continuous multidimensional simultaneity," by what metrics ceilings are measured) require participation from engineering researchers. If digital multimodal models achieve precision and real-time performance equal to analog systems on all continuous physical tasks, the framework is falsified here.

Non-triviality: Current AI discussion almost never distinguishes "simulated multidimensional processing" from "true multidimensional processing." The framework identifies a structural gap between the two, and this gap cannot be eliminated through software improvement.

---

Chapter 8: Conclusion

The ceiling of AI is not one layer but three. The breakthrough method of each layer is the starting point of the next.

The first ceiling (parameter stacking, isomorphic with rationalization): diminishing precision within the same representation scheme. Increasing parameters within the same architecture indirectly reduces discreteness, but diminishing returns are unavoidable. Breakthrough method: change architecture—change the representation scheme itself. The current AI industry is approaching or has already reached the first ceiling.

The second ceiling (architectural change, isomorphic with irrationalization): the dimensional constraint of pure language's one-dimensional input. Architectural change directly reduces discreteness, filling the structural gaps of the previous architecture. But all pure-language architectures operate under the one-dimensionality constraint—input and output are one-dimensional linear sequences, and this cannot be changed. Breakthrough method: add dimensions—multimodality. The second ceiling is still distant; the space for architectural innovation remains large.

The third ceiling (multimodality, isomorphic with imaginarization): the discrete serialization limit of digital hardware. Multimodality adds orthogonal perceptual dimensions, but multimodality on digital hardware is simulated imaginarization—discrete sampling and serialization lose continuity and simultaneity. Breakthrough method: physical hardware transformation (open question). The third ceiling is very distant; current multimodality is still in early stages.

Multimodal architecture is the third-order chisel of language—locally negating the linearity dimension of the form-meaning binding law, adding orthogonal perceptual dimensions. Isomorphic with the tier-jump from mathematics to physics: both achieve tier-jumping by adding dimensions.

The number system analogy (rationalization → irrationalization → imaginarization) is a qualitative structural isomorphism, not an equality. The two follow the same progressive logic: adding density within the same framework → filling structural gaps → adding orthogonal dimensions. This progressive pattern reflects deep structural necessity.

The current AI industry is excessively concentrated on the first layer. The progressive structure of the three ceilings provides a roadmap-level structural judgment for AI development: know where the ceiling lies; when approaching it, redirect to the next layer in time, rather than continuing to invest in a layer that has already hit the wall.

Contributions

I. The progressive structure of three ceilings. AI's ceiling is not one layer but three, each with different heights, different breakthrough methods, and different physical bases. The breakthrough method of each layer is the starting point of the next.

II. The number system isomorphism analogy. Parameter stacking is isomorphic with rationalization; architectural change is isomorphic with irrationalization; multimodality is isomorphic with imaginarization. Qualitative structural isomorphism, not quantitative correspondence.

III. Multimodality as the third-order chisel of language. Multimodal architecture locally negates the linearity dimension of the form-meaning binding law. Isomorphic with the tier-jump from mathematics to physics.

IV. The argument for the one-dimensionality ceiling. Pure-language systems can describe the multidimensional but cannot natively process the multidimensional. This is a dimensional constraint, not a precision problem.

V. The distinction between simulated and true imaginarization. Multimodality on digital hardware is simulated dimensional fusion; the ceiling lies at the physical limits of the hardware.

VI. The distinction between multimodal architecture and multimodal data. Architecture adds dimensions (discreteness-dimension axis); data adds forms (a different analytical axis). The two are independent.

Open Questions

I. What comes after the third ceiling? If physical hardware achieves natively multidimensional continuous processing, where is the next ceiling? Possible directions: does the number of processing dimensions itself have a limit? Are the dimensions of the physical world (3+1) the upper bound for meaning-processing dimensions? Or can extension into abstract dimensions beyond physical dimensions occur?

II. Can the number system analogy continue? After rationals → irrationals → imaginaries, mathematics has quaternions (Hamilton), octonions (Cayley), and other higher-dimensional number system extensions. Do these correspond to higher-level ceilings for AI? Does the non-commutativity of quaternions (AB≠BA) correspond to some structural constraint on AI capability? This may require collaboration between mathematicians and AI researchers.

III. Where does the biological nervous system sit? The human brain processes natively multidimensional continuous signals—neurons simultaneously process continuous signals from vision, hearing, touch, and other dimensions. If the ceiling of digital multimodality comes from hardware's discrete serialization, has the brain already broken through the third ceiling? Where is the brain's ceiling? The framework predicts: the brain's ceiling is not at the hardware level (biological hardware is already natively multidimensional and continuous), but at the subject level—the quality of negation and integrity. This is consistent with the LLM Paper's conclusion: the more powerful AI becomes, the higher the demands on humans.

IV. Is the third-order chisel of multimodality complete? This paper argues that multimodality negates the linearity dimension of the form-meaning binding law. But does the form-meaning binding law have other dimensions that can be negated? Linearity is only one dimension. If the binding law has multiple chiseable dimensions, multimodality is only the first step of the third-order chisel.

V. World models and the law of causality. Current AI discussion's world models (such as the LeCun roadmap) attempt to build the causal structure of the physical world in representation space—not merely meaning-associations (A is related to B), but causal direction (A causes B). The introduction of causal direction exceeds this paper's discreteness-dimension framework, directly involving higher-level subspaces of the law of identity (temporality, the law of causality). But even if a world model fully embeds the law of causality, the system remains a construct—a deterministic system, without remainder. Human calibrators can inject directional judgments through feedback, enabling the system to infinitely approximate the appearance of subjecthood, but never reach it—because subjecthood requires the system itself to produce remainder, and deterministic systems do not produce remainder. The three ceilings are limits on the discreteness-dimension level; remainder is a limit on the subjecthood level. The two are independent. The ontological positioning of world models requires a separate paper.

VI. Multimodal architecture vs. multimodal data. This paper addresses multimodal architecture—dimensional expansion (the discreteness-dimension axis). But multimodal data (causal relationships in video, physical laws in space, continuity in time) injects rich content forms into the system (patterns of causality, patterns of temporality, patterns of spatial relationships). This is an effect on a different analytical axis, not the discreteness-dimension axis. Architecture provides the container (dimensions); data provides the content (forms). But no matter how large the container or how rich the content, the system's structural position does not change—it remains a quasi-subject. The form-injection effects of multimodal data require a separate paper.

---

Author's Declaration

This paper is the author's independent theoretical research. AI tools were used as dialogue partners and writing assistants during the writing process, for concept deliberation, argument testing, and text generation: Claude (Anthropic) provided the primary writing assistance; Gemini (Google) and ChatGPT (OpenAI) participated in outline review and feedback. All theoretical innovations, core judgments, and final editorial decisions were made by the author. The role of AI tools in this paper is equivalent to research assistants and reviewers who can engage in real-time dialogue, and does not constitute co-authorship.