Compute Wall or 90x Headroom? The AGI Debate and Why Agents Can Write GPU Kernels
Two researchers take opposite positions on whether AGI has arrived: one argues that computation is fundamentally physical and that exponential growth will eventually collide with hard limits, while the other argues that compute is far from exhausted and that a roughly 90x efficiency gain is already being realized. This article lays out both arguments, then turns to the engineering takeaways—where compute actually gets wasted, how to measure and optimize utilization layer by layer, and why GPU kernel optimization is a natural fit for AI agents. It includes a closed-loop optimization pattern, correctness and benchmarking practices, safety boundaries, and a practical checklist for turning compute headroom into business results.
1. Where the Debate Begins: Two Articles, One Clash
The question of whether artificial general intelligence has already arrived never really stops, and the latest round of debate started with two articles that reach opposite conclusions.
One researcher focuses on deep learning quantization—model compression, or how to keep a model usable at lower precision and with less compute. His piece takes AGI down off its pedestal with a blunt verdict:
If an AI can solve the IMO but cannot solve any real-world problem, it is not general intelligence.
The other researcher works across both academia and engineering, and his day job is GPU kernel development and compute optimization. His conclusion is nearly the reverse: AGI will become reality, and we may have already achieved it.
The disagreement later moved onto a podcast, and neither side was a bystander. One studies how to run models on less compute; the other studies how to squeeze every last bit of performance out of each GPU. That is exactly why the conversation quickly shifted from the philosophical question of “what is AGI” to a far more concrete one: how much room is left for compute to grow?
2. View A: Compute Is Physical, and Exponential Growth Will Hit a Ceiling
The first researcher’s case against AGI arriving easily rests on a premise that is easy to overlook: compute is physical. Every algorithm ultimately lands on transistors on a chip, data movement across memory, and the power and cooling inside a rack.
2.1 The memory wall: the bottleneck is moving data, not computing
On modern accelerators, floating-point throughput has grown far faster than memory capacity and bandwidth. Processors often spend more time “waiting for data” than “doing math.” The larger the model, the heavier the footprint of intermediate state—weights, activations, KV cache—on memory capacity and bandwidth, and the memory pressure during inference is especially pronounced.
Data movement is also one of the most power-hungry parts of a system. In many accelerator architectures, accessing off-chip memory consumes far more energy than completing a single floating-point multiply-add. In other words, energy consumption itself is putting a ceiling on compute.
2.2 The von Neumann bottleneck: a structural constraint
Separating compute units from memory is a defining trait of the von Neumann architecture. A “memory-compute gap” always sits between the two, and techniques like on-chip caches and multi-level pipelines are fundamentally about easing that gap, not eliminating it. This structural constraint does not disappear just because one generation of hardware doubles in performance—and that is the technical foundation of the “physical limit” argument.
2.3 Power, heat, and data: three real-world constraints
- Power and cooling: Single-chip power draw keeps climbing, and rack power density is approaching the limits of traditional data center power and cooling design. The expansion of compute clusters is increasingly constrained by power supply, cooling capacity, and site conditions—not just chip supply.
- High-quality data: The supply of usable high-quality text on the internet is gradually being consumed, and the marginal return of scaling up pretraining data keeps falling—the so-called “data wall.”
- Diminishing returns: Simply scaling up model and data size shows clear diminishing returns in capability gains on some tasks.
From this angle, the researcher cites a frequently repeated idea: almost all exponential growth eventually collides with resource and physical limits. Surging inference demand and enormous investment in chips and power do keep compute spending climbing in the short term, but the rate of capability improvement will eventually slow—and scaling laws are no exception.
Notably, this view is highly consistent with his field. When scaling compute becomes expensive and its marginal returns diminish, techniques that “express a model with less compute”—quantization, sparsification, distillation—naturally become the key path around the bottleneck.
3. View B: Compute Is Far From Squeezed Dry, and a 90x Dividend Is Being Realized
The second researcher’s rebuttal is equally grounded. He works with GPU kernels and compute utilization every day, so he thinks it is premature to declare that compute has peaked: “We haven’t even truly harnessed the previous generation of hardware yet.”
3.1 Where compute actually gets wasted
In real systems, compute is often badly underestimated, with large amounts of performance lost in invisible engineering details:
- Memory-bound work: Many non-compute-intensive operators are limited by memory bandwidth—or even by fixed overhead on tiny data volumes—so compute units spend much of their time waiting.
- Kernel launch and scheduling overhead: Frequently launched small kernels accumulate meaningful launch cost, and poor grid and thread-block partitioning leaves compute units underutilized.
- Pipeline and parallelism efficiency: Bubbles form in the pipeline between compute, communication, and data loading, while synchronization waits and load imbalance drag utilization down further.
- Distributed communication overhead: Under strategies like tensor, pipeline, and data parallelism, a communication ratio that is too high—or compute that cannot overlap with communication—means new compute is spent mostly on waiting.
- Long-tail failures: As clusters grow, slow nodes hold back overall progress, and failure recovery, checkpointing, and recomputation add further losses.
These losses share one trait: they never show up in a chip’s peak compute, only in whether real workloads finish on time.
3.2 Evaluation lag: the strongest model is not the hardware ceiling
The “strongest models” being benchmarked and used today were often trained on clusters from a year or two ago. Hardware iterates quickly, but toolchains, operator libraries, and parallel strategies take time to mature, and unlocking a new generation of hardware usually requires the ecosystem to adapt gradually.
As a result, using today’s model performance to measure the ceiling of current hardware can systematically underestimate the usable compute available.
3.3 The multiplication logic behind the 90x estimate
The researcher uses an intuitive multiplication estimate to show where compute headroom comes from:
| Dimension | Gain | Level |
|---|---|---|
| New-generation hardware | ~2–3x | Raw compute at the chip and system level |
| System and engineering optimization | ~3x | Turning existing compute into effective compute |
| Larger-scale clusters | ~10x | Total compute |
| Combined | ~90x theoretically | Available compute |
The 90x figure is not a paper calculation—it is real-world headroom being gradually realized across the industry. The most significant item is the middle one: the gains from system and engineering optimization often require no new hardware purchases, yet can raise the utilization of existing compute several times over. This is precisely the value of GPU kernel optimization, operator fusion, and scheduling improvements.
That said, this estimate is best seen as a kind of “order-of-magnitude thinking” rather than a precise forecast. Realizing 90x depends on clearing a series of bottlenecks in turn—memory, energy, communication, scheduling. If any one link fails to keep up, the short-board effect will sharply discount the overall gain.
3.4 Physical constraints and engineering dividends are not opposites
Put the two views side by side and they actually point to the same optimization spectrum:
- Use less: quantization, sparsification, and distillation reduce the compute required per task;
- Use it fuller: operator optimization, parallel strategies, and scheduling improvements raise hardware utilization;
- Buy more: chip iteration, cluster expansion, and architectural innovation raise total compute.
The three stack on top of each other rather than replacing one another. This is exactly why the “slowdown” of scaling laws is more likely to be phased: once a bottleneck is eased by engineering, a new one emerges—and what breaks through it is, again, new algorithms and engineering methods.
4. Different Paths, Same Destination: You Can Debate the Definition, but Changing How We Work Is the Answer
On the definition of AGI, the industry broadly holds two mainstream views:
- Cognitive-capability view: whether a model can cover enough cognitive tasks—for example, matching or surpassing human performance on standardized tests;
- Economic-value view: whether the technology genuinely changes how things are produced, whether it can replace or significantly augment human labor.
Interestingly, as the debate progressed, the two researchers converged on one point: what AGI is called does not matter—what matters is whether it has changed the way we work. This is a reminder to practitioners that rather than getting hung up on the label, it is better to build a pragmatic way of observing progress:
- Replace a single leaderboard with real tasks: pick tasks that come up frequently in your team’s day-to-day—requirement analysis, code review, data analysis—and continuously track completion quality and the level of human intervention required.
- Measure output per unit cost: factor in call cost, inference latency, and retry overhead on failures to assess “effective output per unit cost,” rather than judging only the quality of a single response.
- Track the human-AI collaboration ratio: count the share of tasks where a person shifts from “executor” to “reviewer”—a signal that AI is genuinely embedded in the workflow.
- Continuously accumulate domain data: turn inputs, outputs, and feedback from business scenarios into a reusable evaluation set, so the evaluation evolves as the business does.
5. From Debate to Practice: Why Agents Can Write GPU Kernels
This debate has a落 easily overlooked landing point: if the compute dividend comes mainly from “using existing compute more fully,” then GPU kernel optimization is the key engineering link for realizing that dividend—and it happens to be one of the most vivid recent demonstrations of what agents can do.
5.1 Why this task suits an agent
Kernel optimization is a highly structured “generate–verify–tune” loop:
- The objective is clear: throughput or latency of a single operator;
- Automated criteria exist: correctness tests cover “is it right,” and performance benchmarks cover “is it fast”;
- A mature toolchain is available: compilers, benchmarking tools, and performance profilers.
Unlike open-ended creative work, a bug in kernel code is caught immediately by tests, and slow code is quantified right away by benchmarks. This measurable, feedback-rich nature lets an agent run many iterations on its own, rather than generating once and leaving a human to patch things repeatedly.
5.2 The loop skeleton: illustrative pseudocode
The code below only illustrates the loop structure. It depends on no specific framework, and all function names and parameters are illustrative:
import statisticsfrom dataclasses import dataclass@dataclassclass Candidate:code: strscore: float = float("inf") # Worst latency (ms) across all test scenariosdef optimize(reference, agent, scenarios, max_rounds=20):best = Nonehistory = []for _ in range(max_rounds):# 1. Generate: the agent uses the reference implementation and recent profiling data# to produce a new kernel implementationcand = Candidate(agent.propose(reference, history[-3:]))# 2. Verify: reject the candidate if it fails correctness in any scenarioif not all(verify(cand.code, reference, s) for s in scenarios):history.append({"status": "verify_failed", "code": cand.code})continue# 3. Tune: measure performance across all scenarios and take the worst latency as the scorecand.score = max(benchmark(cand.code, s) for s in scenarios)profile = profile_kernel(cand.code, scenarios[0]) # Profiling, e.g. arithmetic intensity, bandwidth utilizationhistory.append({"status": "ok", "score": cand.score, "profile": profile})if best is None or cand.score < best.score:best = candreturn best
5.3 Correctness first: “right” before “fast”
The first principle of kernel optimization is correctness first—any speedup rests on correct results:
- Tolerance settings: floating-point addition is not associative, so changing tiling and accumulation order introduces numerical error; set sensible relative and absolute tolerances by precision type.
- Input coverage: beyond typical sizes, cover edge cases like unaligned addresses, non-power-of-two dimensions, and very small and very large shapes.
- Regression testing: a replacement kernel should go into the automated test pipeline, re-running every case on each change to prevent optimizations from introducing hidden errors.
import mathdef verify(candidate_code, reference, scenario, rtol=1e-2, atol=1e-3):"""Correctness check: aligns memory layout, then compares element by element; returns False if any scenario fails"""x = scenario.to_device()expected = reference(x)actual = candidate_kernel(candidate_code)(x)if expected.shape != actual.shape or expected.dtype != actual.dtype:return Falsereturn all(math.isclose(float(a), float(b), rel_tol=rtol, abs_tol=atol)for a, b in zip(actual.flatten(), expected.flatten()))
5.4 Performance: benchmarks and profiling are both essential
- Environment isolation: run in a dedicated compute environment to avoid contending for resources with other tasks.
- Warm-up and repetition: discard the first several runs, then measure multiple times and take the median to cancel out first-call initialization overhead.
- Locating the bottleneck: combine indicators like arithmetic intensity, memory bandwidth utilization, and compute-unit occupancy to identify the bottleneck type. For example, a low-arithmetic-intensity elementwise operator will struggle to break the bandwidth ceiling no matter how you rewrite the compute logic—at that point, consider operator fusion to cut data reads and writes.
- Guarding against overfitting: give the agent both profiling data and failure reasons, so it does not just “game the score” on a single test shape.
5.5 Safety boundaries: run the agent in a controlled environment
Kernel code is essentially a program that operates directly on hardware, so isolation and safeguards must be in place before it runs:
- Sandbox isolation: execute inside a dedicated container that loads no extra runtime libraries; prefer portable formats for build artifacts, so you can still fall back if the inference environment lacks an adapter library.
- Resource limits: cap CPU, memory, and execution time, and enforce a timeout.
- Least privilege: block access to the network and sensitive host resources.
- Human review: kernels with significant performance gains still need human review before shipping, with focus on numerical stability, edge cases, and maintainability.
5.6 Real-world boundaries: what agents are and aren’t good at
An agent’s value in kernel optimization lies mainly in automating the “try–verify–improve” process. It can quickly sift through a large set of candidate implementations for a better one and hand the repetitive tuning work to the machine.
But the core design decisions still depend on human experts: how to partition compute and data, how to organize on-chip caches and pipelines, how to trade off parallelism against register pressure. At the current stage, the more realistic framing is this: an agent is an “accelerator” for the kernel engineer, not a replacement.
6. Engineering Checklist: Turning the Compute Dividend into Business Results
For enterprises and R&D teams, the most direct lesson from this debate is: before discussing “whether to buy more compute,” first figure out “how much of our existing compute are we actually using.”
6.1 Measure first
Build a monitoring and alerting system for metrics like compute utilization, per-operator time distribution, memory footprint, and task queue time, and feed the key metrics into a unified logging and monitoring platform. For cloud resources, you can pair monitoring and alerting to configure threshold notifications, so idle resources don’t go unnoticed.
6.2 Optimize by bottleneck, layer by layer
| Layer | Typical approaches | Applicable signal |
|---|---|---|
| Operator layer | Operator fusion, mixed precision and low-precision quantization, efficient kernel replacement | One operator dominates total time; profiling shows memory-bound work or high launch overhead |
| Model layer | Structured pruning, sparsification, distillation, inference caching | Memory footprint or raw compute volume is itself the bottleneck |
| Parallelism and scheduling layer | Overlapping compute and communication, dynamic batching, sensible parallel partitioning | High communication ratio; compute and communication cannot overlap |
| Infrastructure layer | Elastic scaling, tiered task queuing, fast failure recovery | Low resource utilization, long queue times, high recomputation loss from failures |
6.3 Use general cloud capabilities to lower the cost of adoption
- Use containerized elastic GPU compute to handle peaks and troughs in training and inference;
- Use object storage to centrally manage models, datasets, and evaluation baselines;
- Use a message queue to decouple the generate, verify, and tune stages of agent tasks into an asynchronous pipeline;
- Use logging and monitoring/alerting to continuously track performance and cost, forming a “measure–optimize–measure again” loop.
6.4 Connect evaluation with production
Offline evaluation sets should keep pulling in real production traffic and be regression-tested regularly. A model leading on a standard leaderboard does not mean it leads on your own business data.
7. Conclusion: The Answer to the Debate Is Written in Every Run’s Efficiency
Of the two AGI articles, one says it “will not become reality,” and the other says it “will become reality, and may already have.” Their disagreement looks like a debate over definitions, but at its core it is a judgment about compute growth: whether it hits a physical limit first, or still has tens of times of headroom waiting to be realized.
For developers, there is no need to rush to pick a side. The more valuable move is to focus on the fact both sides point to: compute is both constrained by the laws of physics and far from being fully used. From choosing the right operators and making good use of each generation of hardware, to letting agents take part in kernel iteration and optimization, every gain in run efficiency is compute that “grows” without adding a single watt of power. And when a system can reliably do more and more real tasks on a person’s behalf, whether or not we choose to call it AGI, it is already changing the way we work.
