What is the Next Scaling Law and What is the right hardware to run it
Pretraining taught models about the world. Reinforcement learning taught them to solve tasks. Test time compute taught them to think longer. What will the next scaling law be?
The easiest way to understand the current limitation of AI is to imagine hiring the smartest employee in the world, letting them work inside your company for a week, and then erasing nearly everything they learned before Monday morning.
During that week, the employee figured out how your organization actually works. They learned which documents are outdated, which people need to approve a purchase, why the last deployment failed, which customer constraints are real, and which internal rules are routinely ignored. They made mistakes, received corrections, watched the consequences of their decisions, and became materially better at the job.
Then the week ends.
You keep a transcript of what happened. You may even write a summary for the next version of the employee. But the employee’s underlying ability has not changed. The next session begins with roughly the same general intelligence it had before, plus a larger pile of notes to read.
That is approximately how the most advanced AI systems operate today!
Most general-purpose foundation model products remain weight-static during a customer’s deployment, even when the surrounding application preserves conversations, retrieved documents, or user memory.
They can hold more information in context. They can retrieve old conversations. They can store memories in databases. They can reason for longer before answering. But most of the capability developed while completing a real task disappears when the session ends. The model may remember that a supplier delivered late. It does not necessarily develop the deeper operational instinct that would make it structure the next procurement process differently.
A larger context window can produce powerful temporary adaptation. But a larger notebook is not the same as consolidating a skill that persists after the notebook is closed.
This is the problem the next scaling law must solve.
AI has already moved through three scaling regimes
The first scaling regime was pretraining.
Give a model more parameters, more data, and more compute, and it builds a better compressed representation of the world. The model learns language, code, mathematics, history, science, and the statistical structure connecting them. This produced the original foundation-model curve.
The second regime was post-training, increasingly dominated by reinforcement learning across verifiable environments.
Instead of merely predicting internet text, the model attempts tasks, receives feedback, and is trained toward outcomes. More environments, more rollouts, stronger verifiers, and more post-training compute produce agents that can reason, use tools, write code, and recover from mistakes.
The third regime was test-time compute.
The model is given more compute after receiving the problem. It can generate longer chains of reasoning, search across candidate solutions, use tools, run code, inspect its own work, and spend more tokens before committing to an answer. The model’s weights do not improve, but the quality of the individual answer can improve as more compute is spent on it.
Each regime opened a new axis along which the industry could convert additional compute into capability:
Pretraining scaled what the model knew.
Post-training scaled what the model could do.
Test-time compute scaled how hard it could think about one problem.
The missing regime is the one humans rely on most:
Learning from experience after entering the real world.
Dwarkesh Patel who I recently met frames this as continual learning and has a series of blogposts that are worth reading, this piece builds on that framework and follows the argument into the economics and physical infrastructure!
The Next Scaling Law Is Closed-Loop Experience
That distinction between verifiability and grindability matters because reinforcement learning works best where reality can be copied.
A coding repository can be duplicated 10,000 times. Agents can attempt the same task in parallel, run the tests, compare results, and reset the environment when they fail. Chess boards, math problems, games, and simulated robotics environments have the same property.
Most economically valuable work does not.
A company cannot rerun the same acquisition 10,000 times. A hospital cannot reset a patient. A data center operator cannot repeatedly commission the same facility under slightly different electrical failures. A sales team cannot rewind a customer negotiation. A manufacturer cannot fork its supply chain into thousands of parallel worlds and keep only the best result.
These environments may still produce observable outcomes, but they are not easily grindable. The experience arrives slowly, messily, and often only once.
This is exactly where deployment learning becomes valuable.
Continual learning describes the capability we want: a model that gets better through experience. But it does not yet define the scaling variable. It does not tell an investor what grows, a compute buyer what must be purchased, or a data center operator what workload must be supported.
The missing unit is not another token, user, or agent.
It is a completed learning loop.
We will call the underlying relationship the Deployment Scaling Law:
AI capability will scale with the number of valuable real-world episodes a system can convert into verified, durable improvements in future behavior.
One completed loop looks like this:
Observe the environment → take an action → receive an outcome → determine what caused it → rehearse alternatives → produce a candidate update → verify that update → return to production measurably better.
This is not yet an empirically fitted power law like the original parameter–data–compute relationships. It is a proposed engineering scaling relationship: a resource that can be increased systematically and converted into capability.
A more useful operating metric is validated learning throughput:
How much retained improvement a system produces per day, per dollar, or per megawatt-hour after passing retention, safety, and production tests.
That throughput rises when the system encounters more outcome-bearing episodes, extracts clearer learning signals, amplifies scarce events through replay and simulation, and shortens the time from experience to a safe production update. It falls when updates cause forgetting, poisoning, or reward hacking.
The next scaling race is therefore not simply about who can generate the most actions. It is about who can close the most high-quality learning loops and retain the resulting improvement.
New real-world events are scarce. Most iteration volume will come from replaying historical episodes and generating synthetic counterfactuals around them. Real events ground the curriculum; historical regression and simulation provide the volume.
A system with millions of interactions but no observable outcomes does not necessarily learn much. A system with rich outcomes but poor credit assignment learns the wrong lesson. A system that produces good candidate updates but takes six months to test and deploy them compounds too slowly. A system that learns rapidly but destroys old skills is not improving; it is moving capability around.
The next scaling race is therefore not simply about who can generate the most actions. It is about who can close the most high-quality, validated learning loops.
NVIDIA now describes agentic scaling as a fourth scaling law: models take more steps, invoke more tools, execute more code, and interact with more environments. That is directionally important, but it is still primarily a demand law. More actions create more inference, CPU, networking, and context-memory demand. They do not automatically create a more capable model. (NVIDIA Developer)
A million agents that forget everything are a larger inference bill, not a new learning law.
The new law begins when what happened during deployment changes what the system can do next time.
The Most Valuable AI Data Begins After the Answer
The AI industry still treats the prompt and response as the important data but they are not necessarily the full picture.
Suppose a model recommends a particular server configuration. The prompt tells us what the customer asked for. The response tells us what the model recommended.
Neither tells us whether the recommendation was correct.
The learning signal arrives later:
Did the system satisfy the required throughput?
Did it remain within the customer’s power envelope?
Was memory exhausted before compute?
Did the network become the bottleneck?
Did one component delay deployment by three months?
What did the customer replace?
Did the facility support the actual rack density?
Did the system produce the economics originally modeled?
The prompt and answer describe an intention. The downstream environment reveals causality.
That means the economically valuable training record is not:
Question → answer
It is:
Intent → available information → assumptions → action → environmental response → human intervention → downstream outcome
A chatbot transcript tells the model what someone said.
An outcome bearing episode tells the model what happened.
This is why the next scaling law cannot be measured in conversations alone. A billion ordinary conversations may contain less valuable learning signal than one million consequential tasks with clear interventions and observable results.
The highest-value deployment environments will therefore share several characteristics:
This changes the hierarchy of AI data.
Public text was abundant, centrally accessible, and weakly connected to outcomes. Enterprise deployment data is scarce, private, causal, and tightly connected to economic value.
The pretraining era rewarded whoever could collect the largest representation of what humanity had written.
The deployment era will reward whoever can observe what happened next.
Continual Learning Moves the Moat
Dwarkesh’s flywheel is plausible. The best model attracts the most users. Those users generate more experience. The experience improves the model, which attracts even more usage and makes the system harder to replace.
But deployment does not automatically become learning.
For the flywheel to close, four things must be true:
The system must see the outcome.
A model API may see the prompt and response without knowing whether the recommendation made money, caused an outage, passed review, or was ignored. The most valuable signal often appears later, inside the customer’s operational systems.The system must attribute the outcome.
A failed deployment does not reveal what the model should learn. The cause could be the architecture, an unstated requirement, a supplier delay, a firmware issue, or the facility itself. Outcomes only become useful when they are connected to the state, action, intervention, and assumptions that produced them.The provider must have permission to reuse the lesson.
Leading enterprise AI services generally do not train shared models on business or API data by default. That does not settle ownership of every future memory, adapter, or model branch. It does establish that cross-tenant learning is permissioned rather than automatic.The system must be able to consolidate the lesson safely.
An enterprise may own the workflow and outcome history without having the adaptation algorithms, regression suites, replay environments, compute, and rollback systems needed to convert that experience into a better model.
The key distinction is:
Prompt access is not outcome access. Outcome access is not learning permission. Learning permission is not learning capability.
This means neither the lab nor the enterprise automatically owns the moat.
The lab controls the general model, adaptation methods, model internals, and large-scale training infrastructure.
The enterprise controls the workflow, systems of record, human corrections, downstream outcomes, and authority over how its experience is reused.
The application layer may sit between them and see the richest version of the episode: the model’s action, the human correction, the operational write-back, and the final result. That makes coding platforms, enterprise workflow systems, scientific tools, and other deeply integrated applications potential kingmakers.
The outcome depends on who closes the loop.
If a lab controls the model, application, outcome visibility, learning rights, and update pipeline, continual learning can make its moat much stronger.
If the enterprise owns its evaluations, private adaptations, outcome history, and portable learned state, the switching costs may accrue to the enterprise instead.
If the workflow platform owns the outcome graph and can route among models, foundation models may become interchangeable intelligence suppliers beneath it.
The defensible conclusion is not that continual learning makes laboratories invincible, or that it destroys their advantage.
Continual learning moves part of the moat from the model checkpoint to control of the learning loop.
The scarce asset is not deployment data in the abstract. It is permissioned, attributable experience connected to machinery that can convert it into retained improvement.
That will change AI contracts. The next generation of agreements will increasingly govern not only what data a model may read, but what it may remember, adapt from, generalize across customers, and export when the customer leaves.
The Learning-Rights Stack
Data access and learning rights are not the same thing.
An enterprise can permit a model to process information without permitting the provider to retain the interaction. It can allow memory for one tenant without allowing cross-tenant training. It can authorize a private adapter while prohibiting the resulting lesson from entering the shared foundation model.
The rights stack will increasingly look like this:
This leads to several predictions.
First, the same model will eventually carry different prices depending on the rights granted. A zero-retention customer should pay more than a customer willing to let the laboratory learn across interactions. A customer granting cross-tenant generalization rights is supplying a valuable training input.
The next cloud discount will be a data rights rebate.
Second, the strongest laboratory moats will form where the laboratory also owns the application or environment. A lab embedded deeply inside a coding system, browser, consumer product, scientific platform, or robotic environment can observe richer actions and outcomes than a lab providing a stateless API.
Third, enterprise software companies with deep access to workflows and downstream state may become more strategically important than generic AI wrappers. They own the environment in which the useful learning signal is created.
Continual learning therefore does not eliminate laboratory moats. It fragments them.
The future moat has at least three components:
The intelligence prior: the general model and learning algorithm.
The experience environment: the workflow in which actions and outcomes occur.
The learning rights: permission to retain, adapt, merge, and redistribute the lesson.
The leading lab may control all three in some markets. The enterprise will control the second and third in others.
Deployment data may become one of the first major ongoing model improvement inputs generated inside private customer workflows and will be negotiated at the customer boundary.
The Open-Source Model Does Not Need to Win on Day One
This changes how we should think about open models.
Today, open and closed models are compared at the moment of deployment. The model with the better benchmark score appears to be the better choice.
Continual learning turns deployment into the starting line.
For many operational workloads, a slightly weaker open model with a mature proprietary branch could eventually outperform a stronger frontier model that arrives with no company-specific experience.
The architecture would look like:
Shared base model + domain branch + company branch + user state
The shared base supplies general reasoning. The private branch accumulates the company’s skills, policies, evaluators, and workflow knowledge. User-specific state remains smaller and more local.
The model does not need a completely separate full checkpoint for every customer. Systems such as S-LoRA and Punica have already demonstrated that many low-rank adapters can be served efficiently against a shared base model while preserving batching opportunities. S-LoRA’s experiments served thousands of adapters against one base and reported material throughput improvements over the baselines tested.
These systems demonstrate efficient multi-adapter serving. They do not yet solve branch portability across base models, continual consolidation, or safe model merging. (S-LoRA) (Punica)
The long-term open-source value proposition is therefore not just:
“You can download the weights.”
It is:
“You can own the branch, move the branch, and keep everything the system learned while working for you.”
This introduces a deeper form of portability.
Exporting an enterprise’s documents is no longer enough. A company must eventually be able to export:
memories;
adapters;
skills;
workflow policies;
evaluators;
simulation environments;
outcome histories;
update lineage;
and the complete learned branch.
A provider that allows customers to export their documents while trapping the learned state has recreated cloud lock-in at a more valuable layer.
The next enterprise AI battle will be fought over the portability of learned capability, not merely the portability of data.
A Self Learning Model Will Not Finetune Its Full Weights After Every Task
The naive version of continual learning is a model that completes a task and immediately updates itself.
That would be almost impossible to operate safely.
One malicious interaction could poison the model. Two customers could generate contradictory updates. Production replicas would drift apart. Outputs would become difficult to reproduce. Batching would deteriorate. Regulators and customers could not determine which version made a decision. A bad update might not reveal itself until it had already influenced thousands of later actions.
The real architecture will look less like one continuously mutating neural network and more like a hierarchy of state operating at different timescales.
The model’s central decision is therefore not merely how to learn.
It is where the lesson belongs.
A new episode may require:
no change at all;
a factual memory;
a retrieval update;
a new tool;
a different workflow;
an improved evaluator;
a tenant-specific adapter;
a domain expert;
or, rarely, a change to the shared base model.
The system needs an adaptation router:
New experience → ignore, context, memory, tool, evaluator, skill, adapter, expert, or base model
Early research
is beginning to explore pieces of this problem. SEAL allows a model to generate its own fine-tuning data and update directives. SCoL trains a model to select sparse internal update locations and reports better acquisition and retention than several prompting and fine-tuning baselines in its experiments. Google’s Nested Learning treats a model as interconnected optimization processes operating at different update frequencies. These are early research results, not production-ready solutions, but they point toward selective, multi-timescale plasticity rather than constant full-model retraining.
Check related work
SEAL, arXiv 2506.10943, generates self-edits, fine-tuning data, and update directives.
SCoL, arXiv 2605.07076, generates sparse update-location instructions and reports stronger acquisition and retention than its tested baselines.
Nested Learning, from Google Research, treats models as nested optimization problems operating at different timescales.
Palantir got the data model right before the learning algorithm was ready
Palantir ($PLTR) is important here, although not necessarily because it has solved continual neural learning.
Its Ontology is designed to represent enterprise decisions, actions, permissions, and outcomes not simply documents or database rows. Palantir explicitly describes pairing a decision with later observations of its result so the information can support future decisions or be used to retrain and finetune models. (Palantir)
That is the correct substrate.
A vector database can retrieve an old purchase order. It does not inherently know:
why the purchase was made;
which alternatives were rejected;
what assumptions were used;
whether the supplier delivered;
what the employee changed;
or whether the final result was successful.
A prompt log captures the interaction while an outcome linked record adds the state changes, interventions, and downstream results needed for credit assignment and counterfactual testing.
The first useful continual-learning systems will therefore look like self-learning systems before they look like continuously self-modifying foundation models. They will improve through structured memories, actions, evaluators, workflow changes, private adapters, and replay environments while weight-level learning improves more slowly underneath.
The Verification Tax
The largest new compute workload may not be learning itself.
It may be proving that the system learned the correct thing.
Every durable update introduces several questions:
Did the new behavior improve the target task?
Did it damage an older capability?
Does it survive a slightly different environment?
Did the model learn the real lesson or exploit a proxy?
Can an attacker trigger the new behavior?
Does the update still work across stochastic runs?
Should it be promoted globally, privately, or not at all?
Each candidate update therefore creates a shadow workload:
Replay → regression tests → adversarial tests → stochastic repetitions → hardware-specific tests → shadow deployment → canary monitoring
This is the Verification Tax.
The tax scales with update frequency, but it also scales with accumulated capability. As a system becomes more useful, the surface area it cannot afford to damage becomes larger.
The smarter the deployed system becomes, the more old behavior every new lesson must prove it did not break.
Current infrastructure models generally separate training and inference. Continual learning creates a third category:
Production compute: serving users and taking actions.
Learning compute: producing memories, adapters, candidate updates, and simulations.
Assurance compute: checking, attacking, comparing, and validating those updates.
Call the last two categories shadow compute.
A 1,000-agent sensitivity model
Consider an illustrative enterprise deployment. These are assumptions, not a forecast; the purpose is to establish where the break-even points sit.
The production workload is:
1,000 agents × 20 episodes × 50,000
= 1.0 billion production token-equivalents per day
The validation workload is:
2 candidate updates × 10,000 regression cases × 4 runs × 25,000
= 2.0 billion validation token-equivalents per day
The counterfactual replay workload is:
2,000 selected episodes × 5 branches × 50,000
= 500 million replay token-equivalents per day
Under these assumptions, 71.4% of model-side compute happens behind the answer.
This is not a claim that every agent fleet will produce this exact ratio. “Token-equivalent” also hides meaningful differences between prefill, decode, simulation, CPU execution, and backward passes. The model excludes storage writes, data preparation, failed experiments, and the CPU environments in which the agents act.
It is a break-even demonstration.
With two candidate updates per day, four stochastic executions, and 25,000 token-equivalents per regression case, only 5,000 regression cases per update are required for validation compute to equal the entire production workload.
A 1,000-agent fleet completing 20 substantive episodes per day produces 20,000 production episodes per day.
If two candidate updates are each tested against 10,000 regression cases with four stochastic runs, that produces 80,000 regression executions per day.
That is four regression executions per production episode. If the average regression execution costs 25% as much as a production episode, assurance compute equals production serving. At 50%, assurance compute is twice production serving.
NVIDIA’s new Vera CPU rack is designed to support more than 22,500 concurrent reinforcement-learning or agent sandboxes, and NVIDIA explicitly describes CPU execution, tool calls, environment interaction, and evaluation as part of the agentic loop. That shows that hardware vendors already expect environments and validation to become first-class infrastructure categories. (NVIDIA Developer)
This is the first large infrastructure implication of the Deployment Scaling Law.
The market expects agents to increase inference demand. That is correct but incomplete. A deployed system that learns also generates replay, simulation, adaptation, evaluation, and storage demand.
AI’s largest workload may eventually be AI checking AI.
For investors, this means token growth alone may understate infrastructure growth.
For data center operators, it means the future workload will not be a uniform hall of identical GPU racks.
For compute buyers, it means the true cost of an agent is not merely the cost of serving it. It is the cost of safely accumulating capability over time.
This material is for informational purposes only and does not constitute investment advice. Product specifications and performance figures attributed to vendors are company-reported and may not be directly comparable across architectures. Calculations identified as illustrative are sensitivity analyses rather than forecasts. FPX marketplace observations are anonymized, directional, configuration-dependent, and not a formal price index.
FREE SECTION ENDS HERE
Below: why the right computer is a heterogeneous learning plant, why NVIDIA and AMD+Cerebras are already splitting the workload, why certain types of used hardware will get a second life, and where the real component alpha sits across memory, SSDs, networking, power, cooling, and colocation.
What Hardware Runs the Next Scaling Law?
The simple answer is more compute.
The useful answer is that the workload is breaking into several physically different markets.
Serving a response, executing a tool, preserving context, updating an adapter, simulating alternative outcomes, and running regression tests do not have the same arithmetic intensity, memory pattern, latency requirement, synchronization requirement, or tolerance for failure.
The right computer is therefore not one larger accelerator.
It is a learning plant.
The largest mistake would be to take the current training cluster and assume every part of the learning loop should run on the same machine.
NVIDIA Has Already Drawn the Learning Machine
An open secret in the indusrty is that Ilya Sutskever’s SSI is working on continual learning and NVIDIA’s $5 billion investment is a sign that they want to see this next scaling law through into production. SSI Started using TPUs as we covered in a previous article and is now adding Vera Rubin racks to their fleet.
NVIDIA ($NVDA) says Vera Rubin POD combines seven chips and five purpose-built rack-scale systems. We have already analyzed this in depth in one of our pieces that talks abut the architecture and the various workloads it handles. Its public architecture includes Rubin GPU systems, Groq 3 LPX systems, Vera CPU racks, BlueField-4 STX context-memory storage, and Spectrum networking. NVIDIA positions the system for agentic AI rather than continual learning, but the physical division maps closely onto the phases of a future learning loop.
Rubin remains the flexible HBM-heavy engine. It handles large-context processing, prefill, attention, training, and workloads where general programmability matters.
Groq 3 LPX is optimized for fast and predictable token generation. NVIDIA’s design places prefill and decode attention on Rubin while routing latency-sensitive feed-forward and mixture-of-experts decode to the LPX system. The rack contains 256 LPUs, 128 GB of SRAM, and 40 PB/s of aggregate on-chip SRAM bandwidth. These are vendor-reported specifications and performance claims, but the architectural signal is more important than the benchmark: NVIDIA no longer believes one processor is the right answer for every inference phase.
Vera CPU racks handle code execution, tools, retrieval, data processing, and sandboxed environments. NVIDIA reports more than 22,500 concurrent agent or RL sandboxes per rack.
BlueField-4 STX provides a separate context-memory storage tier for persistent KV state and long-running agentic workloads. (NVIDIA Developer)
Spectrum networking moves state between the systems.
Our interpretation is:
Rubin thinks and adapts. LPX reacts. Vera acts. STX remembers. Spectrum coordinates.
NVIDIA does not describe the platform in those terms. But once a model begins acting, preserving state, replaying experience, and updating itself, those are the physical functions the system must perform.
FIGURE 4 — NVIDIA’S FIVE RACKS AS ONE LEARNING LOOP
Groq LPX
Low-latency decode
The GPU is no longer the AI computer. The loop is the computer.
The alpha is the changing bill of materials
The obvious NVIDIA thesis is more GPUs.
The more interesting thesis is that the non-GPU share of the AI factory expands.
A system with large numbers of autonomous agents needs CPU environments, context storage, host memory, DPUs, networking, and assurance capacity in ratios that ordinary training-cluster models do not capture.
The GPU remains the largest individual line item. But the incremental dollar may increasingly flow to the equipment that prevents the GPU from waiting:
CPU sandbox racks;
SOCAMM and server memory;
context-memory storage;
DPUs and SmartNICs;
high-endurance SSDs;
Ethernet switching;
liquid cooling;
and replay capacity.
This follows the same patt ern FPX has documented across earlier infrastructure cycles: once one bottleneck becomes obvious, value migrates toward the next layer keeping the expensive asset idle. Our prior memory and networking research argued that power determines where a facility can exist, while memory and data movement determine whether its accelerators do useful work. Continual learning broadens both bottlenecks. (FPX World)
AMD and Cerebras Confirm the Architecture Is Converging
NVIDIA is not alone in splitting the workload.
AMD ($AMD) and Cerebras ($CBRS) announced a joint disaggregated inference architecture in July 2026. AMD Helios handles prompts, large context windows, and high-throughput processing. Cerebras’ Wafer-Scale Engine handles low-latency decode and token generation. The companies expect the joint system to become available through Cerebras Cloud in the second half of 2026 and model up to five times higher tokens per second per watt than the stated baseline. That figure is vendor-modeled and should not be treated as an independent benchmark. (Advanced Micro Devices, Inc.)
AMD Helios itself connects 72 MI455X GPUs in one scale-up domain. AMD reports 432 GB of HBM4 per GPU, roughly 31 TB across the rack, and 260 TB/s of scale-up bandwidth. (AMD)
The AMD–Cerebras combination matters because it reaches the same architectural conclusion from a different direction:
Process the large parallel context on one machine. Generate latency-sensitive tokens on another.
Disaggregated inference is the precursor to disaggregated learning.
Once the full learning loop is considered, the system will likely disaggregate further:
prefill on one pool;
decode on another;
tools and environments on CPUs;
private adaptation on flexible accelerators;
replay on low-cost asynchronous fleets;
persistent state in a separate memory hierarchy;
consolidation on large training systems.
The strategic battle is therefore not only which chip is fastest.
It is which company controls the orchestration layer that decides where each phase runs and moves state between them without destroying latency or utilization.
NVIDIA’s advantage is that it owns more of the orchestration, networking, and software stack.
AMD’s opportunity is to make the heterogeneous system more open.
Cerebras’ opportunity is to become a high-value phase inside a larger workflow rather than needing to replace the GPU everywhere.
The ASIC Market Is Not One Bet
If sparse, frequent adaptation becomes a material share of AI compute, a hardware category may emerge between the general GPU and the forward-only inference ASIC.
The alternative-accelerator market is often treated as a single category: “AI chips that might compete with NVIDIA.”
That is the wrong frame.
These companies are placing different bets on where the learning loop becomes inefficient.
SambaNova’s SN50 uses a tiered memory architecture combining SRAM, HBM, and larger-capacity memory, with the ability to move between models rapidly. (SambaNova)
Etched says it is validating its first rack-scale frontier inference product using A0 silicon produced on TSMC N4P. Its demand and performance statements remain company-reported and pre-volume. (Etched)
d-Matrix places compute close to memory through its digital in-memory architecture and packages it in a standard PCIe form factor. Its published performance comparisons are vendor projections and workload-specific. (d-Matrix)
Furiosa’s RNGD uses 48 GB of HBM3, 256 MB of on-chip SRAM, and a stated 180-watt profile, making it structurally easier to place inside more conventional enterprise facilities than the densest rack-scale systems. (FuriosaAI)
The market should not ask which of these companies “beats the GPU.”
It should ask:
Which expensive phase of the learning loop can each architecture remove from the GPU?
The Missing Chip Is the Learning Accelerator
Most specialized inference accelerators optimize the forward pass.
Continual learning introduces selected backward passes, adapter updates, optimizer state, branch creation, evaluation, and rollback.
That creates a gap between the general-purpose GPU and the forward-only inference ASIC.
The future learning accelerator would include:
large HBM with substantial on-chip SRAM;
efficient forward and selected backward execution;
native sparse and low-rank updates;
resident adapter and optimizer state;
copy-on-write model branches;
atomic candidate updates;
rapid checksum and rollback;
strong tenant isolation;
cryptographically signed update lineage;
separate partitions for live serving and experimental learning.
The important metric would not be peak FLOPS.
It would not even be tokens per second.
It would be:
Verified durable capability gained per watt-hour.
This creates a strategic risk for pure inference ASICs. A forward-only processor may be extremely valuable for serving, but it cannot own the complete learning loop if every meaningful update must return to a GPU.
The first specialized accelerator vendor to support economically efficient sparse adaptation may create more strategic value than a competitor offering another twofold gain in forward-only throughput.
The H100 Will Die in Premium Serving and Be Reborn as a Dream Machine
The normal GPU depreciation model assumes each new generation pushes the prior generation toward irrelevance.
That model treats AI compute as one workload.
It is not.
Frontier silicon should dominate workloads where memory capacity, power efficiency, tail latency, and scale-up bandwidth directly determine revenue. Rubin- and Helios-class systems will eventually displace Hopper from the most demanding premium serving and large-context workloads.
Replay, simulation, evaluation, distillation, and many regression tests have different requirements.
They often have:
no human waiting for the next token;
tolerance for queueing;
weak tail-latency requirements;
large amounts of independent task-level parallelism;
limited need for one enormous scale-up domain;
flexibility to run in cheaper facilities;
economics measured in cost per tested episode rather than revenue per interactive token.
That is precisely where previous-generation hardware can remain useful.
Not every verification job can move. Hardware-specific latency tests, numerical tests, and production canaries must run on the target system. But much of the functional replay and simulation layer does not need the newest rack.
The hardware lifecycle becomes:
Frontier training and premium serving → mainstream inference → replay and evaluation → salvage
Residual value becomes workload-dependent rather than launch-date-dependent.
FPX’s internal May–July 2026 quote tape placed a used H100 server midpoint at approximately $162,500, based on an indicated $150,000–$175,000 range. A separate new liquid-cooled H100 configuration was quoted at $182,200. These are different configurations and not an apples-to-apples formal index, but the used midpoint remained approximately 89% of the separate new-system quote.
That is not evidence that continual learning is already driving H100 prices.
It is evidence that useful workload demand, availability, configuration, and deployment speed matter more to residual value than the mere existence of a newer GPU generation.
The economic test is simple:
Older silicon wins when its purchase-price discount exceeds its additional power cost, colocation cost, runtime penalty, and operational risk.
That inequality often fails for premium serving. A newer rack can produce more revenue from the same megawatt.
It can hold for asynchronous replay.
A slower simulated episode may still be economically superior when the server costs half as much, the facility is cheaper, and the workload can be distributed across ordinary Ethernet-connected nodes.
The H100 will die in premium serving and be reborn as a dream machine.
For hardware owners and neoclouds, this creates a potential second demand curve underneath interactive inference.
For data center operators, it creates a use case for power-rich facilities that cannot win the latency or density race for frontier serving.
For secondary-market buyers, it makes testability, provenance, networking completeness, and remaining useful workload more important than generation labels alone.
The Obvious Memory Trade Is HBM. The Less Obvious One Is Everything Behind It.
FPX’s previous memory report argued that power determines where a data center can exist, while memory determines whether the racks do useful work. It separated the AI memory hierarchy into HBM, server memory, and storage. (FPX World)
Continual learning strengthens all three layers, but for different reasons.
HBM: active intelligence
HBM holds the active model, attention state, KV cache, and the tensors needed for high-throughput adaptation.
The obvious beneficiaries remain the leading memory suppliers:
Micron ($MU);
SK hynix (KRX:000660);
Samsung Electronics (KRX:005930).
Micron says its HBM4, SOCAMM2, and PCIe Gen6 SSDs are now in high-volume production. Its HBM4 is designed for Vera Rubin, while its SOCAMM2 implementation supports up to 2 TB and 1.2 TB/s of bandwidth per Vera CPU. These are company specifications. (Micron Technology)
HBM remains the highest-value layer, but it is not necessarily the fastest-growing number of bytes.
Host memory: the environment layer
Agent fleets create large CPU and DRAM demand.
The model may run on a GPU, but browser instances, code sandboxes, databases, retrieval systems, simulators, compilers, tool calls, and stateful enterprise applications often run on CPUs.
The relevant unit is no longer host memory per GPU server.
It is host memory per concurrent environment.
As the number of agents increases, the environment fleet can scale faster than the number of model replicas. NVIDIA’s Vera CPU and SOCAMM architecture is an explicit response to that shift. (NVIDIA Developer)
This is one of the least appreciated consequences of agentic and continual-learning infrastructure:
The next memory wall is not only fitting the model into HBM. It is keeping enough warm state close to a fleet of agents acting in parallel.
CXL: stranded memory becomes an economic problem
Memory demand will not arrive evenly.
Some servers will have excess DRAM while others become constrained by KV cache, retrieval, or environment state. That creates a stronger case for memory expansion and pooling through CXL.
Astera Labs ($ALAB) markets its Leo controllers for memory expansion and pooling, with configurations supporting up to 2 TB of additional DDR5 memory per controller. (ASTERA LABS, INC.)
CXL will not replace HBM. The latency and bandwidth profile is different.
Its role is to keep moderately hot state out of the most expensive memory tier while preventing the system from falling all the way to storage.
The future hierarchy becomes:
SRAM → HBM → SOCAMM/DRAM → CXL memory → local NVMe → shared context storage → object storage
Every step is cheaper and larger.
Every step is also slower.
The infrastructure problem is deciding which state deserves to live where.
Continual Learning Creates a Storage Barbell, Not a Generic NAND Supercycle
The basic “AI needs more SSDs” thesis is already visible in the market.
TrendForce reported that enterprise SSD pricing was expected to rise 53–58% quarter over quarter in the first quarter of 2026, followed by a separate research estimate of 48–53% in the second quarter, driven by AI and general-server demand. It also reported broad NAND and server-memory pressure as suppliers redirected capacity toward enterprise and AI products. (TrendForce)
Continual learning is not the demonstrated cause of those increases. Current demand already comes from inference, cloud deployments, long context, general server upgrades, and aggressive procurement.
The new thesis concerns the type of storage that continual learning creates.
There are two different workloads.
The cold experience ledger
Raw trajectories, historical documents, audio, video, environment snapshots, prior model branches, and compliance records are generally:
capacity-heavy;
append-oriented;
infrequently modified;
compressible;
tolerant of higher latency.
This favors high-capacity QLC SSDs, object storage, and, where latency permits, HDDs.
Solidigm’s D5-P5336, for example, scales to 122.88 TB with endurance around 0.6 drive writes per day depending on capacity. It is designed around density and read-oriented workloads rather than extreme write intensity. (Solidigm)
Hot learning state
Replay buffers, persistent KV cache, candidate checkpoints, adapter churn, optimizer state, branch copies, index compaction, and simulator snapshots are different.
They are:
repeatedly written;
frequently read;
latency-sensitive;
affected by write amplification;
costly to recreate during a live learning cycle.
This tier favors higher-performance, higher-endurance enterprise SSDs.
Solidigm’s D7-P5810 is rated for up to 50 random drive writes per day and is positioned for caching, logging, and extreme write-intensive workloads. Its D7-PS1030 targets less extreme but still write-centric and mixed workloads such as checkpointing. (Solidigm News)
The endurance wall is in working memory, not the experience ledger.
A 1,000-Agent Storage Sensitivity
Consider 1,000 agents completing 20 substantive episodes per day:
1,000 agents × 20 episodes
= 20,000 episodes per day
The important storage variable is not the number of tokens generated. It is how much state the operator retains from each episode, how many copies or replay branches it creates, and how much additional writing occurs inside the storage system.
A dimensionally consistent model is:
Physical writes per day
= retained episodes per day
× retained state per episode
× copy and branch multiplier
× write amplification
To make the breakpoint visible, assume all 20,000 episodes are retained temporarily, with a 2× multiplier for replay branches and temporary copies, followed by 2× physical write amplification from compaction, metadata, garbage collection, and checkpoint churn.
Retained state per episodePhysical writes per dayDWPD on a 500 TB hot tier50 MB4 TB0.008500 MB40 TB0.082 GB160 TB0.326.25 GB500 TB1.0
Under these assumptions, the hot tier reaches one full drive write per day when the system retains approximately 6.25 GB per episode.
That threshold moves substantially with system design. If only 10% of episodes are retained, the endurance requirement falls by 90%. Compression, deduplication, selective retention, and recomputation lower it further. Large KV snapshots, simulator states, replay trees, checkpoint copies, and frequent branch creation push it in the opposite direction.
The conclusion is more nuanced than “continual learning creates an endurance wall.”
Compact memories and sparse model deltas primarily create a capacity problem. Persisting large working states and repeatedly branching them can create an endurance problem.
If an endurance wall emerges, it will appear in hot working state before the cold experience archive.
That is the storage-market distinction worth tracking. Capacity-oriented QLC can absorb raw trajectories and long-term archives, while write-heavy working state could support a separate premium for TLC, SLC-assisted cache, and performance enterprise SSDs.
The market implication is not simply that NAND demand rises.
It is that high-endurance enterprise SSD pricing can decouple from raw capacity NAND. QLC absorbs the cold side. TLC, SLC-assisted tiers, and performance SSDs absorb hot state.
The companies exposed to this hierarchy include Micron ($MU), Samsung, SK hynix and Solidigm, and storage-system companies such as VAST Data, WEKA, and $PSTG.
NVIDIA has already introduced CMX as a distinct context-memory storage tier. VAST is integrating shared KV cache and context management directly with BlueField-4 and NVMe infrastructure, while $PSTG is developing FlashBlade capabilities around NVIDIA’s STX and CMX architectures. (VAST Data)
This is the deeper signal:
Inference is becoming a memory system. Continual learning turns that memory system into a write system.
The Network Becomes the Learning-Loop Fabric
FPX’s prior networking report followed a bit from the accelerator package through copper, Ethernet, optics, metro networks, and long-haul fiber. Its central argument was that the network is not a supporting component; it determines whether distributed accelerators behave like one useful machine. (FPX World)
Continual learning adds two more traffic patterns.
Traditional frontier training emphasizes synchronized scale-up and scale-out communication.
Agentic inference adds persistent context access and low-latency service coordination.
Continual learning adds movement between production, replay, adaptation, validation, and storage pools.
The facility now contains at least three major fabrics:
DPUs and SmartNICs become more important because they sit at the boundary between these systems. They can enforce isolation, move storage traffic, manage security, and remove infrastructure services from the host CPU.
Broadcom’s ($AVGO) Tomahawk 6 provides 102.4 Tb/s of Ethernet switching capacity and is positioned for both AI scale-up and scale-out. (Broadcom)
Astera Labs ($ALAB) provides another part of the fabric through CXL memory connectivity.
NVIDIA owns the most integrated version through Spectrum networking, BlueField DPUs, NVLink, Dynamo orchestration, and its storage partners.
AMD is building an open alternative around Pensando, UALoE, and Ethernet. AMD now describes front-end, scale-up, and scale-out networking as three distinct layers, with agentic systems adding persistent-memory and coordination pressure to the fabric. (AMD)
The infrastructure alpha is not just “more bandwidth.”
It is control over the movement of the state that makes learning persistent.
The network is no longer only the fabric connecting GPUs. It becomes the fabric connecting versions of the model.
Continual Learning Splits the Data Center Market
The consensus infrastructure story says more AI means larger campuses.
That remains true for some parts of the workload.
It is not true for the entire learning loop.
Frontier pretraining, large consolidation jobs, synthetic-data generation, and asynchronous simulation want enormous pools of cheap, dense power. These workloads can tolerate geographic distance and are economically attracted to multi-gigawatt campuses.
The live learning loop wants proximity to the environment generating the experience:
enterprise databases;
identity and permission systems;
human operators;
operational tools;
local storage;
regulated data;
and the downstream systems revealing whether the action worked.
Latency, sovereignty, data gravity, and learning rights all favor keeping this portion of the system closer to the enterprise.
The result is a barbell.
Power campuses
Large campuses remain the blast furnaces for:
frontier pretraining;
global model consolidation;
large post-training runs;
massive simulation;
synthetic-data generation;
asynchronous replay.
Metro, sovereign, and enterprise-adjacent facilities
Smaller sites become the factory floor for:
live serving;
tool execution;
outcome capture;
private memories;
customer-specific adapters;
sensitive model state;
regulated workflows;
low-latency agent interaction.
This builds directly on the bifurcation FPX described in Delivered Megawatts and The Right Way to Run a NeoCloud: training and inference increasingly want different locations, densities, networks, and commercial structures. Continual learning makes the split more pronounced by adding private state and adaptation to the inference side. (FPX World)
Continual learning adds a second location requirement. It does not eliminate the mega-campus.
The strategically vulnerable asset may be the generic remote 100–300 MW campus: too small to command the full economics of frontier-scale training, but too distant from users and enterprise systems to own the live learning loop.
A generic remote mid-scale campus will need a clear specialization. It may compete through cheap replay, sovereign capacity, storage-heavy workloads, disaster recovery, regional consolidation, or a particular accelerator ecosystem. A campus that offers neither frontier scale nor proximity becomes harder to differentiate.
Those facilities will need to specialize:
low-cost replay;
sovereign compute;
storage-heavy workloads;
a particular accelerator ecosystem;
disaster recovery;
or regional consolidation.
The learning facility has four zones
A hall of identical GPU racks is poorly matched to this workload.
The reflex zone
Low-latency inference systems handle interactive decode and real-time agent responses.
The plasticity zone
HBM-heavy accelerators handle prefill, adaptation, consolidation, and selected backward passes.
The world zone
CPU- and DRAM-heavy infrastructure runs browsers, code sandboxes, virtual machines, databases, digital twins, and evaluation environments.
The memory zone
Context systems, SSDs, object storage, adapter registries, replay buffers, and outcome histories preserve the state required to learn.
The zones have different rack densities, cooling requirements, load profiles, network topologies, and redundancy economics.
That is the facility-design opportunity.
Vertiv ($VRT) now offers Vera Rubin reference architectures at 12.5, 50, 125, and 250 MW, scaling toward 1 GW. Eaton ($ETN) is integrating grid-to-chip power and cooling into Rubin designs. Schneider Electric’s Vera Rubin reference architecture supports 480 VAC distribution, liquid cooling, and rack power approaching 227 kW. Modine ($MOD) is expanding multi-megawatt heat-rejection systems for high-density AI facilities.
Those reference designs are currently built around frontier AI factories. The continual-learning facility will require the same integration discipline but a more heterogeneous internal mix.
The winning colocation operator will not simply advertise “AI-ready” power.
It will be able to provide the correct ratio of:
liquid-cooled accelerator capacity;
air- or liquid-cooled CPU environments;
context-memory storage;
high-endurance NVMe;
front-end and back-end fabrics;
private connectivity;
security and data-sovereignty controls;
and lower-cost replay capacity.
Where the Real Alpha Sits
The following is an exposure map, not a ranking or recommendation.
The most crowded trade is frontier compute.
The less standardized markets are:
CPU environments per agent;
hot-state bytes per agent;
validation compute per update;
adapter and branch serving;
endurance per learning loop;
and the residual value of previous-generation accelerators as replay fleets.
Those are the variables worth tracking.
The Signals That Would Confirm the Thesis
The Deployment Scaling Law becomes real in the market before anyone publishes a clean coefficient.
Watch for the following:
The thesis would weaken if long context and external memory prove sufficient to create durable whole-job competence without model adaptation.
It would also weaken if enterprises routinely grant broad cross-tenant learning rights without extracting meaningful economics, if private model branches cannot be merged or moved reliably, or if shadow compute remains a small fraction of production serving.
The hardware thesis would be wrong if new accelerators’ energy and performance advantages always overwhelm the acquisition discount on older hardware, leaving no meaningful replay market.
The storage thesis would be wrong if selective retention, compression, recomputation, and eviction prevent hot state from reaching material endurance requirements.
These are not distant philosophical questions.
They will appear in:
enterprise contracts;
cloud pricing;
rack diagrams;
CPU-to-GPU ratios;
storage product mix;
secondary-market prices;
colocation leasing;
and power-and-cooling reference architectures.
The New Procurement Metric Is Learning Capacity
The market currently prices AI infrastructure in GPU-hours, tokens per second, accelerator counts, and megawatts.
Those units describe how much machinery was used.
They do not describe what the system gained.
A GPU-hour does not tell us whether the model completed a useful task, observed the outcome, extracted the lesson, retained it, or passed the tests required to use that lesson safely.
The more useful future unit is learning capacity:
The ability to convert authorized experience into verified, portable, durable capability.
Learning capacity includes:
production episodes served;
environments operated;
outcomes captured;
counterfactuals simulated;
candidate updates generated;
regressions completed;
updates safely promoted;
prior skills retained;
and durable capability gained per dollar and per megawatt.
For compute buyers, this changes procurement.
Buying “the best GPU” is not system design. The workload must first be separated into serving, environments, adaptation, replay, verification, and persistent state.
For data center operators, it changes the product. The facility must support a heterogeneous learning loop rather than only dense accelerator racks.
For hardware owners, it changes residual value. A server that falls out of premium serving may still have substantial economic life in replay and assurance.
For investors, it changes where to look. The next bottlenecks may form in CPU environments, warm memory, SSD endurance, context storage, networking, and validation capacity before they become obvious in GPU shipment numbers.
This is also why the FPX marketplace cannot be organized around one hardware category. The relevant infrastructure now spans compute, complete servers, memory, SSDs, networking, power, cooling, and colocation. A buyer does not ultimately need a GPU. It needs a system capable of completing the learning loop under a real budget, delivery date, power envelope, and operational environment.
The first three scaling regimes taught AI to know, act, and think.
The next one teaches it to accumulate experience.
The next scaling law will not be won by the model that knows the most on Monday. It will be won by the system that learns from Tuesday and is measurably better on Wednesday.
This material is for informational purposes only and does not constitute investment advice. Product specifications and performance figures attributed to vendors are company-reported and may not be directly comparable across architectures. Calculations identified as illustrative are sensitivity analyses rather than forecasts. FPX marketplace observations are anonymized, directional, configuration-dependent, and not a formal price index.