Direct relationships are already emerging. Micron announced an agreement with Anthropic in June, covering memory and storage supply, technical collaboration and a strategic investment. This broadens the story beyond Samsung and SK hynix and shows that the relationship is developing before the reported 2027 procurement shift.

For standard NVIDIA platforms, direct engagement with a memory supplier would still require coordination with the accelerator vendor. HBM must meet the platform’s design and qualification requirements. NVIDIA itself maintains a multi-year memory development and supply partnership with SK hynix.

The potential benefit for model developers is earlier control over supply allocation, delivery schedules and commercial terms. A reservation could, in principle, support systems built by an agreed manufacturing partner. Public disclosures do not establish that these arrangements provide transferable capacity rights or fixed memory prices. Custom ASIC programs offer greater scope for joint design, with the model developer, chip designer and memory supplier determining requirements together.

Editor’s View

Memory Capex: Direct procurement strengthens the case that memory availability is becoming a strategic planning issue for AI developers. Its financial significance depends on the commitments behind the relationship.

Binding volumes and customer funding could improve investment visibility for memory manufacturers. Flexible reservations or price ceilings could leave suppliers carrying capacity risk while limiting their upside. For buyers, securing memory could protect deployment schedules, but lower component costs would reduce the final system price only if the purchasing arrangement passes those savings through.

A change in buyer identity also does not establish incremental demand: memory previously purchased through a cloud or accelerator supplier may simply move to a different contract. The evidence to watch is firm volume commitments, cancellation terms and pricing mechanisms—and whether those commitments support better returns on new capacity.

Aju Press ↗

First published Sep 26, 2026, 03:57 UTC

The evidence is an analyst reading of buyer behavior, not a disclosed contract. TrendForce did not name the CSPs, publish an order volume, or give a 4Q26 price range in its public release. It attributed the revision to major CSPs speeding up deployments and extending enterprise SSD use to more applications. It also said the increase is not confined to high-performance TLC. QLC has become a significant growth driver.

The base is already high. TrendForce estimated that combined revenue of the top five enterprise SSD brands reached US$37.59 billion in 2Q26, up 103.6% QoQ, driven by higher shipments and higher contract prices. Samsung ranked first at US$14.35 billion, followed by SK hynix at US$8.63 billion and Micron at US$6.98 billion. The new release extends that trajectory from 3Q26 into 4Q26. It does not introduce a new direction.

The mechanism runs through workloads beyond LLM training. TrendForce points to agentic AI systems that need large-scale real-time data retrieval and caching, and to vector databases, where QLC’s capacity per dollar suits unstructured data. In China, it said DeepSeek-led architectures are pushing high-capacity QLC for KV cache offloading. If QLC takes a growing share of enterprise demand, suppliers can steer high-layer-count wafers toward enterprise products, where contract pricing is firmer, and away from client and mobile. TrendForce said the shift gives NAND suppliers greater control over capacity allocation and pricing in the enterprise segment.

The consumer side is moving differently. In its weekly spot update on 23 September, TrendForce reported 512Gb TLC wafer spot prices down 1.00% to US$19.883. It cited weak consumer demand and capacity releases from Chinese suppliers. Enterprise contract prices and wafer spot prices are separate markets, and this week they point in opposite directions. That split is the more useful reading: the enterprise segment is tightening through allocation, not through an industry-wide bit shortage.

The event has limits. A forecast upgrade is not a signed order, and an order is not revenue. The release says 4Q26 orders “could” exceed 3Q26 but does not say by how much. It adds no NAND wafer capacity, and it does not address server DRAM or HBM. It also does not show whether the revision reflects new demand or orders pulled forward from 1Q27 ahead of further price increases.

Three questions remain open. First, QLC mix growth could lift bits faster than average selling prices, since QLC is sold on cost per terabyte. Second, long-term agreements with price caps would limit how much of the gap between spot and contract prices suppliers can capture. Third, if Chinese capacity now feeding the spot market moves into qualified enterprise products, it would test supplier pricing power.

Editor’s View

MemoryCapex View: This is a buyer-side demand revision reported by an analyst firm. It is not a price print or a supply event. It supports the case that enterprise NAND pricing into 4Q26 is set by CSP allocation, and that this pricing is diverging from consumer wafer spot.

The strategic logic is mix. Suppliers that can ship high-capacity QLC enterprise drives at scale gain pricing leverage, while those weighted to client NAND face a softer spot market. The milestones to watch are TrendForce’s 4Q26 enterprise SSD contract price forecast, its 3Q26 enterprise SSD revenue ranking, supplier commentary on QLC mix in the next earnings round, and whether weekly TLC wafer spot keeps falling.

First published Sep 23, 2026, 17:30 UTC

The companies did not confirm a plan. SK hynix said Solidigm “is reviewing various options to strengthen its business competitiveness, but no specific plans have been confirmed.” Solidigm said it does not comment on rumors and that no decision has been made. Reuters gave no figures for capacity, capital cost, node or timeline. This is an anonymous-source report of an evaluation, not an announcement.

The core issue is single-site exposure. Solidigm’s only NAND fab is in Dalian, China, acquired from Intel in a $9 billion deal closed in 2020. That plant now runs on U.S. permission renewed yearly: the validated end-user status that let it receive chipmaking tools without shipment-by-shipment licenses expired at the end of 2025, and Tom’s Hardware reported that the 2026 annual licenses for Dalian and Wuxi are designed to allow continued operation and maintenance rather than unrestricted expansion. In a 3D NAND business where cost per bit depends on migrating to higher layer counts, a fab that can be maintained but not freely upgraded is a slow structural disadvantage. A U.S. site would remove that ceiling and reduce exposure to potential U.S. tariffs.

The policy backdrop is two-sided. Reuters noted that the U.S. Commerce Secretary has threatened tariffs of up to 100% on Korean and Taiwanese firms that do not commit to U.S. production, while Seoul is pressing its chipmakers to invest in a domestic cluster and called the U.S. factory question “a matter of controversy.” One source said NAND may draw less Korean resistance than HBM or DRAM because it is seen as less strategically sensitive.

The financial constraint is real. Reuters reported that Solidigm has posted years of losses since the acquisition, and that SK hynix said last month it was reviewing measures to strengthen the unit after reports of a pre-IPO fundraising of about 5 trillion won ($3.6 billion). U.S. production is more expensive than Korean production. Whether a U.S. fab would be funded by SK hynix, by outside capital through an IPO, or by public incentives is not disclosed, and Reuters does not link the two items.

For 2026–27 supply, this changes nothing. A greenfield NAND fab takes years from decision to wafer output, and no decision has been taken. Current NAND tightness driven by AI data-center demand is unaffected, and any output would arrive in a cycle whose demand Reuters itself described as uncertain. Competitive pressure is rising in parallel: CXMT plans a NAND R&D line in Beijing, and Chinese capacity continues to grow.

Editor’s View

MemoryCapex View: This is a risk-management signal, not a supply event. The story is about a fab that can be kept running in China under annual U.S. license but cannot be freely upgraded, and it rests on unnamed sources with no confirmed plan, budget or timeline.

The capital-allocation question is the real one: a loss-making NAND unit weighing a high-cost U.S. fab while SK hynix funds HBM expansion in Korea, in a NAND market more prone to oversupply than DRAM. Milestones to watch are a formal SK hynix or Solidigm announcement, incentive filings for a New York site, the pre-IPO outcome, the 2027 license renewal for Dalian, and SK hynix’s next disclosure on NAND profitability and capex.

First published Sep 21, 2026, 18:00 UTC

The comments land as both Korean suppliers push CXL 3.2 products. Seoul Economic Daily reported in July that Samsung plans to mass-produce its CMM-D 3.0 module on CXL 3.2 within the year, and that SK hynix has shown a second-generation CMM-DDR5 sample on CXL 3.2 and begun customer sales talks, without a production schedule. SK hynix researchers also published ITME, an inference architecture that inserts CXL-hybrid memory between DDR and SSD to hold KV cache; the paper reports a 1.80× throughput gain over NVMe-oF disaggregated storage and up to 35.7% over a CPU-offload baseline.

Read carefully, the two sides are not in conflict. The suppliers position CXL as a tier below DRAM for cold KV cache and capacity expansion; the buyers say it is not HBM. The mechanism that matters for memory demand is whether CXL modules pull DRAM bits into a new socket, not whether they displace HBM. That question is settled by hyperscaler deployment volumes, which neither side disclosed.

The remarks do not change HBM demand or 2026–27 DRAM allocation. They do cap the near-term narrative for CXL as an “HBM alternative” and shift the burden to the suppliers to show paying customers.

Editor’s View

MemoryCapex View: This is a framing correction, not a demand event. Two buyer-side engineers said what the product roadmaps already imply: CXL is a capacity tier, and HBM stays the bandwidth tier. It is a conference-panel report, not a purchasing decision.

The strategic question is whether CXL becomes an incremental DRAM socket in inference clusters or stays a niche. Watch for Samsung’s CMM-D 3.0 production announcement by year-end, any named CXL customer from SK hynix, and hyperscaler references to CXL in earnings or infrastructure blogs.

TrendForce ↗

First published Sep 21, 2026, 02:35 UTC

CXMT is a DRAM company. TrendForce ranked it fourth in DRAM with a 9.5% share in 2Q26; in NAND, Samsung led with a 29.3% share. CXMT has capital to deploy after its 57.92 billion yuan ($8.6 billion) July IPO, and Reuters reported in August that it plans a second DRAM fab in Beijing.

The NAND market is short and expected to stay short: TrendForce estimates a 4–5% supply deficit in 2026, easing only in the second half of 2027, and expects Chinese suppliers to reach nearly 19% of global NAND bit output in 2027. A CXMT line that reached volume would add a second state-backed Chinese NAND supplier beside YMTC, which itself sent LPDDR samples to customers in April.

An R&D line is not capacity. It changes nothing in 2026–27 NAND supply or pricing. A new entrant must develop a 3D NAND process, reach yield and qualify with customers, each step measured in quarters and constrained by export controls on tools. The open questions are whether CXMT intends volume production at all, and how it splits capital between a second DRAM fab and NAND while DRAM and HBM carry the better returns.

Editor’s View

MemoryCapex View: This is a directional signal about CXMT’s ambitions, not a supply event. It is anonymously sourced, with no timeline, node, capacity or budget, and changes nothing in the 2026–27 NAND balance.

The logic is that China’s two memory champions are converging, both with fresh IPO capital and state support. For incumbents the risk is a 2028-and-beyond story about Chinese NAND bit share. Watch for a CXMT statement or filing, equipment orders at the Beijing plant, the identity of the AI-storage customer, and TrendForce’s next NAND supply update.

Reuters ↗

First published Sep 20, 2026, 03:08 UTC

The figure is not usable as data. Intel published no transcript, the remarks reach us through secondary reporting, and the multiple carries no product, no base date and no price type. TrendForce places the comments at Splunk’s .conf26 broadcast on September 16, a day later than the Korean account, so even the venue is unsettled.

Two other numbers from the same remarks are firmer and more useful. Tan said Intel can meet only about 50% of customer demand for processors, and that advanced packaging substrates come from four suppliers, two Japanese and two Taiwanese, whom Intel has to prepay to get attention. That is a large buyer treating prepayment as normal for a scarce input, the same behaviour now visible in memory long-term agreements.

What gives the remarks weight is the speaker’s position. Suppliers describing shortage have an interest in the narrative; a buyer whose costs rise has the opposite one, and is describing delayed projects rather than disputed forecasts. None of it adds supply, and the squeeze he attributes to HBM reallocation is the same mechanism that leaves less capacity for general-purpose memory into 2027.

Editor’s View

MemoryCapex View: This is demand-side confirmation that tightness runs into 2027, not a price data point. The delayed projects, the 50% CPU fill rate and the substrate prepayment are the parts worth keeping. Watch Micron’s next results and guidance, the fourth-quarter and first-quarter 2027 contract price releases, and revisions to PC and smartphone shipment forecasts.

Seoul Economic Daily ↗

First published Sep 17, 2026, 18:05 UTC

The report names the partners absorbing the work. Dreamtech is completing an expansion of its Noida plant in India this month, targeting annual capacity above 50 million units. Hanyang Digitech in Vietnam invested ₩31.5 billion in new machinery and automation in the first half of 2026. SFA Semicon in the Philippines is receiving DDR5 equipment transferred from Onyang, with first-phase operations scheduled for November. Samsung has not commented. These details come from industry sources and partner plans, not company disclosure.

The decision fits a pattern already visible in Samsung’s capital plans. Korean media reported in August that Samsung would break ground this month on a ₩6 trillion HBM plant at Onyang. In May, Reuters reported that Samsung is building a roughly $1.5 billion test plant in Thai Nguyen, Vietnam, for legacy DRAM and NAND, with operations scheduled for November 2027. Leading-edge back-end capacity is being concentrated in Korea for HBM, while commodity assembly and test move to partners and lower-cost sites.

That split matters for how the constraint should be read. HBM is packaging-intensive: each stack requires thinned dies, through-silicon vias, stacking and extensive testing, which consume far more back-end tooling, floor space and engineering time than a standard module. A DDR5 module mounts finished packages on a circuit board. That is mature work that many firms can do. Moving it outside protects scarce Korean capacity for the step that is hardest to replicate.

The move also has clear limits. Outsourcing module assembly adds no DRAM wafer starts. It creates no HBM output until new packaging lines are built, tooled and qualified, and Samsung has not disclosed a completion date for the Onyang plant. In its second-quarter results, Samsung said that “despite efforts to increase production, supply constraints are expected to continue.” Nothing in this report changes that assessment for 2026 or 2027.

Execution risk shifts as well. The report does not say whether outsourced volume includes server RDIMMs, which customers qualify closely, or only client modules. Partners introduce additional quality and scheduling variables, and some assembly margin moves outside Samsung, although that value is small relative to the DRAM inside each module. For DDR5 buyers, the practical question is whether partners scale quickly enough to keep assembly, rather than wafers, from becoming a local bottleneck.

Editor’s View

MemoryCapex View: We read this as evidence that back-end capacity, not only front-end wafers, has become a binding constraint in Samsung’s HBM strategy. Samsung appears to be ranking its Korean back-end floor space by value, and conventional modules are losing that ranking.

The move supports HBM supply growth over time. It adds no near-term bits and does not loosen DRAM supply. The milestones to watch are confirmation from Samsung, SFA Semicon’s November start, equipment orders for Onyang, and whether SK hynix or Micron outsource conventional assembly in a similar way.

TheElec ↗

First published Sep 17, 2026, 06:27 UTC

Announced in August, the Boise-based institution is backed by a planned $10 billion investment over the next decade. Construction is expected to begin in 2027. Its agenda spans memory process technologies, future memory and compute architectures, advanced packaging and next-generation manufacturing, with research extending beyond current product roadmaps.

Hanford brings decades of experience at Synopsys, including leadership of its Design Group and service as chief security officer. She was also the founding CEO of Natcast, the organization previously responsible for operating the National Semiconductor Technology Center. Her documented strengths lie in technology management, electronic design automation and coordinating industry, government and academia. Public biographies do not establish a comparable record of direct leadership in DRAM or NAND development.

That background could help Micron connect expertise across design, manufacturing and packaging, while strengthening recruitment and research partnerships. The appointment itself establishes no technical breakthrough. Its value will depend on the scientific team Hanford assembles and the lab’s ability to translate research into manufacturable technologies with measurable performance, energy-efficiency and cost advantages.

Editor’s View

MemoryCapex View: The larger development is Micron’s commitment to a dedicated institution for sustaining technological competitiveness. We see the lab as a long-term effort to remain competitive in an increasingly demanding industry. Commercial returns remain unproven.

This appointment announces no additional capex and provides no basis for revising near-term production-capacity assumptions. However, the broader laboratory program includes construction, so describing it as entirely capex-neutral would be inaccurate. Micron has not disclosed the split between capital expenditure and R&D expense for the program. The milestone to watch is whether this research organization can turn investment into differentiated, commercially relevant memory technology.

First published Sep 16, 2026, 16:31 UTC

SanDisk targets bandwidth comparable to HBM with eight to sixteen times the capacity at a similar cost. These are company claims, not established system-level economics. The comparison also requires a defined HBM generation, capacity and operating configuration. Greater capacity does not imply proportionately higher inference performance.

The engineering challenge is that bandwidth and access latency are different constraints. Many parallel NAND operations can deliver substantial aggregate throughput while individual reads remain slower than HBM accesses. Effective integration therefore depends on buffering, prefetching and workload scheduling. Recent FLINT research explicitly identifies NAND read latency and flash-management overhead as obstacles that an HBF architecture must address. FLINT research

This makes relatively stable model weights a compelling starting point. A system could retain frequently modified data in HBM while assigning larger, predominantly read-oriented datasets to HBF. The economic benefit would be strongest where memory capacity forces a model across more accelerators than its computational workload otherwise requires. In that situation, additional near-compute capacity could reduce hardware duplication and interconnect requirements. The benefit would be smaller where compute throughput or strict latency targets already determine accelerator count.

The evidence needs careful interpretation. SanDisk’s published performance comparison uses internal simulation of pretrained-weight reads for Llama 3.1 405B, with hypothetical unlimited-capacity HBM as the reference. It does not establish equivalent performance across complete inference services, concurrent users or changing context lengths.

The more recent milestone is ecosystem development. SanDisk and SK Hynix announced an initial HBF specification through the Open Compute Project in August 2026, with Google and Tenstorrent participating in the consortium. Shared interfaces and implementation guidance can reduce adoption friction, although standardization remains distinct from commercial qualification and volume deployment.

For SanDisk, HBF creates a potential route into higher-value AI memory applications. However, attractive customer economics would not automatically guarantee attractive supplier margins. Specialized silicon, stacking, controllers, qualification and yield losses must be absorbed, while an open ecosystem may encourage competing implementations.

Editor’s View

MemoryCapex views HBF as a credible architectural opportunity whose commercial value must be demonstrated at the service level. The decisive benchmark is sustained tokens per dollar and per watt at an acceptable latency—not capacity per stack alone. Successful adoption could expand NAND’s role in AI while preserving substantial demand for HBM; it would not, by itself, establish either wholesale HBM replacement or an immediate earnings uplift for Sandisk.

DigiTimes ↗

First published Sep 16, 2026, 08:31 UTC

Access to Ohio infrastructure could reduce some development work, but it would not provide immediately usable memory capacity. A memory fab requires a production flow configured for the selected technology, equipment installation, process qualification and a yield ramp. The talks do not establish that SK hynix would manufacture memory using Intel’s logic process nodes. Intel’s published schedule places initial operations at its first Ohio fab in 2030–2031; a partnership could change that schedule, but no revised timetable exists. Intel

The distinction between wafer fabrication and packaging is essential. SK hynix already has an advanced HBM packaging project in Indiana, targeting mass production in late 2029. If Ohio ultimately hosts memory wafer fabrication, it could complement that downstream investment. However, describing the proposed venture as an HBM factory would be premature: Reuters could not establish which chips it would produce. SK hynix

Potential cloud-customer participation is the most consequential economic feature. Equity contributions, prepayments or binding purchase commitments could lower SK hynix’s funding burden and reduce utilization risk. Customers, however, would likely seek compensation through pricing, priority allocation or investment returns. Better financing therefore would not automatically translate into higher margins. The relevant test is the project’s return on invested capital after construction costs, ramp losses and contractual concessions.

Political pressure could also influence the decision. Reuters reports pressure from Washington to localize production and potential Korean scrutiny of sensitive technology transfers. U.S. labor, construction and supply-chain costs could make the facility less competitive than an equivalent Korean investment. Subsidies, customer premiums or shared infrastructure might offset that disadvantage, but their value remains unknown. Strategic access to the U.S. market could justify some additional cost without making the project economically superior.

For Micron, a successful project could eventually challenge the differentiation associated with its expanding U.S. manufacturing base. Micron currently targets initial wafer output from its first new Idaho fab in mid-2027, although initial output is distinct from qualified volume shipments. SK hynix’s discussions do not yet establish a comparable production milestone. Micron

Editor’s View

MemoryCapex views the talks as a potential change in the financing and geography of memory supply. Their investment significance will depend on whether U.S. production adds profitable capacity, redirects existing expansion plans, or imposes higher costs primarily to satisfy political and customer requirements.

First published Sep 16, 2026, 07:50 UTC

Editor’s View

MemoryCapex views this announcement as evidence of engineering progress and a potential product-mix opportunity, rather than proof of a durable competitive advantage. Samsung’s earlier 512GB announcement underscores why qualification, manufacturing yield, and cost matter more than capacity alone. Without pricing, shipment volumes, or margin visibility, this milestone does not yet justify higher Micron earnings estimates on its own. (Source)

Micron Newsroom ↗

First published Sep 16, 2026, 06:30 UTC

Interactive map · AI Memory, Step by Step

The AI Memory Hierarchy

One argument, from the physics of storing a single bit to the question of which part of the semiconductor value chain is paid when a particular constraint binds. Open any node to read it.

Independent research by Memory Capex As of 14 September 2026 64 nodes · three nested levels + root Companion to the three-part series

Part 1 — The Physics and Economics of Memory

Part 2 — What AI Actually Does to Memory

Part 3 — Where the Hierarchy Is Going

Tap or click any node to open its explanation; tap or click again to close it. Every node sits below the node it depends on.

Root — the argument Level 1 — the nine branches Level 2 — components of a branch Level 3 — detail within a component

Memory Hierarchy for AI 9 sub-nodes

This map is a single argument, not a glossary. It runs from the physics of storing one bit to the question of which part of the semiconductor value chain is paid when a particular constraint binds. Every branch is a step in that argument, and each depends on the ones above it.

The organising claim is that AI memory is not one market or one technology. It is a hierarchy assembled from incompatible compromises in speed, capacity, persistence, energy and cost. Because no device maximises all of them, systems are built as layers, and the layer that limits a given workload decides where money and engineering effort go.

The first branches establish what memory is and what AI workloads do to it. The middle branches explain why HBM had to be invented, how data is tiered across the hierarchy, and which interconnects and emerging tiers are reshaping it. The final branch translates all of that into exposure.

Open any node for its explanation. Quantitative claims follow the same primary-source register used in the linked three-part series; each node links back to the relevant sourced section.

Read this in full → Part 1 — The Physics and Economics of Memory

Physics and Economics 3 sub-nodes

Before any AI-specific reasoning, memory obeys a physical and economic logic that has nothing to do with machine learning. A processor cannot operate on data that is not in front of it, and getting data in front of it costs time, energy and silicon area.

That cost is not a single number. It splits into several properties that trade against one another, and the trade-offs are not soft preferences — they are rooted in cell structure, interface width, physical distance and manufacturing economics. A design that maximises one property structurally sacrifices another.

The consequence, which the rest of the map depends on, is that a memory system is a portfolio decision rather than a component choice. Systems buy several technologies at once and place each where its particular compromise is least damaging.

This is also why the memory industry does not converge on a single product. Different customers weight the axes differently, so several distinct businesses with different cost structures and capital intensity can coexist indefinitely.

Read this in full → Part 1 — The Physics and Economics of Memory

Data Movement Machine

The useful first model of a computing system is a loop: load, compute, store, move, repeat. Arithmetic receives most of the attention because it is where the visible work happens, but the other three stages determine whether the arithmetic hardware has anything to do.

Framing the machine this way changes what counts as a bottleneck. A processor that is idle because its operands have not arrived is not short of compute; it is short of delivery. The economic implication is that spending on arithmetic capability has a ceiling set by the data path feeding it.

This is the conceptual shift that makes the rest of the hierarchy legible: the processor is not an isolated engine, it is one component of a system whose job is to place the right bytes close enough at the right moment.

Read this in full → Part 1 — The Physics and Economics of Memory

Six Trade-off Dimensions 6 sub-nodes

Saying a memory technology is “better” is incomplete until you say better on which axis. Six independent properties matter, they interact, and none substitutes for another.

The important structural point is that improving one axis usually costs another through a physical mechanism, not through vendor choice. Making a cell smaller raises density and lowers cost per bit, but tightens electrical margins. Widening an interface raises bandwidth, but consumes package area and power. Moving memory closer to compute lowers latency, but competes for the most expensive silicon real estate on the system.

Because the axes are independent, two customers can both be buying “more memory” while solving entirely different problems and creating entirely different demand for silicon, packaging and equipment. Confusing the axes is the single most common analytical error in reading the memory market.

Read this in full → Part 1 — The Physics and Economics of Memory

Capacity

How much information can be held at all. Capacity is a gating condition: a system can be excellent on every other axis and still fail if the working set does not fit.

Its economic signature is demand for bits — more die area, more wafers, more packages. It says nothing about how quickly those bits can reach compute.

Read this in full → Part 1 — The Physics and Economics of Memory

Bandwidth

How much information can cross the memory interface per unit time. Bandwidth is a throughput property of the pipe, not of the reservoir behind it.

Its economic signature is interface width, signalling rate and packaging complexity rather than raw bit count. This is why bandwidth-driven demand and capacity-driven demand can move different suppliers in different directions at the same time.

Read this in full → Part 1 — The Physics and Economics of Memory

Latency

The delay before a single requested access begins to deliver useful data. A system can have very high aggregate throughput and still impose meaningful waiting time on an individual request, which is why latency and bandwidth are not interchangeable measures.

Read this in full → Part 1 — The Physics and Economics of Memory

Energy

Reading, writing and physically moving information all consume power, and the farther data travels and the more interfaces it crosses, the larger that share becomes. The relevant metric is not the power rating of a chip but the energy required to deliver useful data where computation needs it.

Read this in full → Part 1 — The Physics and Economics of Memory

Persistence

Whether stored information survives loss of power. SRAM and DRAM are volatile and depend on continued power; NAND flash is non-volatile, which is what makes it suitable for storage rather than working memory.

Read this in full → Part 1 — The Physics and Economics of Memory

Cost per bit

The economic cost of providing a given quantity of stored information. It incorporates far more than chip price: silicon area, process complexity, yield, packaging, controllers, power and system integration all feed into it.

Because these inputs differ sharply by technology, cost per bit is the axis that ultimately forces the hierarchy to exist rather than collapsing into one universal memory.

Read this in full → Part 1 — The Physics and Economics of Memory

Incompatible Optimization Goals

The hierarchy is not a historical accident or an engineering convenience. It is the system-level response to the fact that the six axes cannot be maximised together.

Moving up the hierarchy improves proximity and access performance while raising the economic cost of every stored bit. Moving down allows far larger capacity at lower cost but with slower access. No single tier can economically replace the others, which is why the structure has survived every generation of memory technology so far.

For an analyst this is the reason the sector has durable, distinct sub-markets rather than one commodity curve.

Read this in full → Part 1 — The Physics and Economics of Memory

Storage Technologies 3 sub-nodes

Beneath the product names sit three genuinely different physical methods of representing a bit, and those methods are what generate the trade-offs above.

The distinction matters commercially because it separates the industry into businesses with different capital intensity, different process roadmaps and different cycle behaviour. It also settles a confusion that recurs constantly in market commentary: DDR and HBM are not competing memory families, they are two architectures built on the same underlying DRAM technology, while NAND belongs to a different family entirely.

Read this in full → Part 1 — The Physics and Economics of Memory

SRAM

Stores state in a circuit of multiple transistors. As long as power is present the circuit holds its value without the periodic refresh DRAM requires.

The trade-off is area: an SRAM cell consumes substantially more silicon per bit than a DRAM cell, so large capacities are prohibitively expensive. That confines SRAM to small, speed-critical structures on the processor die, and makes it a consumer of scarce logic-die area rather than a separate memory market.

Read this in full → Part 1 — The Physics and Economics of Memory

DRAM

Uses a conceptually simple cell built around one transistor and one capacitor. The capacitor holds charge and the transistor controls access; because the charge leaks, the contents must be refreshed periodically, which makes DRAM volatile.

That compact cell is what allows far higher density than SRAM. Samsung lists DDR, LPDDR, GDDR and HBM together as DRAM product families — the clearest public confirmation that HBM is an unusually sophisticated form of DRAM, not a new memory physics.

Read this in full → Part 1 — The Physics and Economics of Memory

NAND

Retains information by trapping charge in a storage structure, so data persists after power is removed. Cells are arranged vertically to raise density.

This makes NAND the economic base of the hierarchy — large persistent capacity at the lowest cost per bit — rather than a substitute for DRAM. A finished SSD is also more than flash: controller, firmware, interface and error correction are a meaningful part of the product and of the margin.

Read this in full → Part 1 — The Physics and Economics of Memory

AI Workload Occupants 5 sub-nodes

Once the hierarchy exists, the question becomes what actually occupies it. An AI workload does not store one undifferentiated blob; it stores several kinds of state with different sizes, lifetimes and growth drivers.

Weight memory is the only component that follows directly from model size, and it is arithmetic: parameter count multiplied by bytes per parameter, where the byte width is set by the numeric format. Everything else scales with how the model is being used rather than how large it is.

This is why parameter count is a weak predictor of memory demand. Two services running an identical model can have radically different footprints depending on training versus serving, context length, batch size and concurrency — and those differences translate into different purchases of capacity versus bandwidth.

Read this in full → Part 2 — What AI Actually Does to Memory

Weights

The learned parameter values. Their memory cost is the one quantity that can be estimated from first principles: parameters multiplied by bytes per parameter.

Numeric format therefore acts as a direct multiplier on weight memory, which is what makes reduced precision such a powerful lever and why format support is a first-order hardware feature rather than a software detail.

Read this in full → Part 2 — What AI Actually Does to Memory

Activations

Intermediate results produced as data passes through the network. Some are transient; during training, forward-pass activations must be retained for the backward pass, which makes them a significant and batch-sensitive consumer of memory.

Read this in full → Part 2 — What AI Actually Does to Memory

Gradients

The error signals computed during the backward pass that indicate how each parameter should change. They exist only in training and are broadly proportional to parameter count.

Read this in full → Part 2 — What AI Actually Does to Memory

Optimizer State

Additional numerical state an optimiser maintains per parameter in order to compute updates. It is the least intuitive component and often the largest surprise in a training memory budget.

Microsoft’s original ZeRO analysis estimates roughly 16 bytes per parameter for a conventional mixed-precision Adam setup covering parameters, gradients and optimiser state together, before activation memory. Training is therefore state-heavy in a way inference never is. [P2:S9]

Read this in full → Part 2 — What AI Actually Does to Memory

KV Cache

The Key and Value tensors retained for previously processed tokens so that a transformer does not recompute all prior attention for every new token. It is the defining memory structure of inference at scale.

Its size is not set by model size alone. It grows with context length, layer count, attention design, numeric precision and — critically — the number of concurrent requests being served. A long-context, high-concurrency service and a short-prompt chatbot running the identical model occupy entirely different amounts of accelerator memory.

That variability is what makes KV cache an operational and economic problem rather than a fixed overhead. Capacity dedicated to cached context is capacity unavailable for weights or batch size, so serving economics become a question of how much context a provider can afford to keep resident.

It is also why memory management software became strategically important: allocation waste in a structure that changes size per request translates directly into wasted accelerator hours.

Read this in full → Part 2 — What AI Actually Does to Memory

Training vs. Inference Dynamics 3 sub-nodes

The same model stresses memory differently depending on what is being done with it, and the difference is not one of degree but of kind.

Training must hold model state, gradients and optimiser state simultaneously, plus retained activations, so it is dominated by the sheer volume of state. Inference discards gradients and optimiser state but introduces the problem of serving many users at once, where the binding resource is often the rate at which weights and cached context can be re-read.

This split is the mechanism that converts AI adoption into specific silicon demand. A world weighted toward training buys capacity to hold state; a world weighted toward large-scale serving buys bandwidth and proximity. Both are “AI memory demand” and they are not the same trade.

NVIDIA’s own framing of inference separates a compute-heavy prefill stage from a bandwidth-sensitive decode stage, which is the technical basis for treating the two phases as distinct workloads rather than one.

Read this in full → Part 2 — What AI Actually Does to Memory

Training: state-heavy

A training step runs a forward pass, computes error, runs a backward pass and updates parameters, holding weights, activations, gradients, optimiser state and communication buffers concurrently.

Because several of these scale with parameter count, training memory grows rapidly with model size and is typically solved by spreading state across many devices rather than by buying a larger single memory.

Read this in full → Part 2 — What AI Actually Does to Memory

Inference Phases 2 sub-nodes

Serving splits cleanly into two phases with opposite hardware profiles, which is why a single average utilisation figure describes neither well.

Understanding the split is what allows an analyst to predict whether growth in inference volume shows up as demand for arithmetic throughput or for memory bandwidth.

Read this in full → Part 2 — What AI Actually Does to Memory

Prefill

Processing the user’s prompt or context. Large matrix operations over many tokens at once make this phase typically compute-heavy.

Read this in full → Part 2 — What AI Actually Does to Memory

Decode

Generating tokens one at a time. Each step re-reads model parameters and the existing cached context while performing comparatively little arithmetic, so arithmetic intensity falls and memory bandwidth becomes the limiting resource.

Because most tokens a production service emits are decode tokens, this phase dominates the economics of large-scale serving.

Read this in full → Part 2 — What AI Actually Does to Memory

Arithmetic Intensity

The amount of computation performed per byte of data moved. It is the single number that decides whether a workload is limited by arithmetic or by delivery.

Roofline analysis formalises this: below a machine-specific threshold, adding compute changes nothing and only a wider or faster memory path helps. It is the bridge between the physics at the top of this map and the investment logic at the bottom.

Read this in full → Part 2 — What AI Actually Does to Memory

HBM and The Memory Wall 3 sub-nodes

This branch answers the question the rest of the hierarchy raises: why was a new, expensive class of DRAM necessary at all?

The answer is a divergence in growth rates. Across GPU generations, peak arithmetic throughput has risen faster than peak memory bandwidth. In one open-access comparison of peak dense FP16 tensor throughput against peak memory bandwidth, the endpoints move from 125 TFLOP/s and 0.9 TB/s to 2,250 TFLOP/s and 8 TB/s — roughly 18x against roughly 8.9x. These are theoretical peaks rather than achieved application performance, but the widening ratio is the point. [P2:S13]

HBM is the architectural response, and its significance for investors is that it is not a premium DRAM part. It is a co-designed subsystem spanning memory dies, logic and advanced packaging, which is why its economics pull in equipment, assembly and foundry players who never participated in commodity DRAM cycles.

Read this in full → Part 2 — What AI Actually Does to Memory

Memory Wall

The structural gap between how fast a processor can compute and how fast the memory system can deliver operands. It is a trend, not a state: each generation adds arithmetic faster than it adds delivery.

The practical consequence is diminishing returns on compute-only scaling, which is what forces architects to spend package area, power budget and cost on the memory subsystem instead.

Read this in full → Part 2 — What AI Actually Does to Memory

HBM Subsystem 4 sub-nodes

HBM does not make DRAM cells faster. It makes the connection between DRAM and compute dramatically wider and physically shorter, by stacking memory dies vertically and placing the stack immediately beside the accelerator on a shared package.

That reframing explains the cost. Building it requires known-good dies, wafer thinning, through-silicon via formation, precise bonding, an increasingly capable logic base die, advanced packaging and thermal management — and yields compound across those steps, so a defect anywhere in the stack discards good dies with it.

It also explains the supplier map. Value migrates from the DRAM cell alone toward deposition, planarisation and metrology equipment, bonding tools, interposer and packaging capacity. For an analyst, HBM is best read as a memory-logic-packaging system whose bottleneck can sit in any of the three.

Read this in full → Part 2 — What AI Actually Does to Memory

Stacked DRAM dies

Multiple thinned DRAM dies placed vertically rather than spread across a board. Stacking is what converts a fixed footprint beside the accelerator into multiples of capacity, and it is the step that introduces compound yield risk.

Read this in full → Part 2 — What AI Actually Does to Memory

Through-Silicon Vias (TSVs)

Vertical electrical connections punched through the silicon so signals travel between stacked dies over very short distances. They are what make a very wide interface physically possible, and their formation and alignment are among the most demanding process steps in the stack.

Read this in full → Part 2 — What AI Actually Does to Memory

Logic Base Die

The die beneath the DRAM stack handles interface and control. In HBM4, the base die has become more logic-intensive: Samsung describes its commercial HBM4 base die as built on a 4nm logic process. [P2:S17]

That shift matters because it pulls a foundry logic process into what used to be a pure memory product, blurring the line between memory makers and logic manufacturing.

Read this in full → Part 2 — What AI Actually Does to Memory

Advanced Interposer Packaging

The substrate that carries the thousands of short, dense connections between accelerator and memory stacks. Without it, a very wide interface cannot be routed at all.

Packaging capacity can therefore become a separate supply constraint on AI accelerators, even when DRAM or logic wafers are available.

Read this in full → Part 2 — What AI Actually Does to Memory

Roadmap

The generational path matters because it shows bandwidth being bought through three separate mechanisms: a wider interface, faster signalling, and tighter physical integration.

HBM4 widens the interface to 2,048 data signals against HBM3E’s 1,024. The JESD270-4 document itself is not publicly readable, so that figure is taken from JEDEC’s public announcement and corroborated independently by Micron and Samsung product statements. HBM4E continues the speed and capacity roadmap rather than doubling the interface again, and as of mid-2026 both Samsung and SK hynix describe it at sample stage. [P2:S17]

On the accelerator side the shipping reference points are H100 at 80GB and 3.35TB/s, H200 at 141GB and 4.8TB/s, and B200 at 180GB and up to 8TB/s. NVIDIA’s Rubin disclosure lists up to 288GB of HBM4 and up to 22TB/s, and NVIDIA stated on its 26 August 2026 earnings call that production shipments had begun. Those remain vendor-stated maxima rather than measured values for every configuration. [P2:S23]

Read together, capacity and bandwidth are scaling on separate tracks — which is precisely why they must be treated as separate demand drivers rather than one metric.

Read this in full → Part 2 — What AI Actually Does to Memory

Memory Tiering 4 sub-nodes

If HBM is so effective, why not put everything in it? Because it is scarce and expensive, and the scarcity is structural rather than cyclical — Micron has described HBM as consuming materially more silicon per bit than conventional DRAM.

So systems tier. Data is placed according to how urgently and how often compute needs it, and it is promoted or evicted as that urgency changes. The scarce resource is not memory in general but fast memory close to compute.

This turns memory management into a scheduling and economic problem: at any moment, which byte has earned a place in the most expensive tier? Software that answers that question well converts directly into higher accelerator utilisation.

Read this in full → Part 2 — What AI Actually Does to Memory

Hot Data — HBM

The immediate working set: active weights and active cached context. Highest bandwidth, highest cost per byte, smallest pool. Anything resident here is displacing something else.

Read this in full → Part 2 — What AI Actually Does to Memory

Warm Data — DDR

System memory used for staging, host-side state and overflow that does not need immediate accelerator access. Larger and cheaper per byte, but a tier removed from compute.

Read this in full → Part 2 — What AI Actually Does to Memory

Cold Data — SSD

Persistent bulk: datasets, checkpoints, model files and retrieval corpora. Lowest cost per byte and effectively unbounded capacity, at the cost of being furthest from compute.

Read this in full → Part 2 — What AI Actually Does to Memory

Data Movement (Promotion and Eviction)

Tiers are not static shelves. Data is promoted upward as it becomes active and evicted downward as it cools, and the traffic between tiers is itself a cost in time and energy.

Because every promotion consumes bandwidth that could have served compute, aggressive tiering can defeat itself. The quality of the placement policy, not the existence of tiers, is what determines whether the hierarchy helps.

Read this in full → Part 2 — What AI Actually Does to Memory

Emerging Interconnects and Tiers 3 sub-nodes

The established hierarchy has two visible gaps: capacity is locked to whichever host owns it, and the distance between expensive HBM and cheap NAND is very large. Three developments address those gaps from different directions.

None of them replaces a tier. CXL changes who owns capacity, direct storage paths shorten the route between persistent data and compute, and High Bandwidth Flash proposes an entirely new rung. Their common claim is that the hierarchy will gain structure rather than collapse.

For an analyst the useful discipline here is to separate what is standardised, what is commercially shipping, and what exists only as a specification target. The three children below sit at three different points on that scale.

Read this in full → Part 3 — Where the Hierarchy Is Going

CXL (Compute Express Link) 3 sub-nodes

A cache-coherent interconnect that lets processors, accelerators and memory devices share an addressable memory space over a standard link. Switching, pooling and capacity-on-demand entered the standard with CXL 2.0. [P3:S28]

Its promise is allocation, not locality. Memory reached across a switch is still physically farther from compute than a stack sitting on the same package, so CXL does not reproduce HBM’s bandwidth or latency and is not a substitute for it. What it changes is that capacity no longer has to be permanently assigned to one host and stranded there.

The gap worth watching is between standardised capability and scaled deployment. Public demonstrations of expansion, pooling and disaggregated configurations exist, and commercial switching silicon exists, but how much pooled memory is actually running across large production AI fleets is hard to establish from public sources.

That uncertainty is itself the investable question: CXL connectivity is a small, concentrated exposure today whose size depends entirely on whether pooling becomes default architecture or stays a specialist deployment.

Read this in full → Part 3 — Where the Hierarchy Is Going

Memory Expansion

Adding addressable capacity to a single host beyond what its native memory channels allow. The simplest use of the standard and the least architecturally disruptive.

Read this in full → Part 3 — Where the Hierarchy Is Going

Memory Pooling

A shared pool that multiple hosts can draw from, so capacity idle on one machine can serve another. This is where the economic argument lives: stranded memory is paid for once and used partially.

Read this in full → Part 3 — Where the Hierarchy Is Going

Composable Capacity

The end state in which memory is allocated to workloads on demand rather than being fixed to a chassis at purchase. Demonstrated publicly; not yet an established default.

Read this in full → Part 3 — Where the Hierarchy Is Going

GPUDirect Storage

A direct transfer path between storage and accelerator memory, removing the host-memory bounce buffer from the payload route.

Two qualifications keep this honest. The payload still moves — nothing is free, and the bytes still cross a physical link. And the CPU still participates in control, even when it is no longer staging the data. What changes is the number of copies, not the existence of the transfer.

The reason it matters commercially is utilisation: at AI scale, storage throughput can decide how well extremely expensive accelerators stay busy, which is why storage performance now appears in AI benchmark suites alongside training throughput.

Read this in full → Part 3 — Where the Hierarchy Is Going

High Bandwidth Flash (HBF)

A proposed NAND-based tier sitting between HBM and conventional SSDs, aimed at the economic gap between very fast, very expensive memory and very cheap, comparatively slow storage.

Vendor-announced targets describe capacities up to 512 GB with bandwidth grades of roughly 0.4 to 3.0 TB/s over a UCIe interface. These are specification targets, not verified production performance. [P3:S41]

The verification limit should be stated plainly: the specification sits inside an Open Compute Project semi-private workstream and is not publicly readable, so the figures rest on the two proposing vendors’ announcements. HBF belongs on this map as evidence that the industry believes the hierarchy needs another rung — not as an established tier.

Read this in full → Part 3 — Where the Hierarchy Is Going

Scaling and Efficiency Levers 2 sub-nodes

Demand for memory is not set by hardware alone. Two very different levers change how much physical memory a given amount of useful work requires: making each device denser, and making software use a fixed device better.

They behave differently as investment signals. Density scaling is capital-intensive, supplier-visible and shows up as equipment and wafer demand. Software efficiency can improve utilization without installing more physical memory, but it can carry engineering, compute, communication or quality costs.

Treating only the first as “real” is the classic error in memory forecasting. Treating the second as a reason demand collapses is the opposite error.

Read this in full → Part 3 — Where the Hierarchy Is Going

NAND Density 2 sub-nodes

Density rises through two independent levers that can be pulled together or separately, which is why NAND has kept scaling long after planar shrink stopped being the mechanism.

The common cost of both levers is management: as cells hold more states and stacks grow taller, controller design, firmware and error correction absorb a growing share of the engineering and of the value of the finished drive.

Read this in full → Part 3 — Where the Hierarchy Is Going

Vertical Layer Count

Building upward: more memory layers in the same footprint. Recent public examples describe 332-layer devices at sampling stage rather than broad volume production. [P3:S33]

Read this in full → Part 3 — Where the Hierarchy Is Going

Bits per Cell (SLC, MLC, TLC, QLC)

Encoding more information per cell — one, two, three or four bits. Each additional bit raises density and lowers cost per bit at the same layer count.

The price is narrower voltage margins, and therefore lower endurance and more demanding management. This is why QLC expands the addressable capacity market without simply replacing TLC.

Read this in full → Part 3 — Where the Hierarchy Is Going

Software Optimization 4 sub-nodes

The physical memory block does not change size. What software changes is how much useful work fits inside it and how often data has to move.

Each of the four levers below buys something and pays for it somewhere else, and stating both halves is what separates analysis from marketing. None of them removes the underlying bandwidth and capacity constraints; they shift where the constraint binds.

The structural point for investors is that this is the one constraint in the whole map with no clean one-to-one listed-equity exposure. FlashAttention, vLLM and DeepSpeed originate substantially in research and open-source work. Efficiency can reduce hardware required per workload while also enabling more workloads, so its aggregate effect on hardware demand is not one-directional.

Read this in full → Part 3 — Where the Hierarchy Is Going

FlashAttention

Reformulates attention as an I/O problem, restructuring the computation to reduce reads and writes between HBM and on-chip SRAM rather than to reduce arithmetic.

Benefit: fewer transfers for the same exact result. Cost: hardware-aware kernels and implementation complexity that must be re-tuned as architectures change.

Read this in full → Part 3 — Where the Hierarchy Is Going

PagedAttention

Manages the KV cache in fixed blocks, analogous to virtual-memory pages, so that a structure which grows and shrinks per request does not fragment the memory around it.

Benefit: far less waste and over-reservation, which raises serving throughput on unchanged hardware. Cost: block management and additional serving-layer logic.

Read this in full → Part 3 — Where the Hierarchy Is Going

ZeRO

Partitions optimiser state, gradients and eventually parameters across devices instead of replicating them on every one, directly attacking the state-heavy nature of training.

Benefit: models that would not fit become trainable. Cost: communication traffic and distributed coordination — the constraint moves from memory to interconnect.

Read this in full → Part 3 — Where the Hierarchy Is Going

Quantization and KV compression

Reducing numeric precision shrinks weights and sometimes cached state, making it the most direct lever on memory footprint.

The counterpoint deserves equal weight: the effect of compression is workload- and model-dependent, and aggressive KV-cache compression can degrade capabilities such as reasoning on some tasks while leaving others intact. Efficiency here is a trade, not a free gain.

Read this in full → Part 3 — Where the Hierarchy Is Going

Investment Value Chain 7 sub-nodes

This is the only branch that names companies, and it does one job: it turns the technical map above into a question about exposure. The sequence is constraint, then supplying layer, then economics — never the reverse.

The reason the sequence matters is that “AI memory demand is rising” is not an investable statement. Rising demand for stored bits, rising demand for bytes per second, a shortage of packaging capacity and a shortage of DRAM wafers are four different conditions that pay four different sets of suppliers. Identifying which constraint is actually binding is the whole analytical exercise.

Two honest limits apply to everything below. First, this is an illustrative exposure map — not a recommendation, not exhaustive, and subject to change. Company names appear only as examples of product exposure to a named constraint, listed alphabetically within each layer, with no ordering that implies preference and no view on valuation.

Second, it ages faster than the rest of this map. Portfolios, packaging relationships and commercial exposure change even when the underlying hierarchy does not, so this branch should be re-checked whenever the series is revised. As of 14 September 2026.

Read this in full → Part 3 — Where the Hierarchy Is Going

Memory Manufacturers

Constraint served: HBM capacity and bandwidth, and DRAM supply generally.

Micron, Samsung and SK hynix each manufacture HBM alongside conventional DRAM families. Their exposure is unusual in that HBM ties a memory product to logic process and advanced packaging execution, so an operational problem in stacking or base-die supply can bind before cell capacity does.

Read this in full → Part 3 — Where the Hierarchy Is Going

NAND / Storage Suppliers

Constraint served: persistent capacity and the storage side of the AI data path.

Kioxia and SanDisk supply NAND flash and data-centre SSDs. This layer responds to dataset size, checkpoint volume, embeddings and retrieval corpora rather than to accelerator bandwidth, which is why it can move on a different rhythm from HBM.

Read this in full → Part 3 — Where the Hierarchy Is Going

Equipment Suppliers

Constraint served: DRAM wafer capacity and process intensity.

Applied Materials and ASML sell into the fabrication of advanced DRAM and, in Applied’s case, into advanced packaging as well. Exposure here is driven by capacity investment decisions rather than by bit shipments, so it leads or lags the memory makers depending on where the cycle sits.

Read this in full → Part 3 — Where the Hierarchy Is Going

Packaging and Logic

Constraint served: stacking yield and advanced packaging capacity.

Amkor, Applied Materials and TSMC participate in assembly, process equipment and heterogeneous integration. Advanced packaging becomes far more central to HBM economics than it is to conventional commodity DRAM modules.

Read this in full → Part 3 — Where the Hierarchy Is Going

Compute Integration

Constraint served: how much memory an accelerator can host and how tightly it is integrated.

Broadcom and NVIDIA design compute architectures whose memory subsystems are a defining specification. Their economics depend partly on keeping expensive arithmetic hardware supplied, which is why accelerator roadmaps now advertise memory capacity and bandwidth as headline features.

Read this in full → Part 3 — Where the Hierarchy Is Going

Connectivity

Constraint served: CXL switching and memory connectivity.

Marvell supplies commercial CXL switching silicon. This is a relatively concentrated connectivity exposure today, and its eventual size depends on how broadly pooled memory is deployed.

Read this in full → Part 3 — Where the Hierarchy Is Going

Software Efficiency — no clean listed exposure

Constraint served: nothing, in the supplier sense — and that is the finding.

Every other row on this map says a constraint binds, therefore spending flows to a layer. Software efficiency works the other way: it reduces the physical memory required for a given amount of useful work. Much of the progress originates in research and open-source projects such as FlashAttention, vLLM and DeepSpeed, so there is no clean one-to-one listed-equity exposure comparable with HBM manufacturing or NAND capacity.

Naming a single public company here would overstate the evidence. The honest reading is that this constraint can reduce hardware needed per workload while also enabling more total workloads; there is no clean one-to-one revenue exposure.

Read this in full → Part 3 — Where the Hierarchy Is Going

Source note: quantitative specifications and dated claims follow the primary-source registers in the three linked research parts; node links point back to the relevant sourced sections.

Illustrative exposure map — not a recommendation, not exhaustive, and subject to change. The Investment Value Chain branch names companies only as examples of product exposure to a named constraint, listed alphabetically within each layer, with no ordering that implies preference and no view on valuation.

MemoryCapex provides independent research and analysis for informational and educational purposes only. Nothing here constitutes investment advice, an investment recommendation, or an offer or solicitation to buy or sell any security.

The author holds, directly or indirectly, positions in substantially all of the publicly traded companies mentioned in this publication or series. Those positions may change without notice.

Where the Hierarchy Is Going, and Who Gets Paid

Series: How AI Memory Actually Works: A Step-by-Step Guide from SRAM to HBM4, NAND and Beyond

Previously in this series

The memory hierarchy exists because no single technology can maximize speed, capacity, persistence, energy efficiency and low cost at the same time. AI training is heavy in model state, while large-language-model decoding often becomes constrained by memory bandwidth. HBM is therefore best understood not as premium commodity DRAM, but as a co-designed memory, logic and advanced-packaging subsystem.

Memory Hierarchy- Part 1

Memory Hierarchy- Part 2

Abstract

The memory hierarchy is no longer a fixed ladder. New interconnects can make capacity composable, storage is moving closer to the active AI data path, NAND continues to scale in two dimensions, and software can change how efficiently every physical byte is used. At the same time, emerging proposals such as High Bandwidth Flash attempt to fill the large economic gap between expensive near-compute memory and persistent flash storage. This final part examines those transitions without assuming that every new technology becomes a large market. It then converts the technical hierarchy into an investment-research framework: first identify the constraint that is actually binding, then determine which part of the semiconductor value chain is directly exposed to relieving that constraint.

Estimated reading time: 15–17 minutes

Revision History

• Version 1.0 — September 14, 2026: Initial publication.

Step 11 — CXL: Making Memory Capacity Composable

THE QUESTION — Can a computing system use memory capacity as a shared resource instead of permanently attaching all of it to one processor?

THE INTUITION

Consider a company choosing between private offices and a coworking floor. A private office is permanently assigned to one team and is immediately available to that team. A shared floor can be reassigned as demand changes, so the same total space may serve more variable workloads. The trade-off is that shared space is less local and requires a system for allocation and access.

THE MECHANISM

COMPUTE EXPRESS LINK (CXL) is an industry-standard interconnect designed to let processors, accelerators and memory devices communicate with coherent memory semantics. In practical terms, it extends the idea of memory beyond DRAM soldered or socketed directly beside one processor.

The simplest CXL use case is MEMORY EXPANSION. A host that cannot economically or physically attach enough local DRAM can access additional memory through a CXL device. The next step is MEMORY POOLING: instead of dedicating every memory device permanently to one host, multiple systems can draw capacity from a shared pool. CXL SWITCHING provides the fabric that connects several hosts and devices. At the architectural limit, DISAGGREGATION separates memory capacity from a specific compute node so software can allocate resources more dynamically.

These concepts are standardized, but standardization should not be confused with deployment at scale. CXL 2.0 formally introduced switching and memory pooling in December 2020.^[S28] The current CXL ecosystem has progressed through later specification generations, and the CXL Consortium was publicly demonstrating memory expansion, pooling and disaggregated-memory configurations at FMS 2026.^[S27][S29] Commercial components also exist: Marvell stated in March 2026 that one Structera CXL switch was in production while a newer generation was scheduled to sample later in the year.^[S30] None of those facts, by themselves, establishes that pooled CXL memory is already deployed broadly across hyperscale AI fleets.

CXL also does not turn remote capacity into HBM. HBM sits inside the accelerator package, connected through an extremely wide local interface. CXL-attached memory is farther away and must traverse an external link, and potentially a switch or fabric. The technologies therefore address different constraints. HBM optimizes the immediate supply of high-bandwidth data to compute. CXL primarily makes larger memory capacity more flexible, shareable and composable.

WHY IT MATTERS FINANCIALLY

CXL introduces a distinct connectivity layer between processors and expandable or pooled memory. That creates exposure for memory suppliers, server architects and connectivity vendors such as Marvell, but the economic value depends on which deployments move beyond expansion into genuine pooling and disaggregation.

The important distinction is that CXL can improve capacity utilization without removing the separate market for extremely high-bandwidth local memory.

ONE-LINE TAKEAWAY

CXL can make memory capacity more flexible, but it does not reproduce the locality or bandwidth of HBM.

Step 12 — Storage Has Entered the AI Data Path

THE QUESTION — When does persistent storage stop behaving like a distant archive and become part of accelerator performance?

THE INTUITION

Think of an airport baggage system. A bag may traditionally pass through several staging areas before reaching the aircraft. If a redesigned route removes an intermediate handoff, the bag can reach its destination with fewer copies and less handling. But the bag still has to travel from storage to the aircraft—the route has improved, not disappeared.

THE MECHANISM

AI systems create large amounts of data that cannot remain permanently in DRAM or HBM. DATASETS contain the material used for training. CHECKPOINTS preserve the state of a training run so computation can resume after interruption. Model files store trained parameters. EMBEDDINGS represent information numerically so systems can compare or retrieve related items. VECTOR STORES organize large collections of those representations, often supporting RETRIEVAL-AUGMENTED GENERATION (RAG). Logs and operational data add another persistent stream.

All of that information eventually has to cross the boundary between storage and active memory.

In a conventional accelerator I/O path, data read from an SSD typically enters host memory before being copied into GPU memory. That adds movement through CPU-managed memory. NVIDIA’s GPUDirect Storage architecture creates a direct DMA data path between storage and GPU memory, avoiding the requirement to use host memory as a bounce buffer for the payload.^[S31] This does not mean the CPU, operating system or file system vanish. The important change is the payload path: fewer memory copies are required before data reaches the accelerator.

The broader performance problem is not unique to NVIDIA’s implementation. MLCommons’ MLPerf Storage v3.0 benchmark, released September 1, 2026, is explicitly designed to measure how well storage systems keep AI accelerators supplied across workloads including training, checkpointing, vector databases and KV-cache storage.^[S52] Its methodology states the underlying issue plainly: when storage cannot deliver training samples as quickly as accelerators consume them, accelerator time becomes idle time.

This changes the economic meaning of storage performance. A storage system is no longer evaluated only by how many petabytes it can retain. Depending on the workload, sustained bandwidth, small-file access, checkpoint write performance, tail latency and parallelism can determine whether expensive compute spends its time calculating or waiting.

That does not make SSDs equivalent to working memory. NAND remains much slower than HBM or DRAM. It means that as compute becomes more expensive and faster, the cost of feeding that compute reaches progressively farther down the hierarchy.

WHY IT MATTERS FINANCIALLY

Storage performance can affect the utilization of accelerators whose cost is far higher than the storage feeding them. That makes enterprise SSD and NAND economics relevant not only to capacity growth but also to data-pipeline performance for suppliers such as Kioxia and SanDisk.

It also creates system-level opportunities in controllers, networking and software, because reducing copies is useful only if the rest of the storage path can sustain the required throughput and latency.

ONE-LINE TAKEAWAY

Storage increasingly affects accelerator utilisation because inactive data eventually has to become active data.

Step 13 — How NAND Gets Denser: Layers × Bits per Cell

THE QUESTION — How can NAND keep storing more information without simply making every memory chip proportionally larger?

THE INTUITION

Imagine an apartment developer with two independent ways to increase the number of residents on the same plot of land. The building can grow taller by adding floors, or each floor can be divided into more apartments. Both increase density, but subdividing each floor more aggressively also makes access, management and error tolerance harder.

THE MECHANISM

Modern NAND density grows through two conceptually separate levers.

The first is VERTICAL LAYER COUNT. Three-dimensional NAND stacks memory structures vertically rather than relying only on two-dimensional scaling across the wafer surface. Adding more usable layers can increase the number of memory cells available within a given footprint. This is architectural density: more physical storage structures are placed above one another.

The second lever is BITS PER CELL. A NAND cell does not have to represent only one binary bit. SINGLE-LEVEL CELL (SLC) stores one bit per cell. MULTI-LEVEL CELL (MLC) stores two. TRIPLE-LEVEL CELL (TLC) stores three. QUAD-LEVEL CELL (QLC) stores four. Micron’s NAND documentation describes this progression directly and notes the corresponding trade-off: increasing bits per cell raises density while generally reducing performance or endurance margins.^[S32]

The two levers are independent. A manufacturer can increase layer count while keeping TLC, move from TLC to QLC at a similar layer count, or do both.

Current product development illustrates that distinction. SanDisk announced sampling of a 332-layer BiCS10 1Tb TLC NAND device on July 2, 2026 and stated an interface speed of up to 4.8 Gb/s.^[S33] On August 4, 2026, Kioxia and SanDisk disclosed a 332-layer QLC technology with vendor-reported bit density above 37 Gb/mm² and a 4.8 Gb/s NAND interface in testing.^[S34] The first is a sampling announcement; the second is a technology disclosure. Neither should be read as proof that all future NAND economics will follow the same path.

Higher density also shifts work into the rest of the storage system. As more information is encoded per cell and flash characteristics become more demanding, the controller and firmware must manage error correction, bad blocks and wear. Micron explicitly lists ERROR-CORRECTION CODE (ECC), bad-block management and wear leveling among the functions handled by NAND controllers and firmware.^[S32]

This is why an SSD is not simply a package of raw NAND. As physical density rises, the value of controller architecture, firmware and error management becomes increasingly intertwined with the underlying flash.

WHY IT MATTERS FINANCIALLY

Layer growth and bits-per-cell growth affect NAND economics differently. Kioxia and SanDisk can gain bit density through both physical architecture and cell encoding, while higher management requirements also increase the importance of SSD controllers, firmware and system qualification.

For capital analysis, more layers can change process-step and equipment intensity, while more bits per cell can change usable capacity, endurance segmentation and the mix of workloads a NAND product can economically serve.

ONE-LINE TAKEAWAY

NAND density scales through two separate levers: more cells vertically and more bits inside each cell.

Step 14 — Software Can Change Physical Memory Demand

THE QUESTION — Can software make a fixed amount of physical memory behave as if the system had more of it?

THE INTUITION

Imagine a fixed-size workbench. Buying a larger bench is one way to increase capacity. But better tool racks, jigs and work sequencing can also let the same surface support more useful work. The bench has not become physically larger; its space is simply being used more efficiently, sometimes at the cost of extra setup, precision requirements or complexity.

THE MECHANISM

Hardware demand is partly determined by software because software decides what data must exist, where it resides and how often it moves.

FlashAttention is a useful example. Standard attention implementations can repeatedly move large intermediate tensors between HBM and much smaller on-chip SRAM. The original FlashAttention paper reformulated attention around this input/output cost, using tiling to reduce transfers between those memory levels while preserving exact attention results (Dao et al., May 27, 2022).^[S35] The benefit is not extra physical HBM. It is fewer expensive data movements for the same computation.

PagedAttention, implemented in vLLM, attacks a different inefficiency. KV CACHE sizes change dynamically as requests arrive, grow and finish. Reserving large contiguous memory regions can waste capacity through fragmentation. PagedAttention divides KV-cache storage into blocks that can be allocated non-contiguously, borrowing the logic of virtual-memory paging (Kwon et al., September 12, 2023).^[S36]

ZeRO addresses training rather than serving. Instead of replicating all parameters, gradients and optimizer states across every device, ZeRO partitions increasingly large categories of model state across participating accelerators (Rajbhandari et al., October 4, 2019).^[S37] The memory saved locally is exchanged for more distributed coordination and communication.

QUANTIZATION reduces the number of bits used to represent selected values. NVIDIA’s TensorRT documentation supports multiple reduced-precision formats and also describes scaling and implementation requirements that accompany quantized execution.^[S38] Lower precision can reduce weight or activation footprint and memory traffic, but compatibility, calibration, accuracy and hardware execution characteristics still matter.

KV-CACHE COMPRESSION pushes the same idea further by retaining less information, representing it more compactly or selectively discarding cached state. The counterpoint is important. An open-access preprint published February 4, 2025 found that the impact of KV-cache compression depends strongly on the task and compression method, with aggressive compression capable of degrading model abilities.^[S39]

Software efficiency is therefore neither irrelevant to hardware demand nor a free substitute for hardware. It changes the amount of useful work extracted from each byte while moving cost into computation, communication, engineering complexity or model-quality trade-offs.

WHY IT MATTERS FINANCIALLY

Software changes the relationship between model growth and physical memory demand, so hardware forecasts cannot simply extrapolate bytes from parameter counts. At the same time, higher efficiency can increase utilisation and lower the cost of serving each workload, potentially enabling more total usage rather than mechanically reducing aggregate infrastructure demand.

The relevant research question is therefore not whether software “replaces” memory, but which memory constraint it relaxes and what new compute, communication or quality trade-off it introduces.

ONE-LINE TAKEAWAY

Software can extract more work from fixed memory, but every efficiency technique changes another part of the trade-off.

Step 15 — Beyond Today’s Hierarchy: Emerging Memory Tiers

THE QUESTION — Is the economic gap between HBM and NAND large enough to support another memory tier?

THE INTUITION

Freight networks already have air cargo and ocean shipping. Air is fast but expensive; ocean freight moves enormous volumes cheaply but slowly. A new transport mode matters only if enough cargo needs a combination of speed, capacity and cost that neither existing option serves economically.

THE MECHANISM

The current hierarchy contains a large gap. HBM offers extremely high local bandwidth but uses expensive DRAM dies, complex stacking and advanced packaging. NAND provides vastly cheaper persistent capacity but at much lower access performance. An intermediate tier would need to offer enough additional bandwidth over conventional storage, enough additional capacity or lower cost than HBM, acceptable power, manageable latency and a practical processor interface.

HIGH BANDWIDTH FLASH (HBF) is an emerging proposal aimed at that space. It uses NAND rather than DRAM and is being positioned as a high-capacity, high-bandwidth tier that could sit closer to processors than conventional SSD storage.

The maturity status matters. The Open Compute Project publicly lists High Bandwidth Flash as a storage project inside a semi-private workstream. The HBF specification itself is therefore not publicly readable through a normal public OCP specification page in our verification as of September 14, 2026.^[S53] The capacity and bandwidth figures below are consequently vendor-announced targets from Sandisk and SKhynix, not values independently verified from publicly readable OCP specification text.

Sandisk and SKhynix announced on August 3–4, 2026 that they had released an initial HBF technical specification through the OCP workstream.^[S40][S41] SKhynix’s announcement describes specification targets covering up to 512 GB of capacity across 8-high and 16-high NAND configurations and bandwidth grades of approximately 0.4 TB/s to 3.0 TB/s. It also states that the proposed processor connection uses UCIe.^[S41] These are specification targets, not evidence of volume production, customer qualification or achieved shipping-system performance.

The distinction matters because a memory technology can be technically credible without becoming an economically meaningful tier. HBF would still need a broader ecosystem: controllers, processor support, packaging, firmware, operating-system integration, reliability qualification and applications willing to manage another level of memory placement.

It should therefore be treated as an architectural hypothesis under standardization, not as an established category comparable today with HBM, DDR or enterprise SSDs.

WHY IT MATTERS FINANCIALLY

An additional tier could redistribute value across NAND, controllers, packaging, interconnects and system software rather than simply replacing an existing product. Sandisk and SKhynix are currently the principal public proponents cited here, but specification leadership does not establish future market share or commercial adoption.

The investment-research question is therefore whether an intermediate tier becomes necessary enough to support a durable ecosystem, not whether an attractive technical specification can be demonstrated.

ONE-LINE TAKEAWAY

New memory tiers exist only if they fill an economic gap that established technologies cannot.

Step 16 — The Investor’s Mental Model

THE QUESTION — When AI memory demand rises, how do we determine which part of the semiconductor value chain is actually being paid?

THE INTUITION

Consider a construction project that has stopped. The cause might be a missing transformer, an unavailable crane, insufficient concrete, too few electricians or an unapproved permit. “Construction demand is strong” tells us almost nothing about which supplier benefits from removing the bottleneck. The first analytical task is to identify what is actually preventing the next unit of output.

THE MECHANISM

The same discipline applies to memory. “AI needs more memory” is too broad to be an investment thesis. The useful question is: which constraint is binding?

If the constraint is HBM capacity or HBM bandwidth, the most direct product exposure sits with HBM and DRAM manufacturers such as Micron, Samsung and SKhynix.^{[S42][S43][S44]} But if the limiting factor is DRAM wafer capacity, the analysis expands to the equipment required to create that capacity, including lithography and process equipment supplied by Applied Materials and ASML.^[S48][S49]

If the bottleneck shifts to stacking yield or advanced packaging, the relevant layer changes again. Amkor provides outsourced advanced packaging and test; Applied Materials supplies process and metrology equipment used in DRAM and advanced packaging; TSMC provides CoWoS integration for logic and HBM.^{[S51][S49][S50]}

If the binding requirement is persistent NAND capacity or SSD throughput, the relevant product layer includes Kioxia and SanDisk.^[S45][S54] If capacity needs to become composable through CXL connectivity, Marvell is an example of a vendor supplying CXL switching and memory-connectivity products.^[S30]

Compute architecture can also change how much value the memory subsystem creates. Broadcom supplies custom AI compute platforms with HBM integration, while NVIDIA integrates HBM into its accelerator architectures.^[S47][S46] These are examples of exposure, not assessments of competitive quality.

Software efficiency is different from the other rows in this framework because there is no clean one-to-one listed-equity exposure. Important gains can originate in research and open-source systems such as FlashAttention, vLLM/PagedAttention and DeepSpeed/ZeRO rather than in a discrete hardware product purchased from one public supplier.^{[S35][S36][S37]} Better software can reduce bytes moved or stored, improve utilisation, enable larger workloads or shift the bottleneck elsewhere. Treating that effect as a direct spending stream to one listed company would therefore misstate the exposure.

The framework is sequential: identify the constrained resource, identify the layer that supplies or relieves it, then examine the economics of that layer. Starting with company names and working backward reverses the analytical process.

WHY IT MATTERS FINANCIALLY

Revenue growth in one layer does not automatically imply equivalent economics elsewhere in the hierarchy. Different constraints alter capital intensity, yield sensitivity, pricing power, utilization and the timing with which demand reaches equipment, memory, packaging, connectivity or storage suppliers.

This framework is intended to identify exposure, not attractiveness. Valuation, competitive durability and cycle position remain separate analytical questions.

ONE-LINE TAKEAWAY

First identify the binding constraint; only then map which part of the semiconductor value chain is exposed.

What I Am Least Confident About

• CXL switching, pooling and disaggregation are standardized capabilities with real commercial components and public demonstrations, but the degree to which pooled memory is already deployed across large production AI fleets is much harder to establish from public sources alone. The article therefore distinguishes standardized capability from demonstrated or scaled deployment.

• The High Bandwidth Flash specification sits inside an Open Compute Project semi-private workstream and is not publicly readable in our verification. HBF capacity and bandwidth figures in this article therefore come from Sandisk and SKhynix vendor announcements and should be treated as specification targets, not independently verified production performance.^{[S40][S41][S53]}

• The 2026 NAND examples in Step 13 represent different maturity states: SanDisk’s BiCS10 TLC disclosure describes sampling, while the Kioxia/SanDisk QLC disclosure describes a technology demonstration. Neither establishes that the quoted specifications characterize broad volume production.^[S33][S34]

• The effects of quantization and KV-cache compression remain highly workload- and model-dependent. A technique that preserves quality for one task, context length or architecture may not preserve it under another, particularly at aggressive compression levels.^[S38][S39]

• Software efficiency has no clean one-to-one listed-equity exposure comparable with HBM manufacturing or NAND capacity. Important advances such as FlashAttention, vLLM/PagedAttention and DeepSpeed/ZeRO originate substantially in research and open-source software, so mapping software efficiency directly to one public company would overstate the evidence.^{[S35][S36][S37]}

• Visual 16 will age faster than the rest of the series because company product portfolios, packaging relationships and commercial exposure can change even when the underlying memory hierarchy does not. It should therefore be reviewed whenever Part 3 is revised rather than treated as a permanent industry map.

Sources

• [S27] Compute Express Link Consortium. About CXL®. Undated; accessed September 14, 2026. Used for the definition of Compute Express Link and the current standards framework. Rights: quote-with-attribution only. Official source.

• [S28] Compute Express Link Consortium. Compute Express Link 2.0 Specification Now Available. December 2, 2020. Used for the introduction of switching, memory pooling and related CXL 2.0 capabilities. Rights: quote-with-attribution only. Official source.

• [S29] Compute Express Link Consortium. FMS 2026 CXL Demos. 2026; accessed September 14, 2026. Used for public demonstrations of memory expansion, pooling and disaggregated-memory configurations. Rights: quote-with-attribution only. Official source.

• [S30] Marvell Technology. Marvell Launches Next-generation CXL Switch, Enabling Memory Pooling to Break the AI Memory Wall. March 17, 2026. Used for commercial CXL switching and memory-pooling product exposure. Rights: quote-with-attribution only. Official source.

• [S31] NVIDIA. GPUDirect Storage Overview Guide. Current documentation; accessed September 14, 2026. Used for the conventional host-memory bounce-buffer path and direct DMA payload path between storage and GPU memory. Rights: quote-with-attribution only. Official source.

• [S32] Micron Technology. Choosing the Right NAND. Current documentation; accessed September 14, 2026. Used for SLC, MLC, TLC and QLC definitions, bits-per-cell trade-offs and controller or firmware functions. Rights: quote-with-attribution only. Official source.

• [S33] Sandisk. Sandisk Announces Sampling of BiCS10 1Tb TLC 3D NAND Flash Memory. July 2, 2026. Used for the 332-layer TLC sampling example and vendor-stated interface specifications. Rights: quote-with-attribution only. Official source.

• [S34] KIOXIA Corporation and Sandisk. New 3D Flash Memory Technology from KIOXIA and Sandisk Achieves Industry-Leading Bit Density with QLC NAND. August 4, 2026. Used for the 332-layer QLC technology example, vendor-reported density and interface test specifications. Rights: quote-with-attribution only. Official source.

• [S35] Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra and Christopher Ré. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. arXiv, May 27, 2022. Used for the I/O-aware reformulation of attention and reduction of HBM-to-SRAM data movement. Rights: quote-with-attribution only; open-access paper. Stable open-access source.

• [S36] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang and Ion Stoica. Efficient Memory Management for Large Language Model Serving with PagedAttention. September 12, 2023; presented at SOSP 2023. Used for PagedAttention, KV-cache fragmentation and vLLM memory management. Rights: quote-with-attribution only; open-access paper. Stable open-access source.

• [S37] Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase and Yuxiong He. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models. October 4, 2019; revised May 13, 2020. Used for partitioning parameters, gradients and optimizer state across devices. Rights: quote-with-attribution only; open-access paper. Stable open-access source.

• [S38] NVIDIA. Working with Quantized Types — NVIDIA TensorRT. Current documentation; accessed September 14, 2026. Used for supported reduced-precision formats and implementation requirements for quantized execution. Rights: quote-with-attribution only. Official source.

• [S39] Xiang Liu et al. Can LLMs Maintain Fundamental Abilities under KV Cache Compression? arXiv, February 4, 2025. Used for the task-dependent quality trade-offs associated with KV-cache compression. Rights: quote-with-attribution only; open-access preprint. Stable open-access source.

• [S40] Sandisk and SK hynix. Sandisk and SK hynix Advance Global Standardization of High Bandwidth Flash. August 3, 2026. Used for the announcement of the initial High Bandwidth Flash technical specification within the OCP workstream. Rights: quote-with-attribution only. Official source.

• [S41] SK hynix. SK hynix Unveils First HBF Standard Specifications with Sandisk. August 4, 2026. Used for vendor-announced HBF specification targets including capacity, bandwidth grades and UCIe connectivity. Rights: quote-with-attribution only. Official source.

• [S42] Micron Technology. HBM4. Current product page; accessed September 14, 2026. Used for Micron’s HBM and DRAM product exposure in the value-chain map. Rights: quote-with-attribution only. Official source.

• [S43] Samsung Semiconductor. HBM4. Current product page; accessed September 14, 2026. Used for Samsung’s HBM and DRAM exposure in the value-chain map. Rights: quote-with-attribution only. Official source.

• [S44] SK hynix. SK hynix Ships Samples of 12-Layer Next-Gen HBM4E. June 18, 2026. Used for SK hynix’s HBM and DRAM exposure in the value-chain map. Rights: quote-with-attribution only. Official source.

• [S45] KIOXIA. Business Products — Memory and SSD. Current product and business page; accessed September 14, 2026. Used for Kioxia’s NAND flash and enterprise or data-center SSD exposure in the value-chain map. Rights: quote-with-attribution only. Official source.

• [S46] NVIDIA. Inside NVIDIA Rubin GPU Architecture: Powering the Era of Agentic AI. July 21, 2026. Used for accelerator architecture and HBM integration exposure. Rights: quote-with-attribution only. Official source.

• [S47] Broadcom. Broadcom Ships 3.5D Face-to-Face Compute SoC, Powering AI Revolution. February 26, 2026. Used for custom AI compute, HBM integration and advanced system-in-package exposure. Rights: quote-with-attribution only. Official source.

• [S48] ASML. 2025 Annual Report. Published February 25, 2026. Used for lithography exposure to advanced DRAM and memory-capacity investment. Rights: quote-with-attribution only. Official source.

• [S49] Applied Materials. Applied Materials Introduces New Systems to Accelerate DRAM and Advanced Packaging for AI Chips. June 25, 2026. Used for DRAM process-equipment and advanced-packaging equipment exposure. Rights: quote-with-attribution only. Official source.

• [S50] Taiwan Semiconductor Manufacturing Company. CoWoS®. Current technology page; accessed September 14, 2026. Used for 2.5D integration and advanced packaging of logic with HBM. Rights: quote-with-attribution only. Official source.

• [S51] Amkor Technology. Amkor Technology Announces Strategic Partnership with NVIDIA to Expand Advanced Packaging and Test. July 23, 2026. Used for outsourced advanced-packaging, heterogeneous-integration and test exposure. Rights: quote-with-attribution only. Official source.

• [S52] MLCommons. MLCommons Releases New MLPerf Storage v3.0 Benchmark Results. September 1, 2026. Used as an independent source for the role of storage performance in keeping AI accelerators utilized and for the inclusion of training, checkpointing, vector-database and KV-cache storage workloads in MLPerf Storage v3.0. Rights: quote-with-attribution only. Official source.

• [S53] Open Compute Project Foundation. Semi-Private Workstreams. Undated; accessed September 14, 2026. Used to establish that High Bandwidth Flash is listed as an OCP Storage semi-private workstream and that the specification itself is not publicly readable through a normal public specification page. Rights: quote-with-attribution only. Official source.

• [S54] Sandisk. Discover a Full Line of Flash Memory Storage Products. Current product-portfolio page; accessed September 14, 2026. Used for Sandisk’s NAND-flash and data-center SSD product exposure in the value-chain map. Rights: quote-with-attribution only. Official source.

No subscription research, paid market data or licensed market-research datasets are used in Part 3. Public accessibility does not imply republication permission; unless explicitly stated otherwise, the sources above are treated as quote-with-attribution only.

Where This Series Lands

AI memory is not one market or one technology; it is a hierarchy built from different compromises in speed, capacity, persistence, power and cost. The hierarchy changes when compute, software or workload behavior changes which constraint matters most. That means technical understanding must come before mapping demand to companies or financial outcomes. Part 1 — The Physics and Economics of Memory established the physical and economic foundation, while Part 2 — What AI Actually Does to Memory showed how AI workloads turn those properties into concrete capacity and bandwidth pressures.

Disclaimer and Ownership Disclosure

MemoryCapex provides independent research and analysis for informational and educational purposes only. Nothing in this publication constitutes investment advice, an investment recommendation, or an offer or solicitation to buy or sell any security.

The author holds, directly or indirectly, positions in substantially all of the publicly traded companies mentioned in this publication or series. Those positions may change without notice.

What AI Actually Does to Memory

Series: How AI Memory Actually Works: A Step-by-Step Guide from SRAM to HBM4, NAND and Beyond

Previously in this series (Part 1)

Compute is economically worthless if the data it needs does not arrive in time. Memory CAPACITY and BANDWIDTH are separate demand drivers. DDR and HBM are both forms of DRAM.

Abstract

Knowing the memory hierarchy is only the beginning. The next question is what an AI workload actually puts inside that memory, and why the answer changes between training and inference. This part begins with the five major occupants of accelerator memory: model weights, activations, gradients, optimizer state and the KV cache. It then builds the arithmetic connecting parameter count, numerical precision and raw weight storage. From there, we separate training from inference and inference itself into prefill and decode, introducing arithmetic intensity as the bridge between computation and data movement. The later steps show how these pressures created the memory wall, why HBM became an architectural requirement, how its roadmap is evolving, and why even HBM cannot economically hold every byte.

Estimated reading time: 12–14 minutes

Revision History

• Version 1.0 — September 13, 2026: Initial publication.

Step 6 — What Actually Occupies AI Memory?

THE QUESTION — When an AI accelerator says it has a certain amount of memory, what is actually occupying those bytes?

THE INTUITION

Think of a moving truck. The truck does not contain only the furniture you ultimately care about. It also carries boxes, packing materials, tools and temporary items needed during the move. Two households with the same amount of furniture can therefore require very different truck space depending on how the move is organized.

THE MECHANISM

The first occupant is WEIGHTS. These are the numerical values learned during training. They encode the model’s learned transformations and form the most obvious connection between model size and memory demand. A PARAMETER is one such learned numerical value.

The second occupant is ACTIVATIONS. These are intermediate results created as input data passes through the model. An activation may exist only briefly, but many activations can be needed simultaneously. During training, some must remain available because the backward pass needs information generated during the earlier forward pass.

The third occupant is GRADIENTS. Training uses gradients to describe how model parameters should change to reduce error. The fourth is OPTIMIZER STATE: additional numerical state maintained by optimization algorithms while updating the parameters. The ZeRO research identified parameters, gradients and optimizer states as major categories of model-state memory that become increasingly difficult to replicate across every accelerator as models scale (Rajbhandari et al., October 4, 2019).^[S9]

The fifth occupant becomes especially important during inference: the KV CACHE, or Key-Value cache. Transformer models retain key and value tensors associated with previously processed tokens so they do not have to recompute all prior attention state whenever the next token is generated. Unlike static model weights, KV-cache consumption can expand and contract with request length and the number of simultaneous requests. The PagedAttention paper specifically identifies this dynamically growing and shrinking cache as a major memory-management problem in large-language-model serving (Kwon et al., September 12, 2023).^[S10]

Weight memory itself begins with simple arithmetic:

raw weight memory = number of parameters × bits per parameter ÷ 8.

Take a hypothetical model with 70 billion parameters. NVIDIA’s current TensorRT documentation defines BF16 as a 16-bit format, FP8 as an 8-bit format, INT8 as an 8-bit integer format and INT4 as a 4-bit integer format (accessed September 13, 2026).^[S8]

Representation: BF16

Working: 70B × 16 ÷ 8

Idealized raw weight storage: 140 billion bytes = 140 GB

Representation: FP8

Working: 70B × 8 ÷ 8

Idealized raw weight storage: 70 billion bytes = 70 GB

Representation: INT8

Working: 70B × 8 ÷ 8

Idealized raw weight storage: 70 billion bytes = 70 GB

Representation: 4-bit

Working: 70B × 4 ÷ 8

Idealized raw weight storage: 35 billion bytes = 35 GB

This is raw arithmetic, not a promise about actual deployment memory. QUANTIZATION can represent model values with fewer bits, but real implementations may also require scaling factors, metadata, alignment or higher-precision components. FP8 and INT8 also occupy the same raw number of bits while representing numbers differently. Weight precision therefore changes one major memory component, not the entire workload footprint.

WHY IT MATTERS FINANCIALLY

Memory demand cannot be inferred from parameter count alone. For Micron, Samsung and SKhynix, the economically relevant demand depends on precision, training configuration, context length and serving concurrency; for Nvidia, Broadcom and other accelerator designers, those same variables influence how much useful work a fixed amount of accelerator memory can support.

Software that reduces one memory component can change hardware utilization without eliminating the demand created by the others.

ONE-LINE TAKEAWAY

Model size sets weight memory; workload state determines the rest.

Step 7 — Training and Inference Stress Memory Differently

THE QUESTION — Why can the same model create one kind of memory problem during training and another when serving users?

THE INTUITION

Film production and film screening use the same finished movie but require very different resources. Production keeps raw footage, edits, intermediate cuts and working files. Screening discards most of that production state, but now every frame must reach the projector continuously and on time.

THE MECHANISM

TRAINING is the process that changes model parameters. A forward pass generates predictions and intermediate activations. A backward pass calculates gradients. An optimizer then uses those gradients and its own state to update the weights. Training must therefore accommodate WEIGHTS, ACTIVATIONS, GRADIENTS, OPTIMIZER STATE and temporary buffers at the same time. ZeRO was developed specifically to reduce redundant copies of these model states across devices rather than requiring every accelerator to hold the full replicated set (Rajbhandari et al., October 4, 2019).^[S9]

INFERENCE is different. The model’s parameters are no longer being optimized, so gradients and optimizer state are generally absent from normal inference. Weights and activations remain, while the KV cache becomes important for autoregressive transformer models.

Inference itself contains two distinct phases. During PREFILL, the model processes the input prompt and constructs the state needed before generating the first output token. During DECODE, it generates subsequent tokens sequentially while repeatedly reading model state and prior context. NVIDIA describes prefill as compute-bound and decode as memory-bound in its Dynamo architecture discussion published March 18, 2025.^[S11] This is a useful architectural model, although the exact balance still depends on model architecture, batch size, sequence length, precision and hardware utilization.

To understand why the phases behave differently, we need one new concept: ARITHMETIC INTENSITY. It measures how much computation is performed for each byte of data moved. The Roofline performance model uses this relationship to distinguish workloads limited by memory bandwidth from workloads limited by peak compute throughput. NVIDIA’s Nsight Compute documentation defines arithmetic intensity as the ratio of compute work to data movement, conventionally expressed as floating-point operations per byte (NVIDIA, November 18, 2020).^[S12]

High arithmetic intensity means that data loaded from memory is reused for substantial computation. Low arithmetic intensity means the processor performs relatively little arithmetic before needing more data. That is why increasing raw compute throughput alone does not guarantee the same performance gain across both phases. If decode spends much of its time supplying weights and cache state to the arithmetic units, faster mathematics cannot fully compensate for insufficient data delivery.

WHY IT MATTERS FINANCIALLY

Training-heavy infrastructure and inference-heavy infrastructure can purchase similar accelerators for different reasons. Nvidia and custom-accelerator designers such as Broadcom must balance arithmetic throughput against usable memory capacity and bandwidth, while Micron, Samsung and SKhynix are exposed to whichever memory constraint becomes binding at scale.

For operators, this distinction also changes economics: a system optimized for maximum training throughput is not automatically the lowest-cost system for latency-sensitive inference.

ONE-LINE TAKEAWAY

Training is state-heavy; decoding often turns memory bandwidth into the limiting resource.

Step 8 — The Memory Wall: Why HBM Had to Exist

THE QUESTION — What happens when each generation adds compute faster than it improves the system’s ability to deliver data?

THE INTUITION

Imagine a call center that keeps hiring agents far faster than it installs phone lines. At first, every new employee raises output. Over time, however, more agents spend part of the day waiting for an available line. The problem is not that the phone system stopped improving. It is that staffing capacity and communications capacity are improving at different rates.

THE MECHANISM

This divergence is the essence of the MEMORY WALL. It is not one fixed bandwidth limit. It is the tendency for arithmetic capability to scale faster than the rate at which data can be supplied from off-chip memory.

A useful historical comparison comes from an open-access preprint by Jack Dongarra, John Gunnels, Harun Bayraktar, Azzam Haidar and Dan Ernst, published November 18, 2024. Comparing peak dense FP16 Tensor throughput against peak memory bandwidth, their table moves from NVIDIA V100 at 125 TFLOP/s and 0.9 TB/s to NVIDIA B200 at 2,250 TFLOP/s and 8 TB/s. On those peak specifications, compute throughput rises about 18× while memory bandwidth rises about 8.9×.^[S13] The comparison uses theoretical peak capabilities rather than achieved application performance, but it clearly illustrates the widening compute-to-bandwidth ratio across those endpoints.

Conventional board-level memory connections struggle with that requirement because adding bandwidth through narrower electrical interfaces eventually raises pin count, power, signal-integrity and physical-routing constraints. HBM changes the geometry.

Multiple DRAM dies are stacked vertically. THROUGH-SILICON VIAS (TSVs) create vertical electrical paths through the stack. Beneath the DRAM sits a LOGIC BASE DIE that manages the interface between the memory stack and the rest of the package. The HBM stacks and accelerator die can then sit on an INTERPOSER, which provides dense, short routing between logic and memory. TSMC’s CoWoS documentation explicitly describes logic dies and HBM stacks integrated across a high-density interposer.^[S14]

The economics are as important as the geometry. Stacking creates more points at which value can be lost. If several component and assembly stages must all succeed, a simplified compound yield is their yields multiplied together. Real HBM manufacturing is more complicated because it uses testing, redundancy and process-specific repair, but the principle remains: a bad die or bad bond can destroy value already embedded elsewhere in the stack.

This makes KNOWN-GOOD-DIE screening valuable. Imec notes that die-to-wafer integration can stack only known-good dies, improving compound yield relative to indiscriminate stacking (May 29, 2024).^[S15] Applied Materials likewise describes HBM and advanced packaging as increasingly dependent on deposition, chemical-mechanical polishing, metrology and defect control to achieve higher-yield stacking (June 25, 2026).^[S16]

HBM is therefore better understood as a co-designed memory, logic and packaging subsystem. Its economics extend well beyond the DRAM cells themselves.

WHY IT MATTERS FINANCIALLY

The value captured by HBM is distributed across more of the semiconductor chain than conventional DRAM alone. Micron, Samsung and SKhynix manufacture the memory stacks; TSMC participates in logic and advanced packaging; Applied Materials supplies process equipment; and companies such as Hanmi, BESI and Amkor are exposed to different portions of bonding, assembly and packaging.

For accelerator designers such as Nvidia and Broadcom, memory architecture increasingly becomes part of system design rather than a component selected after the compute die is finished.

ONE-LINE TAKEAWAY

HBM exists because compute scaled faster than conventional memory could feed it.

Step 9 — HBM3 → HBM3E → HBM4 → HBM4E: The Bandwidth Race

THE QUESTION — How does the industry keep increasing memory bandwidth without simply making every electrical signal run faster?

THE INTUITION

Think of an elevator bank in a high-rise building. More people can be moved per minute in two different ways: the elevators can travel faster, or the building can add more elevator shafts. The highest throughput comes from combining both—but adding shafts is a different architectural change from merely speeding up the existing elevators.

THE MECHANISM

The same distinction helps explain the HBM roadmap. Memory bandwidth is approximately the product of how many data paths exist and how quickly each path transfers data.

HBM3E largely extends the HBM3 architecture through faster signaling and product-level improvements while retaining a 1,024-data-signal interface. HBM4 makes a more structural change: it doubles the interface width to 2,048 data signals. Samsung explicitly describes HBM4 as increasing I/O count from 1,024 in HBM3E to 2,048, while Micron describes its HBM4 bus as 2,048 bits wide.^[S17][S18]

JEDEC published the HBM4 standard, JESD270-4, on April 16, 2025. The standards page itself is publicly linkable, but the full JESD270-4 document was not publicly readable in our web verification; the 2,048-bit interface figure used here therefore comes from JEDEC’s public April 16 announcement and is independently corroborated by Micron’s HBM4 product statement and Samsung’s commercial HBM4 disclosure.^{[S19][S18][S17]}

It is also important to separate a standards roadmap from a vendor product roadmap. A JEDEC standard defines an interoperable technical framework. Memory vendors can then commercialize products at different speeds, stack heights, capacities and process technologies within or beyond particular baseline operating points.

HBM4E illustrates the distinction. As of September 13, 2026, Samsung and SKhynix have both disclosed 12-layer HBM4E samples, each citing speeds of up to 16 Gbps per pin; Samsung announced sample shipment on May 29, 2026, and SKhynix on June 18, 2026.^[S20][S21] These are sample specifications, not mass-production specifications. Neither disclosure describes HBM4E as another doubling of interface width. The useful mental model is therefore an enhanced HBM4-class roadmap emphasizing higher signaling speed, capacity, process technology and customization—not a second automatic doubling from 2,048 to 4,096 signals.

Accelerator evolution shows why both capacity and bandwidth must be tracked separately. NVIDIA’s current HGX documentation lists H100 SXM at 80 GB and 3.35 TB/s, H200 SXM at 141 GB and 4.8 TB/s, and B200 SXM at 180 GB and up to 8 TB/s (accessed September 13, 2026).^[S22]

NVIDIA’s July 21, 2026 Rubin architecture disclosure lists up to 288 GB of HBM4 and up to 22 TB/s of memory bandwidth per GPU.^[S23] On NVIDIA’s August 26, 2026 earnings call, management stated that production shipments of Vera Rubin had commenced earlier that month.^[S24] Rubin is therefore no longer treated here as an unreleased roadmap product; however, 288 GB and 22 TB/s remain NVIDIA-stated up to maximum specifications, not measured values for every shipping configuration.

WHY IT MATTERS FINANCIALLY

A wider interface raises the importance of logic-base-die design, packaging density and signal integrity alongside the DRAM itself. That broadens the economic exposure from Micron, Samsung and SKhynix toward TSMC, advanced-packaging equipment suppliers and accelerator designers such as Nvidia and Broadcom.

HBM4E also illustrates why investors should distinguish a standards milestone, a customer sample, qualification and volume production. Those stages carry very different implications for revenue recognition, yield learning and capital deployment.

ONE-LINE TAKEAWAY

HBM generations raise bandwidth through wider interfaces, faster signaling and tighter compute-memory integration.

Step 10 — Why HBM Cannot Hold Everything

THE QUESTION — If HBM is the fastest large memory near the accelerator, why not place every useful byte there?

THE INTUITION

A Formula One team keeps the tools needed for the next pit stop immediately beside the car. Equipment that might be needed later stays deeper in the garage. Bulky reserves remain in transporters. Moving everything to the pit lane would reduce retrieval time—but the most valuable space would become unusably crowded and expensive.

THE MECHANISM

HBM solves a bandwidth problem by placing sophisticated stacked DRAM extremely close to compute. That advantage also makes it scarce and expensive in physical and manufacturing terms.

HBM consumes leading-edge DRAM wafer capacity, additional stacking resources, logic-base-die content and advanced packaging. Micron stated in its December 17, 2025 fiscal Q1 2026 presentation that HBM then carried approximately a 3-to-1 trade ratio with DDR5 in terms of supply consumption, and that the ratio increases with future HBM generations.^[S25] This is a Micron supply-planning statement, not a universal physical constant, but it demonstrates why shifting a larger portion of DRAM production toward HBM can affect the availability of conventional DRAM.

The practical answer is MEMORY TIERING.

Hot data is data whose immediate availability can directly affect accelerator utilization. Active model state, heavily used portions of a KV cache and temporary working data may deserve HBM residency.

Warm data still needs relatively fast access but does not necessarily justify consuming scarce on-package HBM. System DDR can hold staging data, host-side state or information that software can move into HBM when required.

Cold data has much weaker latency requirements. Model files, datasets, checkpoints and other persistent information can reside on NAND-based SSD storage at a much lower cost per stored byte.

Data therefore moves in both directions. PROMOTION moves information toward a faster, more expensive tier when access becomes urgent. EVICTION pushes information outward when scarce fast memory would be more valuable to something else.

This makes memory placement a scheduling problem and an economic problem at the same time. The correct question is not “Where can this byte fit?” It is “What is the performance value of keeping this byte in the expensive tier right now?”

The supply side reinforces the issue. In its fiscal Q3 2026 prepared remarks, Micron said HBM’s rising trade ratio and increasing process complexity were putting additional pressure on overall memory supply and cleanroom requirements (June 24, 2026).^[S26] That means HBM demand can influence economics beyond HBM itself.

WHY IT MATTERS FINANCIALLY

For Micron, Samsung and SKhynix, rising HBM mix can change both product value and the amount of conventional DRAM supply displaced by each unit of output. For SanDisk and Kioxia, the fact that most data cannot economically remain in HBM preserves a distinct role for NAND-based storage even inside AI infrastructure.

For Nvidia, Broadcom and system operators, better placement algorithms can raise accelerator utilization without purchasing the most expensive tier for every byte—making software policy part of hardware economics.

ONE-LINE TAKEAWAY

The economic problem is deciding which bytes deserve the fastest and most expensive memory tier.

What I Am Least Confident About

• Visual 6 uses illustrative proportions for weights, activations, gradients, optimizer state and KV cache. Their actual shares can change dramatically with model architecture, numerical precision, optimizer choice, batch size, sequence length and serving concurrency.

• The V100-to-B200 memory-wall comparison uses peak dense FP16 Tensor throughput and peak memory bandwidth, not achieved application performance. It demonstrates how the ratio of theoretical compute capability to memory bandwidth has shifted, but individual workloads can experience very different bottlenecks.^[S13]

• NVIDIA has confirmed on its August 26, 2026 earnings call that Vera Rubin production shipments began in August 2026, but the 288 GB HBM4 capacity and 22 TB/s bandwidth figures remain vendor-stated up to specifications. They should not be interpreted as measured performance or as specifications of every shipping configuration.^[S23][S24]

• HBM4E remains a sampling-stage technology in the Samsung and SKhynix disclosures used here. The cited 16 Gbps-per-pin figures are vendor sample specifications, not volume-production specifications.^[S20][S21]

• Micron’s roughly 3-to-1 HBM-versus-DDR5 trade ratio is a company supply-planning disclosure, not an industry-wide physical constant. The ratio depends on generation, process technology, die size, stack configuration and manufacturing execution.^[S25]

Sources

• [S8] NVIDIA. Data Format Descriptions — NVIDIA TensorRT. Current documentation; accessed September 13, 2026. Used for BF16, FP8, INT8 and INT4 bit widths. Rights: quote-with-attribution only. Official source.

• [S9] Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase and Yuxiong He. ZeRO: Memory Optimizations Toward Training Trillion Parameter Models. arXiv, October 4, 2019. Used for the decomposition of training model state into parameters, gradients and optimizer state and the motivation for partitioning those states. Rights: quote-with-attribution only; publicly accessible research paper. Stable source.

• [S10] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang and Ion Stoica. Efficient Memory Management for Large Language Model Serving with PagedAttention. arXiv, September 12, 2023. Used for the dynamic growth and shrinkage of KV-cache memory and memory fragmentation in LLM serving. Rights: quote-with-attribution only; publicly accessible research paper. Stable source.

• [S11] NVIDIA Technical Blog. NVIDIA Dynamo, A Low-Latency Distributed Inference Framework for Scaling Reasoning AI Models. March 18, 2025. Used for the distinction between compute-bound prefill and memory-bound decode. Rights: quote-with-attribution only. Official source.

• [S12] Jackson Marusarz, Max Katz, Charlene Yang and Samuel Williams, NVIDIA Technical Blog. Accelerating HPC Applications with NVIDIA Nsight Compute Roofline Analysis. November 18, 2020. Used for the definition of arithmetic intensity and the Roofline distinction between memory-bound and compute-bound execution. Rights: quote-with-attribution only. Official source.

• [S13] Jack Dongarra, John Gunnels, Harun Bayraktar, Azzam Haidar and Dan Ernst. Hardware Trends Impacting Floating-Point Computations In Scientific Applications. arXiv / CoRR, November 18, 2024. Used for the V100-to-B200 comparison of peak dense FP16 Tensor throughput and peak memory bandwidth. Access: open-access preprint; not treated here as a peer-reviewed journal article. Rights: quote-with-attribution only; republication rights are not assumed. Stable open-access source.

• [S14] Taiwan Semiconductor Manufacturing Company. CoWoS®. Current technology documentation; accessed September 13, 2026. Used for interposer-based integration of logic dies and HBM stacks and high-density die-to-die routing. Rights: quote-with-attribution only. Official source.

• [S15] imec. Imec Demonstrates Die-to-Wafer Hybrid Bonding with a Cu Interconnect Pad Pitch of 2µm. May 29, 2024. Used for die-to-wafer integration, known-good-die economics and compound-yield considerations. Rights: quote-with-attribution only. Official source.

• [S16] Applied Materials. Applied Materials Introduces New Systems to Accelerate DRAM and Advanced Packaging for AI Chips. June 25, 2026. Used for HBM stacking, chemical-mechanical polishing, deposition, metrology, defect control and advanced-packaging yield. Rights: quote-with-attribution only. Official source.

• [S17] Samsung Electronics. Samsung Ships Industry-First Commercial HBM4 With Ultimate Performance for AI Computing. February 12, 2026. Used for Samsung’s commercial HBM4 status, 2,048 I/Os, logic base die and vendor-stated HBM4 specifications. Rights: quote-with-attribution only. Official source.

• [S18] Micron Technology. HBM4. Current product page; accessed September 13, 2026. Used to independently corroborate the 2,048-bit HBM4 interface and Micron’s current HBM4 product specifications. Rights: quote-with-attribution only. Official source.

• [S19] JEDEC Solid State Technology Association. JEDEC® and Industry Leaders Collaborate to Release JESD270-4 HBM4 Standard: Advancing Bandwidth, Efficiency, and Capacity for AI and HPC. April 16, 2025. JEDEC’s public announcement states that HBM4 uses a 2,048-bit interface and identifies the standard as JESD270-4. The standard’s landing page is publicly linkable, but the complete standards document was not directly readable in our public web verification. Rights: quote-with-attribution only; the JEDEC standard itself must not be republished. JEDEC public announcement distributed via Business Wire.

• [S20] Samsung Electronics. Samsung Electronics Begins Shipment of Industry-First HBM4E Samples. May 29, 2026. Used for the 12-layer HBM4E sample status and Samsung’s vendor-stated speed of up to 16 Gbps per pin. Rights: quote-with-attribution only. Official source.

• [S21] SKhynix. SKhynix Ships Samples of 12-Layer Next-Gen ‘HBM4E’. June 18, 2026. Used for SKhynix’s HBM4E sample status and vendor-stated maximum speed of 16 Gbps per pin. Rights: quote-with-attribution only. Official source.

• [S22] NVIDIA. Components — NVIDIA HGX AI Factory (HGX H100, H200, B200). Current documentation; accessed September 13, 2026. Used for H100 SXM 80 GB / 3.35 TB/s, H200 SXM 141 GB / 4.8 TB/s and B200 SXM 180 GB / up to 8 TB/s. Rights: quote-with-attribution only. Official source.

• [S23] Eduardo Alvarez, Vishal Mehta and Farshad Ghodsian, NVIDIA Technical Blog. Inside NVIDIA Rubin GPU Architecture: Powering the Era of Agentic AI. July 21, 2026. Used for NVIDIA’s Rubin GPU architecture disclosure of up to 288 GB HBM4 and up to 22 TB/s of peak memory bandwidth. Rights: quote-with-attribution only. Official source.

• [S24] NVIDIA Investor Relations. Q2 Fiscal 2027 Earnings Call — Corrected Transcript. August 26, 2026. On NVIDIA’s August 26, 2026 earnings call, management stated that production shipments of Vera Rubin had commenced earlier that month. This source establishes shipment status; it does not convert the separate up to architecture specifications into measured values for every configuration. Rights: quote-with-attribution only; transcript content is copyrighted and must not be republished. Official investor-relations source.

• [S25] Micron Technology. Fiscal Q1 2026 Earnings Presentation. December 17, 2025. Used for Micron’s statement that HBM had an approximately 3-to-1 trade ratio with DDR5 at that time and that the ratio increases with future HBM generations. Rights: quote-with-attribution only. Official investor-relations source.

• [S26] Micron Technology. Fiscal Q3 2026 Earnings Call Prepared Remarks. June 24, 2026. Used for Micron’s statements that HBM’s increasing trade ratio, greater process complexity and growing cleanroom requirements further pressure non-HBM supply. Rights: quote-with-attribution only. Official investor-relations source.

No subscription research, paid market data or licensed market-research datasets are used in Part 2. Public accessibility does not imply republication permission; unless explicitly stated otherwise, the sources above are treated as quote-with-attribution only.

Next in This Series

If HBM cannot economically hold every useful byte, how will systems expand, pool and reorganize memory and storage beyond the accelerator package? And as new interconnects, software techniques and emerging memory tiers reshape the hierarchy, which parts of the semiconductor value chain capture the resulting economics?

Memory Hierarchy- Part 3

Disclaimer and Ownership Disclosure

MemoryCapex provides independent research and analysis for informational and educational purposes only. Nothing in this publication constitutes investment advice, an investment recommendation, or an offer or solicitation to buy or sell any security.

The author holds, directly or indirectly, positions in substantially all of the publicly traded companies mentioned in this publication or series. Those positions may change without notice.

The Physics and Economics of Memory

Series: How AI Memory Actually Works: A Step-by-Step Guide from Static Random-Access Memory to High Bandwidth Memory, NAND and Beyond

Abstract

Memory is often discussed as if it were a single component with one job. It is not. Modern computing relies on several memory technologies because no device can maximize speed, capacity, persistence, energy efficiency and low cost at the same time. This first part builds the vocabulary needed to analyze that system from first principles. We start with the movement of data into and out of compute, separate capacity from bandwidth, then build the memory hierarchy from fast local storage to persistent NAND. Finally, we examine the physical logic behind SRAM, DRAM and NAND and explain why DDR and HBM, despite different system roles, are both forms of DRAM. No product roadmap or accelerator specification is required to understand these foundations.

Estimated reading time: 10–12 minutes

Revision History

• Version 1.0 — September 13, 2026: Initial publication.

Step 1 — AI Is a Data-Movement Machine Before It Is a Compute Machine

THE QUESTION — Why can a processor capable of enormous computation still spend time waiting?

THE INTUITION

Think about a busy restaurant kitchen. The stove may be extremely powerful, but the meal is not produced by the stove alone. Ingredients must leave storage, reach the preparation table, be cut and organized, arrive beside the cook, and then be used at exactly the right moment. A faster stove does little if the ingredients are still in the basement.

THE MECHANISM

A processor does not compute on abstract numbers floating in space. Every instruction operates on data that must exist somewhere physically, be retrieved, moved to the appropriate processing unit, transformed, and often written somewhere again.

A useful first model is therefore:

load → compute → store → move → repeat.

The “compute” stage receives most of the attention because it is where arithmetic happens. But the other stages determine whether the arithmetic hardware has useful work to do.

Consider the data required to produce a single output token from an artificial intelligence model. Some information may originally reside in a solid-state drive built from NAND flash. Active information can be held in Double Data Rate dynamic random-access memory (DDR DRAM) at the system level. Data needed by an accelerator may reside in High Bandwidth Memory (HBM). Within the processor, much smaller pieces of information can move into caches built from Static Random-Access Memory (SRAM) and then into the structures immediately feeding the arithmetic units.

This is a hierarchy, not a mandatory conveyor belt. Not every byte involved in every computation literally passes through every level in that exact sequence. Direct transfers, caching, prefetching, and architecture-specific paths can bypass stages. The sequence is valuable because it exposes the underlying economic problem: data located farther from compute is generally available in much larger quantities and at lower cost per stored bit, but reaching it takes more time and usually traverses narrower interfaces.

That physical movement matters. A computation cannot begin until its required operands are available. Once the result is produced, it may itself become an input to the next calculation. Modern computing is therefore a repeated coordination problem between arithmetic and data movement.

This is the first conceptual shift required to understand memory. A processor is not an isolated engine. It is one component inside a system whose job is to place the right data close enough to the engine at the right time.

WHY IT MATTERS FINANCIALLY

Different companies monetize different portions of this path. Micron, Samsung and SKhynix manufacture working-memory technologies; SanDisk and Kioxia are exposed to NAND-based storage; Nvidia and Broadcom design compute architectures whose economics depend partly on keeping expensive arithmetic hardware supplied with data.

The financial question is therefore not simply whether “more memory” is required. It is which layer must expand, how difficult that layer is to manufacture, and how much of the system cost shifts toward moving versus processing data.

ONE-LINE TAKEAWAY

Compute creates value only when the memory system can deliver the right data at the right time.

Step 2 — What Does “Better Memory” Actually Mean?

THE QUESTION — When someone says one memory technology is “better” than another, what exactly are they measuring?

THE INTUITION

Think of a library. One building may hold more books. Another may have wider circulation desks that move many books at once. A third may place the most requested books beside the reader, reducing waiting time. Another may be cheaper to operate, while an archive can preserve material for decades without requiring an active working environment. Calling one of them simply “better” misses the design problem.

THE MECHANISM

Memory technologies are better understood through six separate dimensions. They interact, but none can substitute completely for another.

1. Capacity. How much information can be stored? Capacity determines whether a working set fits at all. A system can have excellent performance on every other dimension and still fail if the required data does not fit in the available memory.

2. Bandwidth. How much information can move through the memory interface in a given period? Bandwidth is a throughput concept. It describes the width of the data-delivery pipe, not the total amount of data stored behind it.

3. Latency. How long does one requested access take before useful data begins to arrive? A system can have high aggregate bandwidth and still have meaningful access latency. Bandwidth and latency are therefore not interchangeable measures.

4. Energy. Reading, writing and physically moving information consume power. The farther data travels and the more interfaces it crosses, the more important data movement can become to system-level energy consumption. The relevant metric is not merely the power rating of a memory chip, but the energy required to deliver useful data where computation needs it.

5. Persistence. Does the information survive when electrical power is removed? SRAM and DRAM are volatile: their stored state depends on continued power. NAND flash is non-volatile and therefore suitable for persistent storage.^[S1][S2]

6. Cost per bit. How much economic cost is required to provide a given amount of stored information? This incorporates more than the selling price of a chip. Silicon area, process complexity, manufacturing yield, packaging, controllers, power and system integration can all affect the effective economics of a memory tier.

No existing technology dominates all six dimensions. SRAM is extremely useful close to compute but consumes substantial silicon area per stored bit. DRAM provides much greater density at the cost of refresh and higher access distance. NAND provides persistent, low-cost bulk capacity but cannot serve the same latency and bandwidth role as memory placed beside or inside a processor.

That is why the memory hierarchy exists. It is not an historical accident. It is the system-level response to incompatible optimization goals.

WHY IT MATTERS FINANCIALLY

These six axes explain why memory markets do not collapse into a single technology. Micron, Samsung and SKhynix can address several forms of DRAM because customers pay for different combinations of density, bandwidth, power and integration. SanDisk and Kioxia optimize a different economic frontier in NAND, where persistence and low cost per bit permit much larger pools of data.

The manufacturing implications also differ. Changes in density, cell design, process complexity and packaging can alter wafer requirements, equipment intensity, yield and capital efficiency, creating different exposures for suppliers such as ASML and Applied Materials even when the end customer simply describes the requirement as “more memory.”

ONE-LINE TAKEAWAY

There is no best memory technology—only different trade-offs among six competing economic and physical objectives.

Step 3 — Capacity and Bandwidth Are Two Different Demand Drivers

THE QUESTION — Why can adding more memory fail to make a computing system any faster?

THE INTUITION

Imagine two factories. The first has an enormous warehouse but only a narrow road connecting it to production. The second has a smaller warehouse but a much wider road. The first can store more material. The second can deliver material faster. Warehouse size and road width solve different problems.

THE MECHANISM

Capacity measures how much information can reside in a memory device or memory pool. Bandwidth measures how much information can move through an interface over time.

These variables are easy to confuse because both can improve when a memory subsystem is upgraded. Economically, however, they answer different questions.

A capacity constraint appears when the required working data simply does not fit. Adding capacity can allow a larger working set to remain resident instead of being divided, moved elsewhere, or repeatedly reloaded.

A bandwidth constraint appears when the data fits, but cannot be supplied to computation quickly enough. Adding more storage capacity does not necessarily help. The relevant intervention is a wider, faster or more parallel path between memory and the processor.

Latency is different again. It measures the delay associated with an individual access. A system may support substantial aggregate data throughput while still imposing meaningful waiting time on a particular request.

For financial analysis, this distinction prevents a common category error. Rising demand for stored bytes and rising demand for bytes transferred per second are not the same economic signal. They can favor different architectures, different amounts of silicon, different packaging choices and different pricing structures.

WHY IT MATTERS FINANCIALLY

For Micron, Samsung and SKhynix, a customer paying for more capacity is not necessarily solving the same problem as a customer paying for more bandwidth. The distinction also affects processor designers such as Nvidia and Broadcom because system architecture determines how much economic value can be extracted from each additional unit of memory and compute.

ONE-LINE TAKEAWAY

Capacity determines what fits; bandwidth determines how fast the processor can use what fits.

Step 4 — Why Computers Need a Memory Hierarchy

THE QUESTION — Why not build an entire computer from the fastest available memory?

THE INTUITION

A hospital does not keep every medical supply on the emergency cart beside the patient. The cart holds a small quantity of items needed immediately. A nearby supply room holds more. Central storage holds much more again. Faster access requires sacrificing space and increasing cost.

THE MECHANISM

A computing system follows the same principle. The information required immediately is kept closest to the arithmetic hardware. Larger pools sit progressively farther away.

At the top are registers and small caches built largely from SRAM. They offer very rapid local access but consume valuable processor area.

Outside that immediate layer sits working memory based on Dynamic Random-Access Memory (DRAM). Different DRAM architectures can be optimized for different system roles. HBM emphasizes wide, high-bandwidth connections close to high-performance compute. DDR and Low-Power Double Data Rate memory (LPDDR) serve broader system-memory roles.

Capacity can also be attached through interfaces such as Compute Express Link (CXL), allowing memory to exist farther from a processor than locally attached DRAM while remaining addressable as part of a larger system architecture.

Below working memory is persistent storage, commonly built from NAND flash and accessed through devices such as Non-Volatile Memory Express (NVMe) solid-state drives. NAND preserves data without continuous power, unlike SRAM and DRAM.^[S1][S2]

The hierarchy exists because moving upward typically improves proximity and access performance but raises the economic cost of each stored bit. Moving downward generally allows much larger capacity at lower cost, but with slower access. No single tier can economically replace all the others.

WHY IT MATTERS FINANCIALLY

The hierarchy creates separate markets rather than one homogeneous memory market. Micron, Samsung and SKhynix participate heavily in DRAM; SanDisk and Kioxia are major NAND suppliers; and changes in the mix between tiers alter wafer demand, process intensity, packaging requirements and system cost.

ONE-LINE TAKEAWAY

The hierarchy exists because speed, proximity, capacity and low cost cannot all be maximized at once.

Step 5 — SRAM, DRAM and NAND: Three Different Ways to Store a Bit

THE QUESTION — What physically makes cache, working memory and persistent storage behave so differently?

THE INTUITION

Imagine storing a message in three ways. One method keeps an active mechanical latch in position. Another stores water in a small cup that slowly leaks and must be refilled. A third traps ink inside a sealed material that remains marked after the machinery is turned off. Each method preserves information differently and creates different costs.

THE MECHANISM

Static Random-Access Memory (SRAM) stores state using a circuit made from multiple transistors. As long as power remains available, the circuit maintains its state without the periodic refresh required by DRAM. The trade-off is physical area: an SRAM cell consumes substantially more silicon than a DRAM cell, making large capacities expensive. That is why SRAM is commonly used for relatively small, speed-critical structures such as processor caches.^[S3]

Dynamic Random-Access Memory (DRAM) uses a conceptually simpler cell built around one transistor and one capacitor. The capacitor holds electrical charge and the transistor controls access to it. Because the stored charge leaks over time, the information must be refreshed periodically. DRAM is therefore volatile and loses its contents when power is removed. Its compact cell structure makes much higher density practical than SRAM.^[S1]

This leads to a crucial industry clarification: DDR and HBM are both DRAM. They differ in interface, packaging, physical placement and system role; they do not represent two unrelated underlying memory families. Samsung explicitly lists DDR, LPDDR, GDDR and HBM as DRAM product families (Samsung Semiconductor, accessed September 13, 2026).^[S1]

NAND flash solves a different problem. Modern three-dimensional NAND can retain information by storing charge in a charge-trap structure, allowing the data to persist after power is removed. Cells are arranged vertically to increase density, making NAND suitable for large persistent storage rather than acting as a direct substitute for SRAM or DRAM.^[S4]

The memory hierarchy therefore begins at the cell level. Different physical methods of representing a bit produce different combinations of speed, density, persistence, manufacturing complexity and cost.

WHY IT MATTERS FINANCIALLY

Cell physics ultimately shapes cost structure. SRAM consumes scarce logic-die area; DRAM economics depend heavily on density and process execution; NAND economics depend on achieving large persistent capacity efficiently through three-dimensional structures. Micron, Samsung, SKhynix, SanDisk and Kioxia therefore compete across markets whose unit economics are rooted in different physical architectures.

ONE-LINE TAKEAWAY

Different ways of storing one bit create fundamentally different semiconductor businesses.

What I Am Least Confident About

• The qualitative ordering in Visual 2 is intentionally directional rather than absolute. In particular, energy efficiency depends heavily on workload, data movement, controller behavior and physical placement.

• “Cost per bit” is a system-level economic shorthand, not a universal spot-price ranking. Process generation, capacity point, yield, packaging and controller requirements can materially change the effective cost relationship between tiers.

• The one-transistor, one-capacitor representation of DRAM is the correct conceptual cell model for conventional DRAM, but it deliberately excludes sense amplifiers, word-line circuitry, refresh logic and other peripheral structures required to make a working memory device.

• Modern three-dimensional NAND commonly uses charge-trap storage, but NAND cell architectures continue to evolve. Visual 5 should therefore be read as a conceptual representation of mainstream 3D NAND, not a universal transistor-level description of every current or future flash architecture.

Part 1 intentionally omits product-level bandwidth, latency, power, density and pricing figures. Those numbers vary by implementation and would make an otherwise foundational explanation age unnecessarily quickly.

Sources

The two background texts supplied by the author informed the teaching sequence and terminology of this guide. Technical claims were independently checked against the primary public sources below.

1. [S1] Samsung Semiconductor. DRAM | Memory | Samsung Semiconductor Global. Undated; accessed September 13, 2026. Supports the conventional DRAM one-transistor, one-capacitor cell model, periodic refresh, SRAM/DRAM structural differences, and the classification of DDR and HBM within the DRAM family. Rights status: quote-with-attribution only. Official source.

2. [S2] Samsung Semiconductor. SSD (Solid State Drive) | Semiconductor Dictionary. Undated; accessed September 13, 2026. Supports the distinction between volatile working memory and NAND-based persistent solid-state storage. Rights status: quote-with-attribution only. Official source.

3. [S3] Samsung Semiconductor. SRAM | Semiconductor Dictionary. Undated; accessed September 13, 2026. Supports the description of SRAM as a multi-transistor memory cell that does not require DRAM-style periodic refresh, and explains why SRAM occupies more silicon area per stored bit than DRAM. Rights status: quote-with-attribution only. Official source.

4. [S4] KIOXIA America. Analyzing Managed Flash Device Lifetime Reliability Part 1: The Fundamentals of TBW, WAF and NAND Flash Memory Endurance. Revision 1.0, March 2024. Supports the description of charge storage and the use of charge-trap technology in modern three-dimensional NAND. Rights status: quote-with-attribution only. Official source.

5. [S5] KIOXIA Corporation. What Is the 3D Flash Memory “BiCS FLASH”? Undated; accessed September 13, 2026. Supports the three-dimensional NAND structure and charge-storage film used in KIOXIA’s BiCS FLASH architecture. Rights status: quote-with-attribution only. Official source.

6. [S6] Compute Express Link Consortium. About CXL. Undated; accessed September 13, 2026. Supports the description of Compute Express Link as a cache-coherent interconnect connecting processors, accelerators and memory-expansion devices. Rights status: quote-with-attribution only. Official source.

7. [S7] Samsung Semiconductor. NVMe (Non-Volatile Memory Express) | Semiconductor Dictionary. Undated; accessed September 13, 2026. Supports the definition of NVMe as a protocol designed for high-performance access to solid-state storage over PCI Express. Rights status: quote-with-attribution only. Official source.

No subscription research, paid market data or licensed third-party datasets are used in Part 1. None of the sources above is assumed to grant blanket republication rights.

Next in This Series

What happens when an AI model begins filling this hierarchy with weights, activations, gradients and temporary state? And why can two workloads using the same model place radically different demands on memory?

Memory Hierarchy- Part 2

Disclaimer

MemoryCapex provides independent research and analysis for informational and educational purposes only. Nothing in this publication constitutes investment advice, an investment recommendation, or an offer or solicitation to buy or sell any security.

The author holds, directly or indirectly, positions in substantially all of the publicly traded companies mentioned in this publication or series. Those positions may change without notice.

A company-level snapshot of nine global memory and storage producers across DRAM, NAND and HDD.

Version 3 · Edition 2026-09-11 · Data as of 11 September 2026 · Retrieved 18:34 America/Los_Angeles

01

Covered universe

Nine producers only. The mosaic, financial table, product matrix and returns chart use this same list.

02

Comparable listed equity value — 7 companies

Same-date local close × latest official basic shares outstanding, translated at same-date FX. This is not industry value, value-chain value or total memory-market value.

Micron $1101.5B. 1,129,393,151 × $975.26

SK hynix $984.8B. 728,865,500 × ₩1,812,000 / 1,341.05

CXMT $575.0B. 67,884,099,077 × CN¥56.73 / 6.6974

SanDisk $239.2B. 146,419,001 × $1,633.35

Kioxia $192.8B. 548,014,638 × ¥54,030 / 153.554

Seagate $188.2B. 226,644,518 × $830.17

Western Digital $161.2B. 360,540,944 × $447.18

03

Latest reported financials

Native-currency issuer figures first. Periods are not synchronized and are not annualized.

Samsung Memory — Filing

SK hynix — Filing

Micron — Filing

SanDisk — Filing

Western Digital — Filing

Seagate — Filing

Kioxia — Filing

CXMT — Filing

YMTC — Filing

04

Product exposure

Issuer-sourced classification. A mark is presence of a commercial line, not share of sales.

05

Operating momentum

Reported facts from the latest issuer documents. Outlook and editorial inference are labelled separately.

1. Reported fact

   Samsung Memory Business sales were KRW 120.8T in 2Q26, KRW 74.8T in 1Q26 and KRW 21.2T in 2Q25. Operating profit, gross margin, operating cash flow and capex are not separately disclosed for Samsung Memory; do not substitute DS or Samsung Electronics group figures.

2. Reported fact

   SK hynix preliminary K-IFRS 2Q26 revenue KRW 79.3187T and operating profit KRW 60.5426T, an issuer-stated 76% operating margin, published 29 July 2026.

3. Reported fact

   Micron fiscal Q3 2026: revenue $41.456B, GAAP gross margin 84.6%, GAAP operating income $33.318B, operating cash flow $25.39B, capex $7.1B.

4. Reported fact

   SanDisk fiscal Q4 2026: revenue $8.965B, GAAP gross margin 84.6%, GAAP operating income $7.037B. FY26 revenue $20.248B is an annual figure, not an annualized quarter.

5. Reported fact

   Western Digital fiscal Q4 2026: revenue $3.747B, GAAP gross margin 54.1%, GAAP operating income $1.563B. FY26 revenue $12.919B is annual.

6. Reported fact

   Seagate fiscal Q4 2026: revenue $3.629B, GAAP gross margin 52.3%, GAAP operating margin 43.0%. FY26 revenue $12.195B and FY26 free cash flow $3.1B are annual.

7. Reported fact

   Kioxia Q1 FY2026: revenue JPY 1,767.1B, operating profit JPY 1,270.0B, operating cash flow JPY 866.3B, PP&E purchases JPY 51.2B.

8. Reported fact

   CXMT H1 2026 unaudited: revenue CNY 150.310B, +873.64% year on year; profit attributable to parent CNY 77.605B versus a prior-year loss.

9. Reported fact

   YMTC: no public audited or periodic financial statements were located; do not substitute analyst estimates. Revenue, operating profit, gross margin, operating cash flow, capex and listed equity value are null.

10. Editorial inference

    Among the seven listed names, the highest latest-period operating margins in this table sit at SK hynix, Micron and SanDisk. That comparison mixes K-IFRS, US GAAP and Japanese GAAP across unlike fiscal calendars and is not a ranking of cycle position.

11. Not claimed

    This edition does not assign a cycle phase.

06

Equity returns

Split-adjusted local-currency price returns to 10 September 2026 (Yahoo Finance v8 adj close). Windows that do not fully exist are shown as N/A for insufficient listing history. Samsung Memory and YMTC are excluded. No benchmark.

07

Methodology and source ledger

Market cap = official basic shares outstanding × 11 September 2026 local closing price ÷ same-date FX. Diluted weighted-average shares are not used. Micron shares are the Form 10-Q cover count as of 17 June 2026 (1,129,393,151). FX: Yahoo Finance KRW=X 1,341.05; JPY=X 153.554; CNY=X 6.6974, all 11 September 2026 closes. Equity returns are derived from Yahoo Finance v8 adjclose: ((end_adjusted_close / start_adjusted_close) − 1) × 100, with start/end trading dates stored in the ledger. Industry-share visuals are omitted: no retrieved source uses one period and one denominator for all nine names.

Company: SK hynix

Metric: basic_shares_outstanding

Value: 728865500 shares

Period: 2026-07-10 → 2026-07-10

Status: official_reported

Source: SK hynix Form 424B4 filed 10 Jul 2026

Company: SK hynix

Metric: last_close_price

Value: 1812000 price_per_share

Period: 2026-09-11 → 2026-09-11

Status: market_data

Source: MarketWatch — 000660.KS 11 Sep 2026 close

Company: SK hynix

Metric: listed_equity_value_usd

Value: 984.8285 USD_billion

Period: 2026-09-11 → 2026-09-11

Status: derived

Source: SK hynix Form 424B4 filed 10 Jul 2026

Company: Micron

Metric: basic_shares_outstanding

Value: 1129393151 shares

Period: 2026-06-17 → 2026-06-17

Status: official_reported

Source: Micron Form 10-Q filed 25 Jun 2026

Company: Micron

Metric: last_close_price

Value: 975.26 price_per_share

Period: 2026-09-11 → 2026-09-11

Status: market_data

Source: MarketWatch — MU 11 Sep 2026 close

Company: Micron

Metric: listed_equity_value_usd

Value: 1101.452 USD_billion

Period: 2026-09-11 → 2026-09-11

Status: derived

Source: Derived from Micron 10-Q shares × MarketWatch MU close

Company: SanDisk

Metric: basic_shares_outstanding

Value: 146419001 shares

Period: 2026-08-07 → 2026-08-07

Status: official_reported

Source: SanDisk FY26 Form 10-K (IR-hosted filing), 7 Aug 2026

Company: SanDisk

Metric: last_close_price

Value: 1633.35 price_per_share

Period: 2026-09-11 → 2026-09-11

Status: market_data

Source: MarketWatch — SNDK 11 Sep 2026 close

Company: SanDisk

Metric: listed_equity_value_usd

Value: 239.1535 USD_billion

Period: 2026-09-11 → 2026-09-11

Status: derived

Source: SanDisk FY26 Form 10-K (IR-hosted filing), 7 Aug 2026

Company: Western Digital

Metric: basic_shares_outstanding

Value: 360540944 shares

Period: 2026-08-07 → 2026-08-07

Status: official_reported

Source: Western Digital Form 10-K filed 7 Aug 2026

Company: Western Digital

Metric: last_close_price

Value: 447.18 price_per_share

Period: 2026-09-11 → 2026-09-11

Status: market_data

Source: MarketWatch — WDC 11 Sep 2026 close

Company: Western Digital

Metric: listed_equity_value_usd

Value: 161.2267 USD_billion

Period: 2026-09-11 → 2026-09-11

Status: derived

Source: Western Digital Form 10-K filed 7 Aug 2026

Company: Seagate

Metric: basic_shares_outstanding

Value: 226644518 shares

Period: 2026-07-31 → 2026-07-31

Status: official_reported

Source: Seagate Form 10-K filed 1 Aug 2026

Company: Seagate

Metric: last_close_price

Value: 830.17 price_per_share

Period: 2026-09-11 → 2026-09-11

Status: market_data

Source: FinanceCharts — STX 11 Sep 2026 close

Company: Seagate

Metric: listed_equity_value_usd

Value: 188.1535 USD_billion

Period: 2026-09-11 → 2026-09-11

Status: derived

Source: Seagate Form 10-K filed 1 Aug 2026

Company: Kioxia

Metric: basic_shares_outstanding

Value: 548014638 shares

Period: 2026-06-30 → 2026-06-30

Status: official_reported

Source: Kioxia TDnet document 2859923, 7 Aug 2026

Company: Kioxia

Metric: last_close_price

Value: 54030 price_per_share

Period: 2026-09-11 → 2026-09-11

Status: market_data

Source: Kabutan — 285A.T 11 Sep 2026 close

Company: Kioxia

Metric: listed_equity_value_usd

Value: 192.8262 USD_billion

Period: 2026-09-11 → 2026-09-11

Status: derived

Source: Kioxia TDnet document 2859923, 7 Aug 2026

Company: CXMT

Metric: basic_shares_outstanding

Value: 67884099077 shares

Period: 2026-06-30 → 2026-06-30

Status: official_reported

Source: CXMT H1 2026 interim report, CNINFO, 29 Aug 2026

Company: CXMT

Metric: last_close_price

Value: 56.73 price_per_share

Period: 2026-09-11 → 2026-09-11

Status: market_data

Source: MarketWatch — 688825.SS 11 Sep 2026 close

Company: CXMT

Metric: listed_equity_value_usd

Value: 575.0089 USD_billion

Period: 2026-09-11 → 2026-09-11

Status: derived

Source: CXMT H1 2026 interim report, CNINFO, 29 Aug 2026

Company: listed_universe

Metric: comparable_listed_equity_value_usd_7cos

Value: 3442.6493 USD_billion

Period: 2026-09-11 → 2026-09-11

Status: derived

Source: Derived calculation

Company: Samsung Memory

Metric: listed_equity_value_usd

Value: null USD_billion

Period: 2026-09-11 → 2026-09-11

Status: not_disclosed

Source: Editorial rule

Company: YMTC

Metric: listed_equity_value_usd

Value: null USD_billion

Period: 2026-09-11 → 2026-09-11

Status: not_disclosed

Source: YMTC company introduction

Company: listed_universe

Metric: fx_usdkrw

Value: 1341.05 KRW_per_USD

Period: 2026-09-11 → 2026-09-11

Status: market_data

Source: Yahoo Finance — KRW=X 11 Sep 2026

Company: listed_universe

Metric: fx_usdjpy

Value: 153.554 JPY_per_USD

Period: 2026-09-11 → 2026-09-11

Status: market_data

Source: Yahoo Finance — JPY=X 11 Sep 2026

Company: listed_universe

Metric: fx_usdcny

Value: 6.6974 CNY_per_USD

Period: 2026-09-11 → 2026-09-11

Status: market_data

Source: Yahoo Finance — CNY=X 11 Sep 2026

Company: Samsung Memory

Metric: memory_business_sales

Value: 120.8 KRW_trillion

Period: 2026-04-01 → 2026-06-30

Status: official_reported

Source: Samsung Electronics Q2 2026 conference PDF, 30 Jul 2026

Company: Samsung Memory

Metric: memory_business_sales

Value: 74.8 KRW_trillion

Period: 2026-01-01 → 2026-03-31

Status: official_reported

Source: Samsung Electronics Q2 2026 conference PDF, 30 Jul 2026

Company: Samsung Memory

Metric: memory_business_sales

Value: 21.2 KRW_trillion

Period: 2025-04-01 → 2025-06-30

Status: official_reported

Source: Samsung Electronics Q2 2026 conference PDF, 30 Jul 2026

Company: Samsung Memory

Metric: operating_profit

Value: null

Period: 2026-04-01 → 2026-06-30

Status: not_disclosed

Source: Samsung Electronics Q2 2026 conference PDF, 30 Jul 2026

Company: SK hynix

Metric: revenue

Value: 79.3187 KRW_trillion

Period: 2026-04-01 → 2026-06-30

Status: official_reported

Source: SK hynix Q2 2026 business results, 29 Jul 2026

Company: SK hynix

Metric: operating_profit

Value: 60.5426 KRW_trillion

Period: 2026-04-01 → 2026-06-30

Status: official_reported

Source: SK hynix Q2 2026 business results, 29 Jul 2026

Company: SK hynix

Metric: operating_margin

Value: 76 percent

Period: 2026-04-01 → 2026-06-30

Status: official_reported

Source: SK hynix Q2 2026 business results, 29 Jul 2026

Company: Micron

Metric: revenue

Value: 41.456 USD_billion

Period: 2026-02-27 → 2026-05-28

Status: official_reported

Source: Micron quarterly results IR page

Company: Micron

Metric: gaap_gross_margin

Value: 84.6 percent

Period: 2026-02-27 → 2026-05-28

Status: official_reported

Source: Micron quarterly results IR page

Company: Micron

Metric: gaap_operating_income

Value: 33.318 USD_billion

Period: 2026-02-27 → 2026-05-28

Status: official_reported

Source: Micron quarterly results IR page

Company: Micron

Metric: operating_cash_flow

Value: 25.39 USD_billion

Period: 2026-02-27 → 2026-05-28

Status: official_reported

Source: Micron quarterly results IR page

Company: Micron

Metric: capex

Value: 7.1 USD_billion

Period: 2026-02-27 → 2026-05-28

Status: official_reported

Source: Micron quarterly results IR page

Company: SanDisk

Metric: revenue

Value: 8.965 USD_billion

Period: 2026-04-04 → 2026-07-03

Status: official_reported

Source: SanDisk FY26 Q4 results release, 5 Aug 2026

Company: SanDisk

Metric: gaap_gross_margin

Value: 84.6 percent

Period: 2026-04-04 → 2026-07-03

Status: official_reported

Source: SanDisk FY26 Q4 results release, 5 Aug 2026

Company: SanDisk

Metric: gaap_operating_income

Value: 7.037 USD_billion

Period: 2026-04-04 → 2026-07-03

Status: official_reported

Source: SanDisk FY26 Q4 results release, 5 Aug 2026

Company: SanDisk

Metric: revenue_fy

Value: 20.248 USD_billion

Period: 2025-06-28 → 2026-07-03

Status: official_reported

Source: SanDisk FY26 Q4 results release, 5 Aug 2026

Company: Western Digital

Metric: revenue

Value: 3.747 USD_billion

Period: 2026-04-04 → 2026-07-03

Status: official_reported

Source: Western Digital FY26 Q4 results release, 5 Aug 2026

Company: Western Digital

Metric: gaap_gross_margin

Value: 54.1 percent

Period: 2026-04-04 → 2026-07-03

Status: official_reported

Source: Western Digital FY26 Q4 results release, 5 Aug 2026

Company: Western Digital

Metric: gaap_operating_income

Value: 1.563 USD_billion

Period: 2026-04-04 → 2026-07-03

Status: official_reported

Source: Western Digital FY26 Q4 results release, 5 Aug 2026

Company: Western Digital

Metric: revenue_fy

Value: 12.919 USD_billion

Period: 2025-06-28 → 2026-07-03

Status: official_reported

Source: Western Digital FY26 Q4 results release, 5 Aug 2026

Company: Seagate

Metric: revenue

Value: 3.629 USD_billion

Period: 2026-04-04 → 2026-07-03

Status: official_reported

Source: Seagate FY26 Q4 press release on EDGAR, 29 Jul 2026

Company: Seagate

Metric: gaap_gross_margin

Value: 52.3 percent

Period: 2026-04-04 → 2026-07-03

Status: official_reported

Source: Seagate FY26 Q4 press release on EDGAR, 29 Jul 2026

Company: Seagate

Metric: revenue_fy

Value: 12.195 USD_billion

Period: 2025-06-28 → 2026-07-03

Status: official_reported

Source: Seagate FY26 Q4 press release on EDGAR, 29 Jul 2026

Company: Seagate

Metric: free_cash_flow_fy

Value: 3.1 USD_billion

Period: 2025-06-28 → 2026-07-03

Status: official_reported

Source: Seagate FY26 Q4 press release on EDGAR, 29 Jul 2026

Company: Kioxia

Metric: revenue

Value: 1767.1 JPY_billion

Period: 2026-04-01 → 2026-06-30

Status: official_reported

Source: Kioxia Q1 FY2026 TDnet filing 2859908, 7 Aug 2026

Company: Kioxia

Metric: operating_profit

Value: 1270 JPY_billion

Period: 2026-04-01 → 2026-06-30

Status: official_reported

Source: Kioxia Q1 FY2026 TDnet filing 2859908, 7 Aug 2026

Company: Kioxia

Metric: profit_attributable_to_owners

Value: 842.2 JPY_billion

Period: 2026-04-01 → 2026-06-30

Status: official_reported

Source: Kioxia Q1 FY2026 TDnet filing 2859908, 7 Aug 2026

Company: Kioxia

Metric: operating_cash_flow

Value: 866.3 JPY_billion

Period: 2026-04-01 → 2026-06-30

Status: official_reported

Source: Kioxia Q1 FY2026 TDnet filing 2859908, 7 Aug 2026

Company: Kioxia

Metric: ppe_purchases

Value: 51.2 JPY_billion

Period: 2026-04-01 → 2026-06-30

Status: official_reported

Source: Kioxia Q1 FY2026 TDnet filing 2859908, 7 Aug 2026

Company: CXMT

Metric: revenue

Value: 150.31 CNY_billion

Period: 2026-01-01 → 2026-06-30

Status: official_reported

Source: CXMT H1 2026 interim report, CNINFO, 29 Aug 2026

Company: CXMT

Metric: profit_attributable_to_parent

Value: 77.605 CNY_billion

Period: 2026-01-01 → 2026-06-30

Status: official_reported

Source: CXMT H1 2026 interim report, CNINFO, 29 Aug 2026

Company: CXMT

Metric: revenue_yoy

Value: 873.64 percent

Period: 2026-01-01 → 2026-06-30

Status: official_reported

Source: CXMT H1 2026 interim report, CNINFO, 29 Aug 2026

Company: YMTC

Metric: revenue

Value: null

Period: — → —

Status: not_disclosed

Source: YMTC company introduction

Company: YMTC

Metric: operating_profit

Value: null

Period: — → —

Status: not_disclosed

Source: YMTC company introduction

Company: YMTC

Metric: capex

Value: null

Period: — → —

Status: not_disclosed

Source: YMTC company introduction

Company: Samsung Memory

Metric: operating_cash_flow

Value: null

Period: 2026-04-01 → 2026-06-30

Status: not_disclosed

Source: Samsung Electronics Q2 2026 conference PDF, 30 Jul 2026

Company: Samsung Memory

Metric: capex

Value: null

Period: 2026-04-01 → 2026-06-30

Status: not_disclosed

Source: Samsung Electronics Q2 2026 conference PDF, 30 Jul 2026

Company: YMTC

Metric: operating_cash_flow

Value: null

Period: — → —

Status: not_disclosed

Source: YMTC company introduction

Company: SK hynix

Metric: price_return_3m

Value: -9.5 percent

Period: 2026-06-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — 000660.KS

Company: SK hynix

Metric: price_return_6m

Value: 97.6 percent

Period: 2026-03-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — 000660.KS

Company: SK hynix

Metric: price_return_1y

Value: 511.3 percent

Period: 2025-09-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — 000660.KS

Company: Micron

Metric: price_return_3m

Value: 9.6 percent

Period: 2026-06-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — MU

Company: Micron

Metric: price_return_6m

Value: 142.6 percent

Period: 2026-03-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — MU

Company: Micron

Metric: price_return_1y

Value: 599.3 percent

Period: 2025-09-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — MU

Company: SanDisk

Metric: price_return_3m

Value: 3 percent

Period: 2026-06-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — SNDK

Company: SanDisk

Metric: price_return_6m

Value: 173.5 percent

Period: 2026-03-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — SNDK

Company: SanDisk

Metric: price_return_1y

Value: 2189.8 percent

Period: 2025-09-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — SNDK

Company: Western Digital

Metric: price_return_3m

Value: -5.9 percent

Period: 2026-06-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — WDC

Company: Western Digital

Metric: price_return_6m

Value: 73.2 percent

Period: 2026-03-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — WDC

Company: Western Digital

Metric: price_return_1y

Value: 386 percent

Period: 2025-09-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — WDC

Company: Seagate

Metric: price_return_3m

Value: 5.8 percent

Period: 2026-06-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — STX

Company: Seagate

Metric: price_return_6m

Value: 124.9 percent

Period: 2026-03-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — STX

Company: Seagate

Metric: price_return_1y

Value: 350.4 percent

Period: 2025-09-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — STX

Company: Kioxia

Metric: price_return_3m

Value: -17.6 percent

Period: 2026-06-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — 285A.T

Company: Kioxia

Metric: price_return_6m

Value: 196.8 percent

Period: 2026-03-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — 285A.T

Company: Kioxia

Metric: price_return_1y

Value: 1566.9 percent

Period: 2025-09-10 → 2026-09-10

Status: derived

Source: Yahoo Finance chart API — 285A.T

Company: CXMT

Metric: price_return_3m

Value: null percent

Period: 2026-06-10 → 2026-09-10

Status: not_disclosed

Source: Yahoo Finance chart API — 688825.SS

Company: CXMT

Metric: price_return_6m

Value: null percent

Period: 2026-03-10 → 2026-09-10

Status: not_disclosed

Source: Yahoo Finance chart API — 688825.SS

Company: CXMT

Metric: price_return_1y

Value: null percent

Period: 2025-09-10 → 2026-09-10

Status: not_disclosed

Source: Yahoo Finance chart API — 688825.SS

MemoryCapex is an independent, evidence-led research platform. From memory technology signals to investment meaning.

This artifact is a pre-publication research note. It is not investment advice, a solicitation, or a recommendation to buy, sell or hold any security. Figures are taken from issuer filings and market-data sources cited in the ledger; they can be revised when a later official print supersedes them. The author and associated accounts may hold long or short positions in named issuers. Coverage of a company is not an endorsement of its securities.

Version 3 · immutable artifact · supersedes Version 2 · built 2026-09-11T19:27:31-07:00 · supersedes: Version 2 · © 2026 MemoryCapex. All rights reserved. memorycapex.com

Memory’s share of AI infrastructure is not a single number. The spread from roughly 3 percent to more than 75 percent is primarily a denominator-and-scope problem, amplified by architecture generation, price timing and whether HBM is bought inside an accelerator. What has changed is the role of memory itself: it is no longer a component cost that is priced after the GPU is chosen. It is becoming a constraint on how capital is allocated.

Central thesis

The differentiated claim is not that memory takes a larger share of CapEx. It is that the same rise creates an endogenous demand-destruction mechanism: suppliers gain pricing power, customers cut memory per unit of compute, and a realized 68 percent path is a late-cycle warning for the buyer’s program before it is a continuation of the supplier bull case.

Disclosure and editorial contract

This note is independent research. It is not a recommendation to buy or sell any security. MemoryCapex does not assume or disclose a trading position in the companies named below. Company names appear as a map of value-chain exposure, not as a ranking. Scenario weights are subjective and should be revised after the next supplier prints. Review date: 5 September 2026, America/Los_Angeles.

• Version: 1.1 EN publication draft — Gate review of 5 Sep 2026

• Primary numeric anchor: TrendForce, 25 Aug 2026: DRAM + NAND at 47% / 68% of major-CSP CapEx

• Base call: The constraint holds through 2027; the 68% path is conditional and, if printed, a warning

• Weights: base 45% / supplier bull 25% / demand destruction 20% / LTA-as-peak 10%

1. Three shares, not one number

Secondary coverage this week translated TrendForce’s 68 percent figure as “68 cents of every construction dollar.” That translation is wrong. Memory content share sits on an engineering bill of materials. Memory procurement share sits on a cloud provider’s treasury in a year of extreme contract prices. Memory revenue share sits on a supplier’s income statement. Treating those three objects as one number is how a cycle gets misread.

The deeper distinction is the one that should travel with this note. Dollar TAM can rise fast while bit demand grows slowly and the supplier profit pool does something else again. A high memory share of CapEx is not proof of rising physical memory intensity. It can be the opposite: fewer bits, much higher prices, and a customer already redesigning the rack.

The June 2026 Semiconductor Industry Association study with Deloitte finds that semiconductors account for more than 95 percent of the content value of a leading AI server rack and more than half of the capital required to build and operate an AI data center. Memory and storage — HBM, DRAM/SRAM and NAND in that report — account for more than 20 percent of the rack’s semiconductor content value. The accelerator alone takes more than 70 percent of semiconductor content, and HBM is typically embedded in the accelerator price rather than purchased as a separate line item.

That last point is an accounting problem as much as a market problem. If HBM is counted inside GPU cost and again inside “memory,” the memory share is double-counted. If it is counted only inside the GPU, the memory industry’s economic role is understated. MemoryCapex’s rule is simple: for supplier economics, HBM is always memory; for a CSP budget, HBM is often bought inside a NVIDIA or ASIC system.

2. The denominator-and-scope problem

An earlier draft used “$15 of every $100” as a narrative hook without deriving it. That hook is gone. What remains is six named objects. They do not share a numerator. Google’s figure is high-performance memory on a server hardware BOM. TrendForce is DRAM plus NAND on a CSP treasury. Bernstein’s mid-2026 rack figure is memory plus storage. SIA/Deloitte is a semiconductor-content teardown. Putting those percentages on one axis invites a comparison the data do not support.

Figure 1. Denominator-and-scope map. Each card is a different object. Do not average the percentages. Review date 5 September 2026.

• Bernstein, GB200-era full build (Nov 2025 coverage): about 3–4% of full data-center CapEx; approximately US$1.0–1.2B per GW. Product scope: foundry / memory / WFE slice of the site. Source layer: secondary sell-side. Measures the entire site: IT, facility and land.

• Same slice after approximately 3× memory prices: about 9–12%. Same memory slice; later prices. MemoryCapex sensitivity. A price shock, not a new BOM.

• Bernstein Vera Rubin note, mid-2026: approximately US$3.2M of a US$9.1M rack, or about 35% of rack; about 24–25% of a modeled US$47B per GW build. Product scope: memory + storage. Secondary sell-side. Rack cost versus a full 1 GW stack.

• SIA / Deloitte, June 2026: more than 20% of rack semiconductor content. Product scope: HBM, DRAM/SRAM and NAND. Primary industry study. Semiconductor content of a rack, not the site.

• Google / Nikhil Cherian, 1 Sep 2026: more than 75% of an AI server hardware BOM. Product scope: high-performance memory — HBM + advanced DRAM, not necessarily NAND/SSD. Primary hyperscaler source at SEMICON Taiwan. Server hardware BOM, not rack or site CapEx.

• Deloitte, 28 Jul 2026, citing SemiAnalysis Apr 2026: approximately 30% in 2026 / 36% in 2027. Hyperscaler investment definition; tertiary path. Hyperscaler data-center investment.

• TrendForce, 25 Aug 2026: 47% in 2026 / 68% in 2027. Product scope: DRAM + NAND. Primary market-research note. Share of major-CSP CapEx.

Table 1. The Deloitte / SemiAnalysis row is labeled tertiary on purpose. A separate Morgan Stanley rack-BOM figure circulating through third-party reprints is omitted for the same reason.

Google’s SEMICON Taiwan 2026 remark is the new primary-source layer. Nikhil Cherian, senior director of supply-chain infrastructure at Google Cloud, said multimodal and mixture-of-experts architectures have pushed AI computing from compute-bound to memory-bound, and that high-performance memory now accounts for more than 75 percent of an AI server’s hardware BOM. The official presentation page carries the same figure. That is not a confirmation of TrendForce’s 68 percent of CSP CapEx. It is confirmation that, on a server BOM, HBM and advanced DRAM can already dominate.

Figure 2. Epoch AI’s stylized 1 GW model (14 May 2026) puts upfront CapEx near US$38B and annualized TCO near US$8.5B, of which servers are about US$5B and energy about US$0.6B. The split is a MemoryCapex reading of the public model. The model does not isolate memory and should not be forced to.

3. The TrendForce identity that has to sit on the page

If the 25 August TrendForce claims are taken together, and if the CSP universe and CapEx definition are held constant, the arithmetic is not optional. Let major-CSP CapEx in 2026 equal C. Memory spending in 2026 is 0.47C. CapEx in 2027 is 1.50C. Memory spending in 2027 is 0.68 × 1.50C, or 1.02C. The 2027 memory-dollar pool is therefore about 2.17 times the 2026 pool — a 117 percent increase. Under those assumptions, 2027 memory dollars roughly equal all of 2026 major-CSP CapEx.

The same note says combined server DRAM and HBM bit supply grows about 27 percent in 2027. That 27 percent and the 117 percent dollar increase do not share a universe: one is server DRAM plus HBM bits, the other is DRAM plus NAND dollars. A blended ASP cannot be recovered from those two published scopes. What can be said is narrower and still useful. A 68 percent path requires a very large price-and-mix contribution on top of bit growth. If long-term agreements cap server DRAM and NAND loosens in the second half of 2027, as TrendForce itself sketches, the 68 percent path is internally strained.

Figure 3. MemoryCapex calculation on TrendForce’s published percentages only, assuming an unchanged CSP universe and CapEx definition. This is a scenario identity, not an observed print.

Some CSPs already reported negative free cash flow in 2026. A memory bill the size of last year’s entire capital budget is not only a supplier windfall. If it prints, it is a late-cycle warning for the buyer’s program. That is the investment reading of 68 percent: not “the bull case continues,” but “the buyer’s constraint is now visible in the budget.”

4. Cycle Call Ledger

Figure 4. Subjective working weights as of 5 September 2026. Revisit after 3Q26 supplier results and Micron’s fiscal fourth quarter.

• Base: constraint holds; 68% path misses — 45%. What should be visible: DRAM tight through 2027; NAND eases from 2H27; CSP CapEx still up but slower; blended memory share rolls off the peak. Falsifier: a 2027 CSP-share print above 65% with NAND still tight.

• Supplier bull: HBM carries mix — 25%. What should be visible: 2027 HBM contracts land in the upper half of +70–140%; HBM profit per wafer reclaims the lead over RDIMM. Falsifier: failed HBM4 talks, or supplier margins falling without a mix explanation.

• Demand destruction: redesign becomes a pause — 20%. What should be visible: two consecutive cuts in hyperscaler CapEx guidance, or a drop in AI-rack orders. Falsifier: CapEx growth holds even with elevated memory prices.

• LTA as a peak tell — 10%. What should be visible: ceilings arrive with weaker floors, thinner volume commitments, or shorter tenors. Falsifier: LTA renewals reset floors higher, not ceilings.

5. Three memory economies

HBM

HBM sits next to the accelerator. Its job is bandwidth, not cheap capacity. TrendForce estimates that HBM will take about 18, 22 and 30 percent of the three major suppliers’ DRAM wafer input at year-end 2025, 2026 and 2027, while producing only about 8, 9 and 13 percent of DRAM bits. At Hot Chips 2026, Micron said the combined overhead of architecture, packaging and manufacturing means HBM3E consumes about three times as much silicon as DDR5 for the same capacity. That comparison is wafer area and resources per equivalent capacity. It is not a packaging-cost or finished-product-cost multiple. The point was restated in 2026, not only in 2024 commentary.

Figure 5. Source claim: TrendForce year-end estimates for Samsung, SK hynix and Micron. Not factory-audited wafer starts.

From early 3Q26, TrendForce says NVIDIA has been evaluating Rubin Ultra HBM options beyond the original 12-Hi HBM4e baseline: 8-Hi HBM4e, 12-Hi HBM4 and 8-Hi HBM4. The final specification has not been set. TrendForce also says the generation’s primary objective is higher I/O speed; raising GPU shipment volume is secondary. If NVIDIA cuts stack height, that is not only a supply workaround. It is a strategic choice between next-generation I/O and the number of GPUs that can be built from scarce wafers. Secondary reports have circulated a 192 GB SKU figure. MemoryCapex has not independently verified the originating desk and does not put that number in the main table.

In 1H26, some CSPs and server OEMs cut RDIMM mixes from 96 GB or 128 GB modules toward 32 GB or 64 GB. NVIDIA halved SOCAMM capacity on Vera Rubin Superchip modules after judging that LPDDR5X tightness is likely to last through 2027.

System DRAM

TrendForce estimates that HBM and RDIMM together absorb about 51 percent of DRAM bit supply in 2026. Dell Technologies chief operating officer Jeff Clarke, on the company’s recent earnings call, described the constraint as “DRAM, DRAM, DRAM, followed by NAND, NAND, NAND.” The quote is from coverage of that call in early September 2026. AI is pressing DRAM from both sides: direct HBM demand rises, and the wafer pool left for conventional server DRAM shrinks.

NAND and enterprise SSD

On 1 September 2026 TrendForce put top-five enterprise SSD revenue at US$37.59 billion in 2Q26, up 103.6 percent quarter on quarter. Samsung US$14.35 billion; SK hynix Group US$8.63 billion; Micron US$6.98 billion; Kioxia US$4.64 billion; SanDisk US$2.98 billion. Micron grew 126.3 percent, the fastest of the five. The same research house expects DRAM to stay tight in 2027 while NAND likely eases in the second half as new capacity arrives. The combined 47 / 68 percent figure should not be read as one cycle.

Figure 6. Source: TrendForce, 1 September 2026.

6. Prices, LTA ceilings, and an ambiguous signal

TrendForce’s published contract path is: server DRAM up a cumulative 64 percent in 2H25 and about 270 percent in 2026; enterprise SSD up about 35 percent then 235 percent; HBM contracts possibly up 70–140 percent in 2027. An independent non-TrendForce check sits in Micron’s Form 10-Q for the quarter ended 28 May 2026: DRAM average selling prices rose in the low-260 percent range year on year, while bit shipments rose only in the low-20 percent range. Most of that revenue jump is price, not volume.

Figure 7. Source claim: TrendForce, 25 August 2026. Three windows, not one comparable bar. 2026 panels are cumulative annual estimates.

From 2Q26, some U.S. CSPs signed multi-year agreements with price ceilings. TrendForce therefore sees 3Q26 server DRAM contracts up 13–18 percent quarter on quarter — still rising, no longer exploding. Price ceilings reduce suppliers’ upside convexity. They are not, by themselves, a peak signal. An LTA can cap price and still deliver volume commitment, allocation certainty, a deposit, a floor, or a multi-year book. The peak-tell reading becomes stronger only if ceilings arrive with weaker floors, thinner volume commitments, or shorter tenors.

Both the “shape of pricing power changed” reading and the “internal 2027 view is softer than the public number” reading remain live. The public TrendForce note confirms the ceilings and the 13–18 percent 3Q26 print. It does not adjudicate the rest of the contract.

7. Four economic layers — and actual P&L

Bit share, wafer share, revenue share and gross profit per wafer do not have to move together. TrendForce reports that in 1Q26, revenue and profitability per wafer for a 64 GB DDR5 RDIMM overtook HBM, because annual HBM contracts did not keep pace with the conventional DRAM spike. That inversion strengthened suppliers’ incentive to reprice HBM4 in 2027 negotiations. The public note confirms the inversion and that 2027 talks moved to HBM4. It does not say the talks “snagged” on that gap.

• Bit share: physical volume. Common trap: makes HBM look smaller than its factory cost.

• Wafer share: pressure on fab capacity. Common trap: treated as if it were revenue.

• Revenue share: price and mix. Common trap: treated as if it were durable profit.

• Gross profit per wafer: supplier value capture. Common trap: least visible in public narrative.

Without supplier P&L, that framework stays theoretical. The verified prints as of 5 September 2026 are listed below as separate observations. They are not one comparable metric.

• SK hynix, 2Q26 calendar: 76% operating margin; revenue KRW 79.32 trillion; operating profit KRW 60.54 trillion. Source: company release, 29 Jul 2026.

• Samsung Device Solutions, 2Q26 calendar: chip-unit operating profit about KRW 89.2 trillion on revenue about KRW 127.5 trillion; operating margin about 70%. Source: company print; Aju Press, 29 Jul 2026.

• Micron, FQ3-26 ended 28 May 2026: GAAP gross margin 84.6%; non-GAAP 84.9%; Cloud Memory GM 83%; Core Data Center GM 87%. Source: Form 8-K / press release, 24 Jun 2026.

• Micron guidance, FQ4-26: revenue about US$50B ±1; gross margin about 86%. Source: same release.

Figure 8. Metric cards, not a ranked bar chart. Operating margin and gross margin are different objects. They show that value capture in 2026 is an observation. Durability into 2028 is still an assumption. Micron’s fiscal fourth quarter is the next scheduled test.

8. Who collects the memory dollar

A larger memory share of AI infrastructure enlarges the addressable pool across HBM, system DRAM and enterprise NAND. That is not a stock ranking.

• Samsung: HBM, server DRAM and enterprise NAND exposure; 2Q26 SSD revenue leader. Broadest product surface; conversion to profit depends on yield and mix.

• SK hynix / Solidigm: strong HBM and server DRAM; second in 2Q26 SSD; 2Q26 HBM revenue share 50% according to Counterpoint. Both ends of the hierarchy; 76% operating margin in 2Q26.

• Micron: strong and expanding HBM; server DRAM; fastest 2Q26 SSD growth; Counterpoint 18% HBM revenue share in 2Q26. Present in all three buckets; FQ4 tests margin durability.

• Kioxia / SanDisk: no HBM and limited server DRAM; fourth and fifth in 2Q26 SSD. Levered to whether flash stays an active inference tier.

For SK hynix, a July 2026 Form 424B4 associated with the group cites IDC at 56.4 percent HBM revenue share in 1Q26. Counterpoint puts 2Q26 at 50 percent, Samsung 33 percent and Micron 18 percent. Those are revenue shares, not bit shares or profit shares.

9. Bull counter-case

The opposing reading of the thesis is that 68 percent is not a denominator problem. It is a path-dependence problem. If 2027 memory dollars equal all of 2026 CSP CapEx, configuration cuts may not be enough and the program itself shrinks. The 2018 memory cycle ran a smaller version of the same loop: high prices, high margins, then demand destruction.

MemoryCapex does not yet have a complete timing model for the shift from redesign to pause. The operational tell used here is two consecutive cuts in CapEx guidance among the four large U.S. CSPs, or a drop in AI-rack orders at Dell and peer OEMs. Until that evidence prints, a program pause is a case, not the base case.

Source-concentration risk is high. Almost the entire contract-price path in this note comes from TrendForce. The independent anchors today are Micron’s 10-Q and SK hynix’s official margin. The next anchors should be those companies’ 3Q26 prints and, if published cleanly, a DRAMeXchange contract series.

Wafer-fab equipment is a second-order read-through only: buyer CapEx becomes memory revenue, memory revenue becomes memory CapEx, memory CapEx becomes equipment demand. That chain belongs in one sentence. It is not a separate investment thesis in this note.

10. Falsifiable dashboard

• DRAM + NAND share of major-CSP CapEx: break threshold: full-year 2026 print below 35%, or a 2027 print above 65% with NAND still tight. Window: each TrendForce update.

• Server DRAM contract: break threshold: negative quarter-on-quarter change in 4Q26 without an inventory explanation. Window: 3Q26–1Q27.

• Enterprise SSD contract: break threshold: decline of more than 15% quarter on quarter before mid-2027 while HBM is still tight. Window: 2H26–2H27.

• Rubin Ultra configuration: break threshold: generation ships at 12-Hi HBM4e with no leaner SKU. Window: NVIDIA or ODM disclosure.

• Supplier margin: break threshold: compression of more than 15 percentage points quarter on quarter at SK hynix or Micron without a mix explanation. Window: 3Q26 and FQ4-26.

• Four-CSP CapEx guidance: break threshold: two consecutive guidance cuts. Window: 2026–2027 earnings.

• LTA structure: break threshold: ceilings with weaker floors, thinner volume, or shorter tenor. Window: 2H26–2027 renewals.

11. Central message

If an AI data center invests one hundred dollars, how many of those dollars reach memory? It depends on what the hundred dollars are, and on what “memory” includes. On a GB200-era full-site build, Bernstein’s public coverage implied about three to four dollars. The same slice after 2026 prices can approach ten dollars. On a Vera Rubin rack, memory plus storage is about thirty-five dollars of the rack and about twenty-four to twenty-five dollars of a modeled one-gigawatt stack. On an AI server hardware BOM, in Google’s wording for high-performance memory, more than seventy-five dollars. On a CSP treasury, in TrendForce’s wording for DRAM plus NAND, forty-seven to sixty-eight dollars.

Those figures describe one market through different denominators and different numerators. The direction is the same. Memory is no longer priced after the GPU is chosen. It now affects how many GPUs can be bought, how a rack is designed, what a token costs, and whether the whole CapEx program can be sustained.

The investment question is therefore not “does memory take more of CapEx?” It is whether a larger dollar pool, slower bit growth and a customer already cutting memory per unit of compute still convert into durable gross profit and free cash flow for the manufacturers. Today’s SK hynix and Micron prints say value capture has begun. The 1.02C identity says the same capture, if it reaches 68 percent of 2027 CSP CapEx, can break the buyer’s program. That print would be a late-cycle warning, not a reason to extend the supplier bull case by another year.

The investment question

Of each new dollar spent on AI infrastructure, how much reaches memory — and how much of that, after price, mix, LTA ceilings and customer redesign, becomes durable gross profit and free cash flow? A realized 68 percent path answers the first half of that question. It does not answer the second.

Sources and method

Confirmed facts in this note are limited to figures that can be tied to a dated primary document or to a named research-house press note. Everything else is labeled as a source claim or as MemoryCapex interpretation. Percentages taken from different denominators and different product scopes are never added or averaged.

1. Epoch AI, “Total cost of ownership of a one-gigawatt AI data center,” 14 May 2026.

2. SIA and Deloitte, “Powering AI: The Semiconductor Ecosystem at the Foundation of Data Centers,” 1 June 2026.

3. Bernstein via Investing.com and November 2025 coverage: foundry / memory / WFE each about 3–4% of GB200-era data-center CapEx, about US$1.0–1.2B per GW.

4. Bernstein via mid-2026 coverage: Vera Rubin rack about US$9.1M; memory + storage about US$3.2M (~35% of rack; ~24–25% of a modeled US$47B per GW).

5. Deloitte Insights, “Memory chip crunch greater than expected,” 28 July 2026; 30/36 percent cited to SemiAnalysis, April 2026.

6. TrendForce, 25 August 2026, 47/68 percent and the contract-price path.

7. TrendForce, 4 August 2026, Rubin Ultra configuration review, I/O priority, 50–60 percent HBM bit growth in 2027.

8. TrendForce, 30 July 2026, DRAM/NAND divergence in 2027.

9. TrendForce, 1 September 2026, enterprise SSD revenue in 2Q26.

10. TrendForce, July 2026, 3Q26 server DRAM contracts +13–18% QoQ under LTA ceilings.

11. TrendForce / official reprints, 1Q26 HBM wafer-revenue inversion versus 64 GB DDR5 RDIMM.

12. SEMICON Taiwan 2026 presentation and Commercial Times / TrendForce News, 1–2 September 2026: Nikhil Cherian; high-performance memory more than 75% of AI server hardware BOM.

13. ServeTheHome and Tom’s Hardware coverage of Micron at Hot Chips 2026: approximately 3× wafer area / resources for HBM3E versus DDR5 at equivalent capacity.

14. SK hynix Newsroom, 29 July 2026: official 2Q26 results.

15. Samsung Electronics 2Q26 results; Device Solutions figures via Aju Press, 29 July 2026.

16. Micron Form 8-K and press release, 24 June 2026; Form 10-Q for the quarter ended 28 May 2026.

17. Dell Technologies earnings call; Jeff Clarke, 1–2 September 2026.

18. Counterpoint Research, 2Q26 HBM revenue share: SK hynix 50%, Samsung 33%, Micron 18%.

19. SEC Form 424B4 associated with the SK hynix group, July 2026, IDC 1Q26 HBM revenue share.

#HBM #DRAM #NAND #AIMemory #AIInfrastructure #MemoryPricing #MemoryCycle #EnterpriseSSD #NVIDIA #VeraRubin #Micron #Samsung #SKHynix

Report parameters

Period: 2Q26 (calendar)
Published: Sep 1, 2026
Primary source: TrendForce press release
Coverage: Top five suppliers
Combined revenue: US$37.59B
Sequential change: +103.6% QoQ
Bit / ASP split: Not disclosed
Credibility: 8 / 10 · Importance: 9 / 10

Credibility reflects source type, not source quality: these are one research firm’s supplier estimates, released publicly as a preview of paid data, with no vendor confirmation and no unit disclosure. Importance reflects the breadth of read-through across the NAND supply chain.

Combined enterprise SSD revenue at the five largest suppliers reached US$37.59 billion in the second quarter of 2026, up 103.6% sequentially, TrendForce reported on September 1. Samsung led with $14.35B, holding a 38.2% share. SK hynix Group, including Solidigm, took $8.63B. Micron grew fastest in the group, up 126.3% to $6.98B. Kioxia reached $4.64B and SanDisk $2.98B.

TrendForce attributes the increase to two forces moving together: higher NAND contract prices and stronger enterprise SSD shipment volumes, and expects demand to hold up in 3Q26 as cloud providers expand data-center capacity, generative AI agent services increase storage requirements, and NVIDIA GB-series deployments scale.

The rankings did not change. What changed is the shape of the curve and, underneath it, roughly two shared points.

Figure 1. Three quarters, 3.8× — top-five enterprise SSD revenue. Source: TrendForce quarterly enterprise SSD releases (Mar 13, Jun 11 and Sep 1, 2026). Segment values in US$ billion; totals rounded.

Revenue and QoQ per TrendForce. Share and Δ share are MemoryCapex calculations from the same figures.

What the release does not contain

TrendForce names higher contract prices and stronger shipment volumes without weighting them. Its own earlier release supplies the weights. In 1Q26, enterprise SSD contract prices rose roughly 80% amid historically low inventory — and SanDisk grew bit shipments about 20% in a quarter when its enterprise SSD revenue more than tripled.

Where the split is visible, the price-and-capacity mix did nearly all the work. A 103.6% revenue quarter is not evidence of a 103.6% demand quarter.

This is the single number that decides how the quarter should be read, and it is the one number nobody has published. Any thesis built on this release is provisional until enterprise bit growth is disclosed.

Measuring the broadening properly

The claim worth testing is that AI capital spending is reaching memory through a wider set of channels than HBM alone. There is a clean way to measure it. TrendForce’s separate August 18 release put total NAND brand revenue at $68.87B in 2Q26, up 77%. Enterprise SSD accounted for 54.6% of all NAND brand revenue, up from 47.4% a quarter earlier — the first time the segment has cleared the half mark.

That 7.2-point mix shift, not the doubling, is the actual measure. The rest of NAND still grew 53% sequentially into visibly weak smartphone and PC demand, which is repricing rather than demand. The broadening is real. It is roughly a third the size the headline implies.

Figure 2. Enterprise SSD as a share of each supplier’s NAND revenue, 2Q26. Source: MemoryCapex calculation from TrendForce enterprise SSD (Sep 1, 2026) and NAND brand revenue (Aug 18, 2026) releases.

Two drivers the headline hides

HDD substitution. TrendForce named nearline HDD supply constraints as one of three drivers of the enterprise SSD ramp in its 4Q25 report, alongside AI inference workloads and server refresh. That component is displaced demand, not new demand, and it unwinds when HDD supply normalizes. Attributing the entire ramp to AI capital spending is an attribution error that the release itself does not make.

Capex starvation. TrendForce’s NAND release notes that capital expenditure prioritization toward DRAM and HBM is limiting new NAND capacity additions. This is the strongest argument for pricing durability in the whole data set, and it appears in neither the enterprise SSD headline nor in most coverage of it. Incumbents are not short of intent to add NAND capacity; they are short of capex that HBM has first claim on.

The share story is Micron versus SK hynix

Samsung’s share was unchanged to a tenth of a point at 38.2%. The only material movement in the quarter was roughly 2 points transferring from SK Hynix Group to Micron. SK hynix grew 86.0% — the slowest in the group — despite Solidigm’s position in ultra-high-capacity QLC, the exact product category TrendForce credits for the segment’s growth. That tension deserves an explanation before anyone treats Solidigm as the franchise asset in high-capacity enterprise QLC.

Micron’s technology commentary aligns with its own disclosure: in results for the quarter ended May 28, the company confirmed that its G9-based PCIe Gen6 SSD had entered high-volume production and that shipments of its 245TB QLC SSD had begun. These are two different products — the Gen6 drive is TLC, the 245TB drive is QLC on Gen5 — and neither carried meaningful revenue in the calendar quarter TrendForce is measuring. There are 2027 revenue events being confirmed early.

The risk is not only in China

TrendForce warns that rising output from Chinese suppliers and growth in China’s domestic cloud market could reshape the competitive landscape. Two additions belong beside it. The growth rate — +51.7%, +86.1%, +103.6% — accelerates every quarter, a signature of a price shock; adoption curves decelerate. And Samsung’s cited growth driver is QLC on a 176-layer node, a fully depreciated generation. Excellent for gross margin today, and a reminder that the supply response here is cheaper to mount than a shortage narrative implies.

MemoryCapex Take

The signal is the mix, not the multiple

Enterprise SSD crossed half of all NAND brand revenue for the first time this quarter. That is the finding worth keeping: the transmission of AI capital spending into memory is broadening beyond HBM and server DRAM into storage, and it is now measurable as a mix rather than inferred from a growth rate.

We would not yet call the quarter structural. Two of the four forces behind it — the contract-price reset and HDD substitution — are cyclical, and a third, the PCIe 4.0-to-5.0 transition, is a one-time step rather than a recurring rate. Only the fourth, capex starved by HBM, argues for durability, and it is a choice suppliers can reverse when relative prices move.

The defensible claim is narrower than the headline: enterprise SSD is taking a durably larger share of NAND revenue, on pricing that is not yet proven durable. On positioning, Micron is the cleanest read-through — it took share, and 59% of its NAND revenue now sits in enterprise SSDs. Kioxia at 43% and SanDisk at 33% remain levered to the consumer cycle, which the market has decided no longer matters.

Falsification test: If enterprise SSD bit shipments grow by less than roughly 25% in 3Q26 while revenue stays near current levels, this is a price cycle wearing an AI costume — and normalized quarterly revenue sits far below $37.6B. We will revisit this piece against that number when it is published.

Sources. TrendForce, “Higher Prices and Shipments Drive Top Five Enterprise SSD Vendors’ Revenue to Nearly US$37.59 Billion in 2Q26,” September 1, 2026; TrendForce NAND Flash brand revenue release, August 18, 2026; TrendForce enterprise SSD releases, June 11, 2026, and March 13, 2026; Micron Technology fiscal Q3 2026 results, June 24, 2026

#EnterpriseSSD #NAND #QLC #AIInfrastructure #Micron #Samsung #SKHynix #Kioxia #SanDisk

MemoryCapex Investment Deep Dive | September 1, 2026

Executive thesis

The first phase of the AI-memory trade was relatively simple: more accelerators required more HBM, and the suppliers that could qualify and ship HBM at scale captured a disproportionate share of the upside.

That thesis is still valid. It is no longer sufficient.

Inference, longer context windows, retrieval-augmented generation and agentic AI are turning the Memory Wall from a single bandwidth problem into a four-variable constraint: bandwidth, capacity, power and cost per token. The industry is responding with faster local memory, tighter compute-memory integration, larger system-memory pools, flash-based context tiers and increasingly aggressive software techniques that reduce memory demand itself.

Bernstein’s recent memory work captures the broad direction: different AI workloads create different requirements across HBM, conventional DRAM, NAND and storage, widening the opportunity beyond a pure HBM cycle.[1][2] But the investment conclusion cannot stop at “more AI means more memory.”

AI is creating economic pressure for a deeper and more heterogeneous memory hierarchy, but history warns that new intermediate tiers are not automatically durable. Each tier must survive competition from improving adjacent layers and from software that reduces memory demand altogether.

That leads to the central MemoryCapex question for 2026–2029:

Which memory companies can convert AI workload growth into durable pricing power, gross profit per wafer and free cash flow after accounting for substitution, software efficiency, customer architecture and the memory cycle?

Figure 1. MemoryCapex’s five-path framework. Four paths add or reorganize hardware; Reduce lowers the bytes required per token. The paths can coexist physically while competing economically.

1. Start with the cycle, not the technology roadmap

Any technology thesis in memory is incomplete without locating the industry in the cycle.

The reported economics in mid-2026 are extraordinary. Micron’s fiscal third-quarter revenue reached $41.46 billion. Cloud Memory gross margin was 83%, Core Data Center gross margin 87%, and Mobile & Client gross margin also 87%.[3] The last figure matters: this is not an AI-only upcycle. Pricing power has spread well beyond HBM and data-center products.

SK hynix reported a 76% operating margin in Q2 2026 — a different metric and therefore not directly comparable with Micron’s gross margins — while SanDisk reported an 84.6% gross margin in fiscal Q4 2026.[4][5]

The investor question has therefore changed. We are no longer asking whether memory earnings recover. We are asking how much of today’s scarcity economics survives normalization.

The stock market is already asking that question. In early July, major memory shares entered a technical bear market despite record earnings.[6] SK hynix later reported record absolute results but still disappointed expectations and sold off, a classic late-cycle signal: the marginal stock-price driver can shift from the level of earnings to the direction and breadth of earnings revisions even while reported fundamentals remain excellent.

MemoryCapex’s current-cycle read is late expansion/scarcity, not early recovery. That is not a call that the cycle has peaked. It means the downside mechanism has changed. The key risk is no longer weak demand today; it is future normalization in pricing, supply discipline, memory content per unit of compute, or investor willingness to capitalize unusually high earnings at prior-cycle multiples.

2. The Bernstein paradox: the memory bull case creates its own constraint

Bernstein’s June sector work contains one of the most important contradictions in the current memory thesis.[2]

According to the public summary of the note, conventional DRAM prices had risen roughly 4.5x since Q3 2025, while HBM pricing moved far less because annual contracts locked in earlier pricing. At that point in the cycle, conventional DRAM could generate more than twice the revenue and nearly three times the gross profit per wafer of HBM.

That does not mean suppliers can move committed HBM wafers back to commodity DRAM. HBM qualification, customer commitments, packaging constraints, and strategic relationships limit that response. The better interpretation is an opportunity-cost signal:

Higher conventional-DRAM gross profit per wafer
→ greater economic cost of allocating scarce DRAM capacity to HBM
→ stronger supplier pressure in 2027 HBM negotiations
→ Higher HBM ASPs are needed to restore relative economics.

Bernstein consequently expects HBM pricing to rise sharply in 2027 and placed its 2027 EPS forecasts for Samsung, SK hynix and Micron 25%–38% above the consensus available at the time.[2]

But the same pricing power creates the bear case. Bernstein estimates that rising HBM, conventional DRAM, and NAND could increase the cost of an equivalent AI data-center deployment by up to 30%.[2] This is a modeled incremental capex impact in the public summary — not evidence that memory itself accounts for 30% of the total AI system bill of materials.

The reflexive risk is straightforward:

The more profitable memory becomes, the stronger the incentive for accelerator designers, hyperscalers and model developers to reduce memory dollars per token.

That is why the new architectural paths are not simply a list of bullish memory technologies. Several are customer-funded responses to memory scarcity and cost.

3. What is already expected? Bernstein versus current market consensus

A structural bull case has little investment value if the market already discounts an even stronger outcome.

As of the article lock, analyst-consensus price targets remain highly constructive across much of the sector. Micron’s reference price is $958.73 compared with an average 12-month target of about $1,513. SK hynix’s reference price is KRW 1,653,000 (about $1,203) versus a consensus target near KRW 3,216,025 (about $2,341). Samsung’s reference price is KRW 260,000 (about $189) versus consensus near KRW 471,545 (about $343). SanDisk’s reference price is $1,566.70 versus consensus near $2,125. Kioxia closed at JPY 51,000 (about $319) on September 1 versus consensus near JPY 110,944 (about $694).[15][16][17][18][19]

The U.S.-dollar equivalents use IMF representative rates for September 1, 2026: KRW 1,373.6 per USD and JPY 159.77 per USD.[19]

Bernstein’s published targets are different. Its June targets were $1,300 for Micron, KRW 3,300,000 (about $2,402) for SK Hynix, and KRW 440,000 (about $320) for Samsung.[2] Bernstein maintained a negative view on Kioxia with a JPY 40,000 (about $250) target price, while the broader consensus is strongly positive.[18] Bernstein later raised SanDisk to $3,000.[20]

These are external analyst targets, not MemoryCapex ratings. They are shown only to indicate what may already be discounted in market expectations.

Figure 2. Targets are shown against a common reference price for each stock. Bernstein notes that current consensus estimates are not synchronized in date or methodology; this is an expectations snapshot, not a relative exit-multiple calculation.

A revealing tension appears in Micron. Bernstein’s June 22 target was $1,300 even though its 2027 earnings estimates for the major memory names were 25%–38% above the consensus available at the time. By late August, the broader Micron consensus target had risen to roughly $1,500.

Those observations are not perfectly synchronized, so they should not be used to calculate a precise relative exit multiple. But the direction is still informative: one of the sector’s most aggressive earnings forecasts did not translate into the most aggressive valuation target.

For a late-cycle investor, that is a warning that the debate is shifting from how high earnings can go to what multiple unusually high earnings deserve.

The valuation risk is therefore two-dimensional. Consensus can be wrong about earnings, but the market can also compress the multiple even if earnings remain exceptionally strong. In a cyclical industry, getting the earnings direction right is not enough if investors increasingly treat those earnings as peak-cycle rather than durable.

4. Why inference changes the demand mix — and why KV cache must be derived, not merely cited

Training and prefill remain compute- and bandwidth-intensive. HBM is central because the accelerator needs enormous local bandwidth.

Decode is different. Model weights, and especially the KV cache, are repeatedly accessed. As context length and concurrent users rise, KV cache capacity can become a system constraint.

A simplified decoder-only baseline makes the mechanism visible:

Raw KV bytes ≈ 2 × layers × KV heads × head dimension × cached tokens × bytes per value

Consider a hypothetical 80-layer baseline. This is not a named production model; it is a simplified decoder-only example designed only to show raw memory scale.

• 8 KV heads

• head dimension 128

• 128K active context

• one active sequence

At 16-bit precision, the raw KV cache is approximately 43 GB. FP8 reduces the raw storage to about 21.5 GB. A 4-bit representation reduces it to 10.7 GB, before implementation overhead.

At 32 concurrent full-context sequences, the corresponding raw 16-bit footprint is approximately 1.37 TB before implementation overhead.

Figure 3. Simplified raw KV footprint. Production systems can differ materially because of sharding, paging, cache sharing, GQA/MQA/MLA, eviction, quantization, and software implementation.

This scaling relationship explains why Bernstein can reasonably argue that KV cache may exceed model-weight memory in large deployments.[1] But whether it does so in a real system depends on architecture, context, concurrency, precision, and cache-management policy.

RAG adds persistent data: vector databases, embeddings and source datasets. Agentic AI adds longer-lived state, tool outputs, and multi-step context.

The economic question is therefore more important than “how much HBM can fit beside the GPU?”

It is:

Which data truly requires HBM, which can sit in system DRAM, which can move into NAND, and how much of that demand can software eliminate before new memory is purchased?

That produces five responses:

Widen / Shorten / Pool-Expand / Tier / Reduce.

5. Widen: HBM4 remains the highest-confidence near-term profit pool

The most direct response to the Memory Wall is more bandwidth besides compute.

The word “widen” is literal. HBM4 doubles the interface width from 1,024 to 2,048 I/O. Micron’s HBM4 combines that wider bus with pin speeds above 11 Gb/s, delivering more than 2.8 TB/s per stack.[7]

Figure 4. Widen and shorten attack different physical constraints. The d-Matrix energy comparison is a company-reported early-silicon example for early silicon and should not be generalized to zHBM or HBC.

Micron’s 36GB 12-high HBM4 is in high-volume production for NVIDIA Vera Rubin.[7] Samsung began sampling 12-high HBM4E in May 2026 at a stable 14 Gb/s pin speed, scalable to 16 Gb/s, with up to 3.6 TB/s per stack and 48GB capacity.[8] SK hynix began volume HBM4 shipments in Q2.[4]

This is the highest-confidence part of the thesis because it is attached to shipped or sampled products, customer qualification, and visible revenue.

Yet HBM economics are becoming more complex. The relevant metric is not HBM bit growth alone. It is the gross profit per wafer after DRAM opportunity cost, advanced-packaging cost, yield loss, and contract pricing.

In a scarcity cycle, conventional DRAM can temporarily out-earn HBM on a wafer basis. In a normalized cycle, HBM’s structural premium may reassert itself. Investors need to track relative economics rather than assume it.

NVHBM: bandwidth, platform control, and accelerator shoreline

NVIDIA’s NVHBM architecture moves the custom memory controller and PHY into the HBM base die. NVIDIA claims up to 30% more bandwidth per stack, 15% lower HBM power, up to 67% less PHY/support area, and up to 25% more compute-die area for additional XPU capabilities versus standard HBM4E.[9]

The last point matters because NVHBM attacks a less visible scarcity: accelerator shoreline.

Every wide off-die interface consumes finite die-edge routing and PHY area. This is the same physical constraint that weakens some proposals for accelerator-attached CXL. NVIDIA attacks it from the opposite direction: compress the HBM interface footprint and move more memory-interface intelligence into the base die.

For memory suppliers, customization raises engineering complexity and qualification barriers — which is positive for defensibility. But it can also shift bargaining power. If NVIDIA defines the controller, interface, and platform architecture while qualifying multiple memory vendors, more HBM volume does not guarantee that memory suppliers capture the full customization premium.

Custom HBM can raise switching costs while simultaneously shifting part of architectural value capture toward the accelerator-platform owner.

6. Shorten: the physics is data-movement energy

The second response is to move each bit a shorter distance rather than simply moving more bits across a wider interface.

The underlying unit is picojoules per bit (pJ/bit).

At Hot Chips 2026, d-Matrix presented early silicon for its Raptor architecture, which bonds a 4nm compute die face-to-face onto custom DRAM. The company reported 0.37 pJ/bit across the short vertical interface versus roughly 2.4 pJ/bit for moving data into an HBM4 base die in the comparison it presented. Raptor targets more than 100 TB/s from 32GB per card.[21]

Those numbers should not be generalized to Samsung zHBM or Qualcomm HBC because directly comparable pJ/bit disclosures do not exist across all three architectures. But Raptor demonstrates the physical mechanism:

Shortening the data path can reduce the energy tax of moving each bit by multiples — at the cost of capacity, integration complexity and yield risk.

For memory investors, that matters because the profit pool can migrate. If 3D integration materially reduces data-movement energy, value can shift from “more HBM bandwidth” toward bonding, logic-memory co-design, and package-level yield execution.

7. zHBM: when memory becomes part of the accelerator

zHBM deserves more weight than a typical future-product paragraph because it changes where memory sits relative to compute.

Conventional HBM uses 2.5D integration: the GPU or XPU and HBM stacks sit side by side on an interposer. Samsung’s zHBM concept moves toward true 3D integration by stacking HBM vertically above the accelerator, minimizing data-movement distance.[10]

Samsung’s August FMS disclosure described zHBM as a concept model, rather than a commercial product. The company said a next-generation interface incorporating zHBM could deliver approximately 8x the performance of HBM5, more than 10x the memory density, roughly 3x the energy efficiency, and more than a 50% reduction in thermal resistance. Samsung also said customer-specific IP could be integrated into the interlayer between memory and the accelerator.[10]

At SEMICON Taiwan on September 1, industry reporting cited a different comparison baseline — HBM4E — for another set of zHBM system-level targets. Those figures should not be combined with the FMS disclosure in a single quantitative model because the baselines differ and both remain architectural targets rather than measured production specifications.[22]

The strategic implication is Samsung-specific.

Samsung owns DRAM, foundry capability, and advanced packaging. In HBM,4 it already designs a 4nm logic base die internally. If post-HBM5 architecture moves toward deeper 3D logic-memory integration, Samsung’s vertical integration could become more valuable than it is in today’s HBM market.

But integration cuts both ways. zHBM highlights the importance of hybrid copper bonding, wafer bonding, thermal management, customer-specific co-design, and package-level yield. A defect or thermal problem can destroy more value in a more integrated package — a form of yield coupling.

Samsung says first-generation zHBM specifications are still being defined and expects products based on them starting in 2029.[23]

That makes the correct investor hierarchy:

• HBM4: current earnings

• HBM4E: near-term qualification, share and margin battle

• NVHBM / custom HBM: emerging interface architecture and bargaining-power shift

• zHBM: 2029+ strategic optionality with potentially large Samsung-specific consequences

The central uncertainty is not whether zHBM can improve system performance. It is whether Samsung can translate that integration into higher gross profit per accelerator after yield, thermal, and packaging costs.

8. HBC: a direct challenge to the assumption that more AI compute means more HBM

Qualcomm’s High Bandwidth Compute attacks the same data-movement problem from another direction.

Qualcomm describes HBC as a purpose-built near-memory architecture that bonds compute with highly accelerated memory bandwidth in a 3D-stacked silicon solution. Its accelerator roadmap starts from an LPDDR5X-based AI200 platform, and Qualcomm explicitly cites its LPDDR expertise as part of the foundation for HBC.[11]

The key point is that HBC is not simply another HBM generation.

Qualcomm says AI250 with HBC Gen 1 is designed for 133 TB/s of effective memory bandwidth per card, an 18x increase over the AI200, with sampling targeted for mid-2027. Qualcomm also claims a 6x bandwidth-per-watt advantage versus HBM-based solutions when normalized at the card level.[11]

Those figures must not be compared directly with Micron’s 2.8 TB/s per HBM4 stack. HBC is an effective card-level metric; HBM4 is a physical per-stack bandwidth metric.

The investment signal is clearer than the headline number: Qualcomm is explicitly trying to improve inference economics without paying the conventional HBM architecture’s full data-movement cost.

If HBC succeeds, AI workload can grow rapidly while HBM content per unit of compute grows more slowly.

That is the difference between AI growth and memory-revenue growth.

9. Pool / Expand: The requirement is real; CXL is not yet the proven winner

AI systems cannot place unlimited HBM beside every accelerator. Working-memory capacity outside the accelerator must grow.

The strongest current evidence is not CXL. It is system DRAM already shipping. Micron has 192GB SOCAMM2 in high-volume production for Vera Rubin, and SK hynix has also reported growth in SOCAMM2.[7][4]

CXL is one proposed mechanism for expansion, pooling, and disaggregation. The CXL Consortium continues to promote memory tiering, KV-cache offload, and disaggregated memory for AI inference.[12]

But the framework should remain technology-neutral.

SemiAnalysis argued in 2024 that PCIe/CXL faces shoreline-efficiency constraints and that NVIDIA GPUs did not expose CXL in the main AI path.[13] That analysis is now more than two years old and cannot by itself prove the 2026 adoption picture. Nor does promotion by a standards consortium prove commercial deployment.

MemoryCapex therefore defines the path as Pool / Expand, with CXL as one candidate.

The bullish memory point is broader: if inference working sets expand, system DRAM can become a larger part of AI memory content even if CXL never becomes the dominant accelerator-memory fabric.

10. Tier: HBF is a real NAND opportunity — but history says middle tiers often die

High Bandwidth Flash may be the most important new architecture for NAND investors.

SK hynix and SanDisk released the first open HBF specification through OCP in August. The specification supports up to 512GB using 8-high or 16-high NAND stacks and three bandwidth grades ranging from roughly 0.4 to 3.0 TB/s. It also uses UCIe to connect HBF into heterogeneous processor packages.[14]

That means HBF is better understood as a near-compute NAND chiplet tier, rather than a conventional storage device.

But the missing specifications may matter more than the maximum bandwidth.

NAND still has microsecond-class array-read behavior versus nanosecond-class DRAM, and it has finite program/erase endurance. These properties are especially relevant when HBF is used for workloads involving continuous KV-cache updates rather than for largely read-heavy model weights or context.

SanDisk and SK hynix have not disclosed production-level HBF latency or endurance figures.

That silence is investment-relevant.

HBM-class bandwidth should not be interpreted as HBM-class memory behavior.

The success test for HBF is therefore not simply peak bandwidth. It is whether software can hide NAND latency, manage write amplification and endurance, and identify a workload large enough to justify a permanent new tier.

The historical counter-case: intermediate tiers can be squeezed out

The strongest challenge to the “deeper hierarchy” thesis is history.

Intermediate memory technologies can be squeezed from both directions: the fastest tier keeps getting faster while the cheapest tier gets denser and “good enough.”

Intel Optane is the most relevant warning. 3D XPoint was designed to occupy the gap between DRAM and NAND, reached commercial production, and had the backing of a major platform vendor. Intel still ended future Optane development, saying the technology had become impractical to deliver at the required scale as a single-source supplier.[24]

Figure 5. HBF has structural advantages over Optane — it uses conventional NAND economics and an open chiplet-oriented standard — but it still must prove latency, endurance, software and total-system economics.

Why might HBF be different?

Unlike Optane, HBF does not require a novel memory medium or a separate manufacturing ecosystem. It leverages high-volume NAND, open OCP standardization, and UCIe-based chiplet integration.[14]

Those are genuine structural advantages.

They do not guarantee survival.

Google and Tenstorrent’s participation in standardization is evidence of ecosystem interest, not proof of production design wins or volume adoption.

The burden of proof is on HBF to show that the incremental system value of a new tier exceeds the software, integration, and operational complexity of maintaining it.

The new falsifier is explicit:

If HBF/CMX remain confined to narrow workloads while improving HBM, system DRAM and conventional enterprise SSDs absorb the same demand, the AI memory stack may polarize rather than deepen.

11. CMX: validation of a context tier, not automatic validation of HBF economics

NVIDIA’s CMX validates the need for a lower-cost context tier.

NVIDIA describes CMX as a flash-based context-memory storage architecture for KV cache and long-context inference. In target workloads, it claims up to 5x tokens per second and 5x power efficiency versus traditional storage approaches.[25]

Those are system-level token and power metrics, not raw memory-bandwidth claims.

For NAND, CMX is structurally positive because it moves flash closer to active inference.

For the memory sector as a whole, it is not automatically positive. If CMX removes expensive context from DRAM/HBM to cheaper NAND, total memory bits can rise while memory dollars and gross profit per accelerator shift downward in the hierarchy.

12. Reduce: the fastest substitution path operates above the silicon

The four hardware paths treat memory pressure as something hardware must accommodate.

That is incomplete.

Model architecture and serving software can reduce the amount of memory required per token before any additional HBM, DRAM, or NAND is purchased.

Examples include:

• GQA / MQA: reduce the number of key/value heads relative to standard multi-head attention; the KV-cache reduction is model-dependent and can be several-fold.

• KV-cache quantization: moving from 16-bit storage to FP8 roughly halves raw storage; moving from FP8 to 4-bit can approximately halve it again, subject to implementation and accuracy trade-offs.

• Sliding-window attention: fixed-window architectures can bound the active cache by the chosen window.

• PagedAttention: reduces fragmentation and redundant duplication and improves throughput by 2–4x at similar latency in the original vLLM work.[26]

• Prefix sharing / eviction / cache policy: improve utilization of installed memory.

NVIDIA notes that FP8 KV cache is already widely used in production and that NVFP4 can reduce the KV cache footprint by about another 50% compared to FP8.[27] Mistral showed that a fixed sliding-window implementation could use a rotating buffer and halve cache memory in its published 8K example.[28]

For memory investors, Reduce is the fastest substitution risk in the framework because software can diffuse in quarters rather than fab cycles.

Any memory-demand forecast that extrapolates KV-cache growth from context length alone is incomplete unless it also models reductions in KV heads, precision, residency, sharing and reuse.

The Jevons counterforce

Efficiency does not automatically mean lower total memory demand.

Lower memory cost per token can be reinvested into:

• longer contexts

• larger batch sizes

• higher concurrency

• richer agentic workflows

• more inference volume

Both forces can operate simultaneously:

Memory intensity per token falls while total memory consumption rises because the number and complexity of tokens expand faster.

The investment variable is therefore not model efficiency in isolation.

It is:

the net change in total memory bytes and memory dollars per unit of deployed compute.

13. Five value regimes: technology success and shareholder success are different things

Memory innovation can create at least five distinct economic outcomes:

Value regime: More memory dollars

Examples: HBM4, HBM4E, SOCAMM2

Memory effect: Mostly incremental

Investor question: Do ASP and yield support higher gross profit per wafer?

Value regime: Mix shift

Examples: HBF, CMX

Memory effect: More NAND bits; potentially fewer DRAM dollars

Investor question: Does total memory gross profit rise or merely migrate?

Value regime: Hardware efficiency

Examples: HBC, near-memory compute

Memory effect: Lower data-movement cost

Investor question: Does memory content per unit of compute fall?

Value regime: Platform capture

Examples: NVHBM, custom HBM

Memory effect: More co-design, more platform control

Investor question: Who owns the interface economics?

Value regime: Software substitution

Examples: GQA/MQA, KV quantization, PagedAttention

Memory effect: Fewer bytes per token

Investor question: Does workload growth outrun efficiency gains?

A useful simplified framework is:

Memory revenue ≈ workload growth × memory content per system × ASP × product mix

and

Memory gross profit ≈ revenue − wafer/fab cost − advanced-packaging cost − yield penalties.

A technology can increase workload growth while reducing memory content. It can increase the number of bits while shifting the mix from DRAM to NAND. It can raise ASPs until customer elasticity triggers substitution.

This is why gross profit per wafer, gross profit per accelerator, and memory dollars per deployed unit of compute matter more than bit growth alone.

14. Company positioning through 2029

SK hynix — strongest current HBM position, but expectations are already extreme

SK hynix combines HBM leadership with conventional DRAM, SOCAMM2 and NAND exposure through Solidigm, and participates in HBF standardization.[4][14]

The strength is breadth without sacrificing HBM leadership. The principal risk is the other side of that success: very high expectations, large capital requirements, and eventual margin normalization as Samsung and Micron close HBM gaps.

Key metrics: HBM4/HBM4E share, relative gross profit per wafer, capex discipline, SOCAMM2 growth and NAND mix.

SK hynix’s Indiana advanced-packaging facility is scheduled to begin next-generation HBM production in the second half of 2029, creating strategic U.S. capacity but also adding long-duration capital intensity.[29]

Micron — the cleanest observable AI-memory execution, with high cycle sensitivity

Micron now has HBM4, SOCAMM2 and PCIe Gen6 enterprise SSD in high-volume production.[7]

The strength is observable execution across HBM, system DRAM, and storage without relying on distant architecture roadmaps.

The risk is that current margins are exceptionally high and Micron has less publicly visible optionality today in zHBM/HBC-style 3D architectures than Samsung does.

Key metrics: Cloud/Data Center margins, 2027 HBM pricing, SOCAMM2 growth, capex and bit-supply discipline.

Samsung Memory — widest option set; execution determines whether breadth becomes value

Samsung has the broadest architecture portfolio: HBM4/HBM4E, conventional DRAM, NAND, enterprise SSDs, internally designed HBM logic base die, and the zHBM roadmap.[8][10]

Its strategic advantage is vertical integration.

If post-HBM5 architecture shifts toward 3D logic-memory co-integration, Samsung could control more of the value chain than its peers.

But technology breadth is not the same as economic value. Customer qualification, yield progression, base-die execution, and package-level gross profit determine whether breadth translates into market share and cash flow.

SanDisk — the purest listed bet on NAND moving closer to inference

SanDisk has no HBM exposure. Its upside comes from NAND economics and the possibility that HBF upgrades flash from passive storage into an active inference tier.

That optionality matters, but investors should keep the maturity distinction clear: HBF is still in the specification, sampling, and ecosystem formation stages.

The key monitoring items are HBF latency/endurance disclosure, production design wins, enterprise-SSD mix, and whether a new NAND tier earns structurally better gross margins than conventional storage.

Kioxia — the clearest reminder that industry structure and stock valuation are separate questions

Kioxia has strong NAND and enterprise-SSD exposure but no HBM hedge.

Current consensus is strongly positive, while Bernstein’s JPY 40,000 (about $250) target is materially below both the current share price and the consensus.[18]

That divergence is useful because it shows that the same NAND structural thesis can support very different stock conclusions depending on normalized NAND margins, valuation, and the expected duration of current pricing.

HDD — downstream capacity spillover, not the core Memory Wall thesis

Seagate and Western Digital can benefit if AI-generated data becomes a persistent exabyte demand.

But HDD sits far from the active inference path, and the evidence chain is weaker.

For MemoryCapex, HDD should remain a downstream capacity monitor rather than receive the same analytical weight as HBM, DRAM, and NAND.

15. 2026–2029: separate shipped earnings from strategic optionality

Figure 6. Evidence maturity, not a forecast of market share or revenue.

The investment horizon should be divided into three layers.

2026–2027: scarcity, HBM4 and repricing

HBM4 ramps. HBM4E qualification intensifies. System DRAM expands. 2027 HBM contract pricing becomes a core earnings variable.

This period is driven primarily by supply allocation, yield, contract economics, and customer qualification.

2027–2028: inference-tier experimentation becomes measurable

HBC sampling and HBF/CMX design activity can reveal whether the new architectures are incremental to memory content or substitutional.

The most important questions will be:

• Does HBF disclose competitive latency and endurance?

• Does HBC show real customer adoption?

• Do software-side reductions materially lower memory bytes per session?

• Does system DRAM continue to grow even if CXL remains limited?

2029: architecture begins to matter more than today’s hierarchy

Samsung expects zHBM-derived products from 2029 onward.[23] SK hynix plans next-generation HBM production at its Indiana packaging facility in H2 2029.[29]

By then, the industry may compete as much on 3D integration, base-die logic, customer-specific IP, hybrid bonding, and memory-centric system design as on conventional stack specifications.

But the historical warning remains: not every new tier survives long enough to become a durable profit pool.

16. Scenario framework through 2029

MemoryCapex does not assign precise probabilities because the evidence does not support statistical confidence. The scenarios are ranked qualitatively.

Base case — structurally higher memory value, margins normalize from scarcity peaks

AI workload continues to grow strongly.

HBM4/HBM4E demand remains robust. System DRAM grows with inference. HBF/CMX create incremental NAND demand but remain selective rather than broadly substituting HBM. Software efficiency improves, but context length, concurrency,y and token volume grow fast enough to keep total memory consumption rising.

Supplier capex remains disciplined enough to avoid a severe glut.

Gross margins fall from 2026 extremes but remain above prior-cycle norms.

Bull case — workload growth outruns hardware and software efficiency

AI inference expands faster than memory-efficiency gains.

HBM repricing in 2027 sticks. HBM4E/custom HBM increases dollar content. System DRAM grows sharply. HBF creates a new high-value NAND tier rather than simply cannibalizing DRAM.

Supplier discipline remains unusually strong, and long-term agreements reduce downside cyclicality.

In this case, normalized margins can remain structurally above historical expectations.

Bear case — scarcity economics accelerate substitution and supply response

Memory cost per AI system becomes too high.

Hyperscalers and accelerator vendors are aggressively adopting HBC, HBF/CM, X, and software-side reductions. Memory bytes per active inference session fall faster than context length, concurrency, and inference volume rise.

At the same time, 2027–2029 are arriving faster than demand.

HBM shipments can still rise while gross profit per wafer compresses. NAND bit demand can grow while ASPs and margins normalize sharply.

Excellent technology adoption can coexist with poor stock returns.

A second bear-case path is tier collapse: HBF/CMX fail to earn a durable position because HBM/system DRAM improve above them while conventional NAND/SSD becomes sufficient below them.

17. The MemoryCapex cycle dashboard

The most important metrics to monitor are:

1. Gross profit per wafer: HBM vs conventional DRAM. The clearest direct measure of allocation economics.

2. Contract pricing: HBM, server DRAM, and enterprise NAND, especially for 2027 negotiations.

3. Supplier capex and bit-supply growth: whether scarcity becomes overbuilding.

4. HBM qualification and share shifts: SK hynix, Micron and Samsung.

5. System DRAM content: SOCAMM2 and server-memory content per AI node.

6. NAND in inference: HBF/CMX design wins, shipments, latency/endurance disclosure and enterprise-SSD mix.

7. Memory share of AI-system economics: customer response to memory dollars per token.

8. Stock/fundamental divergence: whether record earnings still produce negative share-price reactions.

9. zHBM milestones: customer engagement, Gen-1 specification, thermal results and packaging yield.

10. Consensus revision breadth: whether earnings revisions remain positive or begin narrowing before reported fundamentals weaken.

11. Memory intensity per inference session: KV precision, KV-head count, active context retention, prefix-cache hit rate and bytes per concurrent user.

12. Tier survival: whether HBF/CMX moves from specification and reference architecture into repeatable production workloads with attractive memory-vendor margins.

MemoryCapex view

The Memory Wall is not an engineering story that later becomes relevant to investors. It is increasingly the mechanism that determines the economics of memory.

Widen can increase memory dollars per accelerator through HBM.

Shortening can improve performance through zHBM, HBC or 3D logic-memory integration, while potentially reducing data movement and memory content per unit of effective compute.

Pool / Expand can broaden AI demand for system DRAM even if CXL itself never becomes dominant.

Tier can create the most important new NAND opportunity in years — but history shows that intermediate memory tiers survive only when their system-level economics justify the complexity.

Reduce can attack memory demand before new silicon is purchased, and it can move faster than any fab cycle.

That makes the next phase of AI memory neither a winner-take-all technology race nor a simple sector-wide bull case.

The competition is for durable value capture.

Today’s scarcity has pushed memory profitability to exceptional levels. That profitability is itself increasing the economic incentive for customers to redesign the stack around lower memory cost per token.

The paradox is central to the cycle:

The more profitable memory becomes, the more economically valuable it becomes for customers to use memory more efficiently.

Through 2029, the strongest memory company will not necessarily be the one with the most bits, the highest peak bandwidth, or the broadest roadmap.

The likely winners will combine:

technology leadership + supply discipline + durable pricing power + control of enough of the architecture to retain the economics after substitution.

For investors, the final question is no longer simply how the Memory Wall will be broken.

It is:

When it is broken, how much of the value created will remain in the income statements and free cash flow of SK hynix, Micron, Samsung Memory, SanDisk and the rest of the memory industry?

Methodology and evidence notes

This article is industry and investment research, not a recommendation to buy or sell any security. Site-wide ownership and conflict-of-interest disclosures apply.

Company technical-performance claims are identified as vendor claims where applicable. Bernstein’s research is used in public summaries because MemoryCapex does not have the full proprietary notes cited here. Third-party analyst ratings and price targets are presented as market-expectation evidence of market expectations, not as MemoryCapex ratings.

The original Widen / Shorten / Extend framing was inspired by the Market Signal video, The Future of AI Memory Is Taking Shape. MemoryCapex expanded the framework into Widen / Shorten / Pool / Tier / Reduce and added the cycle, substitution, tier-survival, and value-capture lenses.

For all non-U.S.-dollar monetary figures, MemoryCapex provides an approximate U.S.-dollar equivalent using contemporaneous exchange rates where relevant.

Sources

1. Investing.com, “AI’s memory stack: What investors need to know,” Aug. 29, 2026. Source

2. Investing.com, “Bernstein lifts price targets on memory stocks to reflect higher HBM prices,” June 22, 2026. Source

3. Micron Investor Relations, fiscal Q3 2026 results, June 24, 2026. Source

4. SK hynix Newsroom, Q2 2026 business results, July 29, 2026. Source

5. SanDisk, fiscal Q4 2026 results, Aug. 5, 2026. Source

6. Yahoo Finance, memory stocks enter bear market, July 7, 2026. Source

7. Micron, HBM4 product page and high-volume-production release, Mar. 16, 2026. Source ; Source

8. Samsung Global Newsroom, HBM4E samples, May 29, 2026. Source

9. NVIDIA Technical Blog, “NVIDIA NVLink Fusion Brings NVHBM to Next-Generation AI Infrastructure,” Aug. 26, 2026. Source

10. Samsung Global Newsroom, zHBM at FMS 2026, Aug. 5, 2026. Source

11. Qualcomm, Dragonfly / High Bandwidth Compute roadmap, June 23, 2026. Source

12. CXL Consortium, FMS 2026 AI-memory use cases. Source

13. SemiAnalysis, “CXL Is Dead In The AI Era,” Mar. 16, 2024. Source

14. SK hynix, “First HBF Standard Specifications with SanDisk,” Aug. 4, 2026. Source

15. Investing.com / MarketScreener, Micron analyst consensus, accessed Sept. 1, 2026. Source

16. Investing.com, SK hynix and Samsung analyst consensus, accessed Sept. 1, 2026. Source ; Source

17. Investing.com, SanDisk analyst consensus, accessed Sept. 1, 2026. Source

18. Investing.com / S&P Global-derived consensus, Kioxia, accessed Sept. 1, 2026. Source

19. IMF, Representative Exchange Rates, Sept. 1, 2026. Source

20. Investing.com, Bernstein raises SanDisk target to $3,000, June 29, 2026. Source

21. Tom’s Hardware, d-Matrix Raptor Hot Chips 2026 coverage, Aug. 26, 2026. Source

22. EDaily / Korea Times, Samsung zHBM at SEMICON Taiwan, Sept. 1, 2026. Source

23. Yonhap, Samsung U.S. executive on zHBM timeline, Aug. 28, 2026. Source

24. Intel Optane business update. Source

25. NVIDIA Technical Blog, BlueField-4-powered CMX, Mar. 16, 2026. Source

26. Kwon et al., “Efficient Memory Management for Large Language Model Serving with PagedAttention,” arXiv, 2023. Source

27. NVIDIA Technical Blog, “Optimizing Inference for Long Context and Large Batch Sizes with NVFP4 KV Cache,” Dec. 8, 2025. Source

28. Mistral AI, “Mistral 7B,” 2023. Source

29. Reuters / SK hynix, Indiana next-generation HBM production base, Aug. 27–28, 2026. Source

FX reference used in this article

For non-U.S.-dollar monetary figures, U.S.-dollar equivalents are based on the September 1, 2026 IMF representative exchange rates: KRW 1,373.6 per USD 1 and JPY 159.77 per USD 1. Approximate USD equivalents are rounded for readability.[19]

#HBM #HBM4 #DRAM #NAND #AIMemory #MemoryCycle #MemoryHierarchy #AIInfrastructure #CXL #NVHBM #zHBM #HBF

Position & Independence Disclosure: the author, and/or any investment vehicle or entity he owns, manages, or is affiliated with, personal or corporate, in any jurisdiction, past, present, or future, should be assumed to hold long /short positions in most or all companies and securities discussed on this site. Memory Capex does not provide investment, financial, legal, or tax advice, and does not solicit, accept, or manage outside capital. Full disclosure.