NVIDIA stopped buying HBM for capacity. The same method sizes the high-performance durable storage tier underneath it. 

Ken Claffey, VDURA. Response piece to SemiAnalysis, “Long Live the Short King: Why 4-hi HBM Wins” (Sept 13, 2026). Last month SemiAnalysis published “Long Live the Short King: Why 4-hi HBM Wins”. It is about memory stack heights on next-generation accelerators. It is also, though it never says so, the best argument I have read this year for how to size and architect a high-performance durable storage tier. 

Their case: an HBM4/4e cube delivers the same bandwidth whether it is built four dies high or twelve, because bandwidth is set by the 2,048-I/O interface, not by the dies behind it. Stack height buys capacity and nothing else. So how much capacity do you actually want, at what price, on the most expensive memory technology ever manufactured? 

Much less than you think, and the reason is worth stating precisely, because it is easy to overstate. Divide stack bandwidth by tokens per second per user and you get the gigabytes you can read inside a single token step: roughly 33 GB at 100 tokens per second. Capacity above that line is not unreachable. It holds KV cache, it serves concurrency, it does real work. What it cannot do is turn into more served tokens once bandwidth is the binding constraint. And bandwidth binds first. In their modelling of a 2.8-trillion-parameter model on an NVL576-class system, at a 105 tokens-per-second interactivity target, 12-hi delivers no throughput advantage over 8-hi at all. With the latency target relaxed entirely, 8-hi buys 8% over 4-hi and 12-hi buys 10%, for TCO premiums of 12.1% and 26.3%. 

The memory is there. The bandwidth to exploit it is not. That is stranded capacity, and it is an economic death rather than a physical one: a 26% premium for an attribute that returns 10% at best, and nothing at the interactivity you are actually selling. 

NVIDIA appears to have made the call already, and not marginally. Rubin Ultra is reported at 192 GB of HBM per GPU against a plan of record of a terabyte, less than a fifth of what was intended, and down from 288 GB on the generation it replaces. Per compute die it falls from 256 GB to 96 GB. Read that plainly. HBM became expensive and scarce enough that the largest buyer of it on earth stopped treating it as a place to put data. It now buys enough to saturate the bandwidth it needs, sizes what comes with it against what the model requires resident, and lets the rest fall down the hierarchy. 

Two things have changed around that arithmetic, and they point the same way. 

Storage has stopped being overhead and started being a product. SemiAnalysis, reviewing the GPU cloud market, named performance storage as one of the two services most providers do not yet offer and an obvious way for them to expand. A line item you resell is a line item whose cost you own. 

And the money behind these builds costs more than it did. This infrastructure is financed against contracted revenue, so when financing gets more expensive, every unit of capex that does not convert into sellable capability comes off the margin. 

Which puts a specific question in front of anyone running a GPU cloud: where do you take capex out without taking capability out? Accelerators are scarce and priced accordingly. Power is the binding constraint on the building itself. DRAM has gone the way of flash. Almost every line on the quote is defended by something you actually need. 

The storage tier is the exception, not because storage matters less, but because the waste in it is not capability. It is capability you have already paid for and are not receiving. That is the rarest thing on an AI build: a line you can cut without losing anything. 

Economics drives architecture 

It is worth being clear about why NVIDIA changed, because it was not a technical discovery. Nothing about HBM physics moved. The price moved, and the availability moved, and the architecture followed. 

That is the normal direction of travel, and it cuts both ways. Much of the storage architecture being sold today was designed between 2020 and 2024, when NAND fabs were overbuilt, prices were falling, and memory makers were posting losses to hold share. In that world an all-flash design was not extravagant. It was the rational answer: media was cheap and getting cheaper, so capacity on the fast tier cost almost nothing and bought real simplicity in exchange. 

Those were genuine conditions, and that was a sound response to them. Then the market turned, and it has not turned back. The architectures have not moved; they are still being sold, with the same assumption welded into them, into a market where the assumption no longer holds. 

Which leaves the question this article is about. The economics of HBM changed, and the architecture changed with them. The economics of everything below HBM have changed too: DRAM, NAND, and the systems built out of both. So what should that do to the tiers underneath? 

We build those tiers, and we have reached the same conclusion about flash that NVIDIA reached about HBM. The rest of this is the arithmetic. 

The same principle governs flash 

Bandwidth is a property of the device, set by whichever limit binds first, the interface, the controller, or the media itself. Capacity is a property of what sits behind it. They scale independently, and the premium tier in any hierarchy is premium for its bandwidth, not its capacity. 

An HBM cube gets 2,048 I/Os whether it is 4-hi or 12-hi. A PCIe Gen5 x4 NVMe drive gets its roughly 14 GB/s whether it is 7.68 TB or 61.44 TB. In both cases the bandwidth-to-capacity ratio collapses as capacity grows, and every terabyte past the point where bandwidth binds is bought at premium-media prices for value a cheaper medium could have supplied. 

HBM figures below are SemiAnalysis’s; the SSD figures are from Kioxia’s published Data Center SSD datasheet (CD9P series, PCIe Gen5, BiCS FLASH TLC); the HDD figure is a typical nearline specification. 

	Sequential read	Bandwidth per TB
HBM4e, 4-hi (128 GB)	—	~26,000 GB/s
HBM4e, 12-hi (384 GB)	—	~8,700 GB/s
Kioxia CD9P, 7.68 TB	14.8 GB/s	~1.93 GB/s
Kioxia CD9P, 61.44 TB	13.5 GB/s	~0.22 GB/s
Nearline HDD, 30 TB	~0.28 GB/s	~0.009 GB/s

Note what the flash rows do, and note which metric they are measured on. None of the things that bound a drive’s bandwidth (interface, controller, media) scale with capacity. Capacity scales on its own. So the two numbers a buyer cares about move in opposite directions. 

The cleanest demonstration is inside a single family, where nothing changes but the capacity. Kioxia’s CD9P line, one datasheet, one NAND type: 

Kioxia CD9P	Sequential read	Sequential write
7.68 TB	14,800 MB/s	7,000 MB/s
15.36 TB	14,800 MB/s	7,000 MB/s
30.72 TB	13,500 MB/s	7,000 MB/s
61.44 TB	13,500 MB/s	7,000 MB/s

Eight times the capacity. Identical write bandwidth, and nine percent less read. The drive does not get faster as it gets bigger, because none of what limits it got bigger; only the NAND behind it did. 

Now put a price on each metric. Dollars per terabyte is the number capacity-led designs optimize, and the 61 TB drive wins it comfortably. Dollars per gigabyte-per-second is the number you are buying on when you size storage to feed GPUs, and on writes the 61 TB drive is eight times worse: eight times the cost, for exactly the same 7,000 MB/s. 

That is the actual choice in front of anyone filling a drive slot, and it is not a close one if bandwidth is what you came for. The metric that looks best on the quote is the one you are not buying on. 

Which raises the obvious question. If dollars per terabyte is the metric you are optimizing, what are you buying flash for at all? That is the argument in one line. Flash earns its premium by delivering bandwidth; buy it for capacity and you have hired the expensive medium to do the cheap medium’s job. 

NVIDIA looked at a 3× trade on HBM and declined it. The capacity-led flash designs the industry has been selling ask you to accept more than twice that. 

The conclusion carries across. The marginal terabyte inside a fast device is worth exactly what you can move through it. Past the point where bandwidth binds, it moves nothing, and its value collapses to plain dollars-per-terabyte, at which point you should have bought the cheapest dollars-per-terabyte medium available and put the difference where it earns a return. 

What the difference is worth 

That last phrase, the cheapest dollars-per-terabyte medium available, is doing a lot of work, so here is the number behind it. 

We publish a quarterly Flash Volatility Index tracking enterprise SSD pricing against nearline disk, built from public market data on a uniform commodity-pricing basis. It is named for volatility rather than inflation, and the shape is why: 

	30 TB TLC enterprise SSD	Multiple vs nearline disk
Q2 2025	$3,062	6.2×
Q1 2026	$17,500	23.2×
Q4 2026	$22,600	14.3×

Two things in that table matter, and they are different things. 

The multiple reset. It went from 6.2× to 23.2× in three quarters, as flash outran disk while multi-year hyperscaler agreements locked up manufacturing capacity and AI demand met a supply base that had stopped expanding. It has not come back. Three quarters later it is still above 14×, triple where it sat before 2025. 

And the absolute price kept climbing. A 30 TB enterprise SSD has gone from $3,062 to roughly $22,600, with the steepest part of that in the past year. The multiple coming down from its peak while SSD prices kept rising means nearline disk has roughly tripled as well; nobody got a discount this year. But the gap that opened in 2025 has not closed, and planning on it closing is a bet on a supply picture nobody is currently forecasting. 

The historical flash-to-disk premium sat nearer 4–5×, low enough that putting a few extra petabytes on the wrong medium was an accounting footnote. At roughly 14×, sustained for a year, it is the largest single decision on the quote, and it recurs annually rather than once. 

Data reduction does not rescue the comparison: it applies to both media, so the ratio between them holds; it raises capacity per device without raising bandwidth, which makes the stranding worse rather than better; and AI data (weights, checkpoints, tokenized corpora) reduces poorly to begin with. 

Worth saying where we stand, since it is our index. VDURA sells all-flash systems and mixed-media systems, and the method in this article is how we size both. The index exists because our own sizing depends on knowing this number accurately, not because we need it to land anywhere in particular; at 4× the same method put far more capacity on flash, and if it compresses again it will do so once more. A vendor who sells only one medium needs this number to come out one way. We do not. 

The stranding you can no longer afford 

SemiAnalysis named one place money gets stranded. There is a second, one tier down, and it shows up in the storage line of a cluster build. 

Stranded HBM capacity is capacity you paid for and cannot turn into tokens. Stranded SSD performance is the mirror image: speed you paid for and cannot fully use. 

An NVMe drive is a performance medium: speed is what the premium buys. Put one behind a stack that delivers a fraction of its rated throughput and the capacity arrives in full while most of the premium does not. Past the saturation point each additional drive adds marginal throughput and a great deal of capacity, the same diminishing return that made 12-hi a bad trade, one tier down. You paid for a performance device and are running it as a capacity device. 

The two are not the same kind of failure. The HBM case is economics: the memory works perfectly and the last increment does not justify its cost. The SSD case is engineering: the speed is in the device, fully paid for, and the architecture cannot get at all of it. 

What has changed is whether you can live with it. When a 30 TB enterprise SSD cost $3,062, stranded flash was noise: a line nobody audited, because getting it wrong changed nothing material. At $22,600 it is not noise. The storage tier is a visible fraction of what a cluster costs to build, and waste inside it has stopped being denominated in dollars. It is denominated in accelerators. 

Price is only one of three things moving. HBM per GPU is going down. And KV cache that used to live and die inside a serving node is now reused across steps and sessions, so it has to be reachable by more than the node that made it. It does not necessarily have to be durable; it is derived, and recomputable at a price. It does have to be somewhere large and fast. That is a storage problem whether or not anyone chooses to call it one. Each pushes the same way: the flash tier is becoming more expensive and more load-bearing at once. 

Workload pressure only moves one way 

The modelling was not the end of it. SemiAnalysis constrained HBM on a serving deployment and watched what broke. Throughput held across almost the entire operating curve, degrading only above roughly 70 concurrent users. They report this as vindication for shorter stacks, and it is. 

But look at the other number. Cutting per-GPU KV-cache headroom by 36–44% caused DRAM offload to activate ten times more often. 

Nothing disappeared. The pressure moved down a tier, and the tier below absorbed ten times the traffic. The experiment did not show that capacity stopped mattering; it showed that the tier underneath was fast enough to hide the change. 

NVIDIA has names for the tiers that pressure moves through. In the hierarchy it published alongside its inference context memory work, G1 is GPU HBM, G2 is system DRAM, G3 is local SSD, and G4 is shared file and object storage. Its own account of the mechanism is that as context grows beyond HBM the KV cache spills into G2 and G3, and that it can exhaust G1 through G3 and be pushed down to G4. That is the same arrow SemiAnalysis measured, drawn by the company doing the cutting. 

What matters is the state of each tier it points at. G2 is DRAM, up 205% over the same window as the flash figures above, which is why the paper notes that freeing HBM wafer capacity would help conventional DRAM supply, a polite way of saying DRAM is constrained too. G3 is the flash inside the GPU node: roughly a drive per GPU, a bill that grows with the accelerator count, and NVIDIA’s own description is that it “grows with the number of GPUs but remains too small to cover all KV needs” and, being tied to a single node, “doesn’t scale easily.” It is scratch. It serves the node it sits in, and nothing else in the cluster can reach it. Which leaves the tier this article is about: the shared, durable, high-performance one that everything else falls onto. 

NVIDIA’s own response is the clearest acknowledgement that this is real and that it lands lower. It has announced a further tier between G3 and G4 (G3.5, Ethernet-attached flash shared across a pod, availability still to be confirmed) on the stated grounds that growing context exhausts the tiers above it, and that pushing what spills onto general-purpose storage “drives up power consumption, and directly limits cost-efficient AI scaling.” What decides the bill is how much bandwidth your storage delivers per device, and how well it holds that under load, across the whole AI pipeline, not just the KV cache. 

FIGURE 1: Cutting G1 does not reduce the working set; it relocates it. Solid bar: the displacement SemiAnalysis measured. Dashed: where it continues as context lengths and session counts grow. Tier definitions are NVIDIA’s. 

Reads are easy. Writes set the limit. 

Checkpointing is a write. KV cache eviction is a write. Ingest is a write. And in training it is the checkpoint that defines the bandwidth requirement in the first place: synchronous, bursty, with several thousand GPUs idle until it lands. A system sized on read numbers has been sized on the direction that was never holding anyone up. 

The HBM analogy does not prepare you for this one either. HBM reads and writes across the same bus at broadly the same rate; storage does not. Even at the device level, before a filesystem or a protection scheme or a network has touched it, the CD9P writes at roughly half its read rate. 

Then the architecture takes its own cut, and quoted figures tend to be measured in the most flattering place available. A write has to compute its protection, land durably, and keep going after the absorbing buffer in front of the media is full. Nearly every architecture has such a buffer: battery- or supercapacitor-backed DRAM, storage-class memory staged ahead of QLC, an SLC region on the drive itself, a separate flash staging tier. They differ in mechanism and they all have a size. But a checkpoint from several thousand GPUs is not a burst in the sense the buffer was designed to absorb. It is a sustained drain, and the number that matters is the rate the system holds once the buffer is saturated and it is destaging to durable media as fast as data arrives. 

The number of drives you buy is set by whichever direction your architecture is worst at. If your system writes at a quarter of what it reads, you do not size to the read number; you buy drives until the write number is met. That can be four times as many drives, carrying four times the capacity, none of which anyone asked for. Sizing on read and discovering the write gap in production is how a flash tier quietly doubles. 

So ask for both, measured at steady state, sustained long enough to outlast whatever buffer sits in front of the media, and at the occupancy you will actually run at. Not on a system fresh out of the box. 

How you size a flash tier 

SemiAnalysis sized HBM in two passes. Bandwidth first: how many stacks to hit the throughput and interactivity target? Then capacity: does what those stacks hold cover the model? If it does, height beyond that is money. 

A high-performance durable storage tier sizes the same way. The whole of it reduces to one line: 

N = R ÷ (d × E) 

N is the number of drives you buy. R is the bandwidth you require. d is what one SSD delivers. E is the fraction of d your architecture actually gets through to a client. Everything else follows from N: the capacity you end up owning is N times the drive capacity, the flash spend is N times the drive price, and the cost of a delivered gigabyte per second is the drive price divided by d × E. 

Worth sitting with that line, because of where it says the leverage is. R is set by your accelerators. d is set by whoever made the drive, and everyone is buying from the same short list of suppliers. E is set by the architecture, and it multiplies all of the rest. Halve it and you buy twice the drives, carrying twice the capacity nobody asked for, drawing twice the power. Buy a larger drive instead and you raise the capacity term without raising d, which improves exactly one number on the quote and worsens every other one. 

In practice that is five steps, and almost never the order a quote arrives in. 

Start with bandwidth, both directions. There is a published per-GPU requirement for read and for write, but it is a range rather than a number, and it moves by more than an order of magnitude depending on the workload and on the class of deployment you are building for. Training on tokenized text, multimodal pipelines and physical AI are not the same requirement, and a reference design sized to the bottom of that range has no headroom in it by construction. Establish which figure applies to you, multiply by the fleet, add the checkpoint aggregate, and bound the result by what the fabric between storage and GPUs can carry. Those numbers have nothing to do with how much data you own. 

Then count devices, not terabytes. Required bandwidth divided by what your architecture delivers per SSD gives you N. Not the capacity, the drives. 

Then take the smallest drive that gets you to N. This is the step that mirrors 4-hi and the one the market has been taught backwards. Bandwidth is a per-device property. A 61 TB QLC drive does not give you four times the bandwidth of a 15 TB TLC drive; it gives you less, with four times the capacity attached. If you need N devices, the cheapest way to own N devices is N small ones. Buying high-capacity SSDs to reach a bandwidth number is the 12-hi mistake, committed in flash. 

Then check what capacity came along. N drives gives you some number of petabytes whether you asked or not. Does it cover the working set (active dataset, the checkpoints you need to restore from quickly, the context that has to stay warm) plus enough headroom that a burst can land while the drain to disk is still catching up? Usually it does, comfortably, because the bandwidth requirement is demanding and the working set is not large. If it does not, size up deliberately, knowing what you are paying for. 

Then put the rest downstairs. Every petabyte beyond the working set belongs in the capacity tier, still inside the namespace, still able to answer a read, just not on performance media. 

They sized HBM capacity to the model. You size flash capacity to the working set. Same method, one tier down. 

The new economics set two hard problems 

There is a fair objection to all of this, and it is worth taking seriously. People went all-flash when flash cost four or five times what disk cost. They were not being careless. They were buying their way out of tiering. 

And they were right to. Tiering, as the industry had built it, was designed for a different job: enterprise lifecycle management, where data cooled on a predictable schedule, migration ran overnight, and a recall that took minutes was acceptable because a person was waiting on it rather than four thousand GPUs. Pointed at an AI workload, that machinery failed in exactly the ways you would expect. It moved the wrong data. It moved it at the wrong time. And it took front-end bandwidth to do so. All-flash was never really a media preference. It was the price of not having to tier at all. 

So the new economics do not simply make the old answer correct again. They raise the bar. An architecture that wants to take capacity off flash now has to solve two hard engineering problems that legacy tiering never solved, and they are not solved by the same work. 

The first is minimizing N. Every additional GB/s you extract per SSD is a drive you do not buy, with its capacity, slot, power, license and refresh attached. 

This is not a tuning exercise. Getting most of a device’s rated throughput all the way out to a client is a whole-stack problem (the node, the network, the data-protection scheme, the metadata path, the filesystem and the client) and it is won by not blocking anywhere. Every gateway, every controller pair, every point where traffic funnels through a head node or serializes on a lock is bandwidth the drives could have delivered and did not. This is the problem a true parallel filesystem exists to solve: clients reading and writing across many devices at once, directly, with nothing in the middle deciding whose turn it is. 

And it has to hold as the system grows. Plenty of architectures look efficient at a rack and lose linearity well before tens of petabytes, because something in the path (metadata, a protection domain, an aggregation tier) stops scaling with the device count. Per-SSD throughput at one petabyte tells you very little about per-SSD throughput at thirty. The spread between architectures here is not ten or twenty percent. It is multiples, and it widens with scale. 

So the drive count is set by the vendor’s engineering, not by what the drive can do. That is E, and the difference between d and d × E is pure overhead: the inefficiency gap between buying flash for the bandwidth it delivers and buying flash to compensate for a stack that cannot reach it. 

The second is a different problem entirely. NVIDIA can strip HBM capacity because the tier beneath is host DRAM, a small multiple away in latency, which is exactly why the serving deployment SemiAnalysis tested survived a tenfold increase in offload. The fall from flash to disk is not a small multiple. It is orders of magnitude. 

So the whole strategy stands or falls on the tiering, and not on whether you have some, but on whether it is fit for this workload. Fast enough to keep up with what a cluster actually produces, and smart enough to be moving the right things while it does. That is three requirements, not one. 

The capacity tier has to be built for throughput, not for dollars per terabyte alone. A disk tier specced like an archive (too few devices, each one as large as possible) cannot absorb what a flash tier hands it, and that is a purchase decision you do not get to revisit later. 

The mover has to be fast, and it has to be right about what moves, when it moves, and how. Bandwidth alone is not enough; promote or demote the wrong data and you pay for the transfer twice when it comes straight back. 

And neither of them can be taking front-end bandwidth away from the GPUs while they work. 

Which is why the common shape struggles here: a fast parallel filesystem, a separate object store behind it, and a fleet of external data movers bridging the two across a network. Three products, two namespaces, and a drain rate that nobody owns. The flash tier is a buffer, and a buffer only works if it empties as fast as it fills. Get any link in that chain wrong and the flash fills anyway, and then your sustained write rate is whatever the drain can hold, in the middle of a checkpoint, with the cluster waiting on it. 

And that assumes the chain is healthy. Storage nodes are servers, and feeding a few thousand GPUs takes a hundred or more of them. At ordinary server failure rates that is several node failures a year, and concurrent ones are an operating reality rather than an edge case. A rebuild competes with the drain for the same devices and the same network, so a drain rate quoted on a healthy system tells you very little about the window when you most need it to hold. In a chain of three products, each has its own failure domain, its own degraded mode and its own rebuild behavior, and the interaction between them is in nobody’s datasheet. It shows up the first time it happens to you. 

“Put a capacity tier behind it” is not a design. The media choice is the easy half. 

Neither problem holds still 

And neither of them is solved once. The fleet grows, models change shape, context lengths climb, interactivity targets move, and the working set grows alongside all of it. N is not a number you calculate at purchase; it is one you will recalculate several times over the life of a cluster, usually under time pressure, usually with the cluster already in production. 

So the architecture has to expand online, at a granularity that matches the way budget actually arrives rather than in forklift increments. And the two problems have to stay in step with each other. Adding flash without adding the capacity tier and the movers to match does not buy headroom; it builds the filling problem faster. A bigger buffer in front of the same drain fills later, not never. 

The ratio that has to hold is bandwidth per petabyte, at every size the system will ever be, not the one size on the quote. Ask a vendor what happens to it at three times the capacity, and whether getting there is an expansion or a migration. 

Where that money should have gone 

There is a version of this where you do everything right at the top of the stack and hand it back one tier down. 

The saving from 4-hi is real. Spend it on flash you did not need, bought to compensate for an architecture that could not deliver bandwidth from fewer devices, and you have saved nothing. You moved the waste down a tier and paid freight on it. 

The money is not neutral where it lands. Every dollar in stranded flash performance is a dollar that could have gone further up the stack, where it compounds: more accelerators, which is more compute and more HBM together in the ratio the hardware was designed around, and more network between them. Those purchases move dollars per token. A petabyte of flash nobody reads moves nothing. 

Power is the harder version, and the one that does not forgive you later. AI data centers run out of power before they run out of floor space or budget: the binding constraint is the substation, not the purchase order. Every kilowatt drawn by storage you did not need is a kilowatt not turning into tokens, and that allocation was fixed when the building was specified. SemiAnalysis treats tokens per watt as a first-class metric, and NVIDIA puts it more bluntly still: “every megawatt of power is ultimately judged by how many useful tokens it can deliver.” That does not stop being true when the watts are drawn by a storage rack. 

Which makes this an AI factory question rather than a storage one. Storage and accelerators draw on the same budget and the same power envelope, so the question is not how much your storage costs. It is what fraction of both you are willing to leave sitting in devices whose performance you cannot reach. 

We built for this 

Our mission is to build the most efficient high-performance storage infrastructure for AI. Efficient in the sense this article has been using the word: the gap between what the hardware is capable of and what the system actually delivers, closed as far as engineering allows, kept closed as the system grows, and still closed under real operating conditions, with a node down and a rebuild running. Efficiency that only holds on a healthy system at demo scale is specmanship, and it does not survive contact with a production cluster. 

Those two problems are where that gets decided, and they are the reason VDURA looks the way it does. Both were bets taken early, because neither can be retrofitted. 

A true parallel filesystem, with nothing in the middle. Clients read and write across every device directly: no gateway tier, no controller pair, no head node deciding whose turn it is. Protection is computed on the client, so a write does not funnel through a storage head to have its parity calculated on the way past; that is where most architectures give up half their write bandwidth. Metadata scales with the system rather than serializing on a single service. Together that is what holds delivered throughput per device roughly linear from one rack out to tens of petabytes. Which is why we treat delivered GB/s per SSD as the number that matters most. It decides how much flash anyone has to buy, and it is almost never on a quote, mostly because it is almost never asked for. 

One filesystem across both media. One namespace, one control plane, one data plane, not a filesystem with an object store bolted on behind it. Placement and movement are automatic by default, with policy controls for the teams that want to assert them: pin a dataset, keep a project resident, set the rules for a tenant. Nothing depends on an administrator maintaining placement by hand for the system to hold its performance. 

High-performance S3 in the same namespace. The other reason to put an object store behind the filesystem is that you need S3 and the filesystem does not speak it. Ours does: the same data, the same namespace, no second copy and no second system to hold in step. 

Movement that keeps up. The drain is where this strategy lives or dies, so it gets all three of the things it needs rather than one. Data movers run in parallel inside every storage node, not as a separate fleet bridging two systems, so moving capacity scales with the cluster instead of falling behind it, and adding a node adds its share of drain with it. The capacity tier behind them is built for throughput rather than for dollars per terabyte alone: enough devices, streaming, sized to absorb what the flash hands over. And placement is context-aware: the system decides what moves, when it moves and in which direction from what the workload is actually doing, because moving the wrong data quickly is still the wrong answer. 

Expansion online at node granularity, rebuilds spread many-to-many. Rebalancing is automatic, and repair work is distributed across the cluster rather than concentrated on a partner node, which is what keeps the drain running while the system is repairing itself. 

We did not arrive at mixed media recently, or reluctantly. We built on the view that the 2020–24 flash prices were an anomaly rather than a trend, and that any architecture assuming they would hold was going to be stranded when they did not. That view is now simply the market. 

Six questions 

The question to put to a storage vendor is not what the drives can do; everyone is buying from the same short list of suppliers. It is what the architecture around them does. And the first one eliminates most of the field. 

• Can your system use both media natively, under one namespace, with the data path and the movers built in, not a separate object store behind it, not a third-party capacity tier, not a fleet of external movers bridging the two across a network? 

• How many GB/s does your system deliver per SSD, read and write, measured through the filesystem to clients at steady state, sustained past the point any write buffer is saturated, and at realistic occupancy, not quoted from a drive datasheet? 

• How many drives does that make me buy to meet my bandwidth requirement? 

• How much capacity arrives with them, and how much of it did I ask for? 

• What happens to my front-end bandwidth while the tiering is running? 

• What happens to all of the above with a device down, and with a storage node down? How much of the quoted bandwidth remains, how long until it is back to full, and is the recovery automatic or does it need an engineer at a console, and what does a second failure during that window do? You are signing availability commitments out of these numbers, and a figure measured on a healthy system does not tell you whether you can meet them. 

Answer those six and you have the real price of a bandwidth target. Most vendors stop at the first. Of those that do not, most answer the second with a number copied off a drive datasheet and the rest not at all. 

NVIDIA asked the equivalent question about HBM and answered it by buying less of the expensive thing. Anyone buying storage can ask it too, but the arithmetic only comes out in your favor if the architecture underneath is efficient enough to make it work, and the tiering beneath it good enough to survive the drop. 

The short king thesis is right. It is just not finished. It stops at the edge of the accelerator package, and the logic does not.