Running Slurm Where Compute Is Abundant

Abundance and price are the same fact: the market deep enough to always have capacity is the market that sets the floor. The identical machine trades 82% apart between regions; the cheapest qualifying spot vCPU sits at roughly a sixteenth of typical on-demand. Multicloud chooses the machines — cloud, region, and instance type — from a live catalog across AWS, Google Cloud, and Azure, so your queue is sized by that market instead of by one reservation, and priced at its floor. Two workloads carry the whole argument, from here to the last page: the simulation — a molecular-dynamics run that must hold 2,000 cores together in one region for weeks — and the sweep — a nightly Monte Carlo risk run, 500,000 vCPU-hours a month, that can be anywhere at all. The simulation gets 57% off and its calendar cut from 28 days to six; the sweep runs at ~90% below on-demand. Two doorways — spill-over capacity behind the Slurm cluster you already run, or a complete Slurm environment behind one URL and your SSO.

---

1. The problem: HPC is a queue problem first, and a bill problem second

The capacity you need exists. Your queue just can't reach it — and reach is also price, because the pool deep enough to absorb your peak is the same pool that holds the floor.

Every Slurm site knows both halves. The queue half: a deadline meets a fixed-size cluster, the queue backs up, and researchers wait days for cores that are sitting idle somewhere they can't reach. The bill half: in scientific simulation, genomics, rendering, and backtesting, compute isn't overhead — it is the cost of the product. And it is bought from a market whose price spreads a single-cloud setup never sees. Three facts about that market:

• The same machine costs wildly different amounts in different places. The identical AWS c6g.8xlarge (32 cores, 64 GB) listed at $0.68/hr in Mumbai and $1.24/hr in Frankfurt on the day these numbers were pulled (Multicloud live catalog, 2026-07-22). Same silicon, same SKU, same everything — except the postcode. That is 82% riding on a choice most teams make once, in a config file, and never revisit.
• Spot is a whole discount class, not a coupon. Spot instances are a cloud's unsold inventory: identical hardware, sold cheap, reclaimable at short notice. The cheapest one meeting real constraints that day cost $0.0021 per core-hour — about a sixteenth of comparable on-demand. And it behaves like a market, because it is one: the floor moves, continuously.
• The floor is never reliably in one cloud. In the same pull, the fifteen cheapest spot cores anywhere were spread across AWS, Google Cloud, and Azure — ten regions on four continents. That day the floor sat in Azure; a team locked to either other cloud paid 61–79% more, however shrewdly they shopped inside their own. Tomorrow it sits somewhere else. Picking one cloud is a bet that goes wrong on a schedule you don't control.

Meanwhile the tools an HPC team reaches for all stop at one cloud. Every vendor ships a respectable way to run Slurm on that vendor's cloud — and each one quietly manages you deeper into it. Slurm's own scaling hooks have the same horizon: the script that creates nodes calls a single cloud's API, because someone at your site wrote it against a single cloud's API. Nothing in the standard toolbox answers the question that actually moves the bill: which cloud and region should this batch land on today?

This paper describes an architecture that fixes the reach, not the scheduler: Slurm — unmodified, including SchedMD's own Kubernetes operator — running on a capacity layer that continuously buys the cheapest qualifying nodes across every region of AWS, Google Cloud, and Azure (Multicloud), with Cloudflare R2 as a cloud-neutral data plane. It enters your world through either of two doorways: spill-over capacity behind the Slurm cluster you already run, or a complete Slurm environment served behind one URL and your SSO. Nothing migrates either way.

The two workloads this paper follows

Architecture arguments are cheap in the abstract, so this paper picks two workloads now and refuses to let go of them. They sit at opposite ends of everything that matters — how much the pieces talk to each other, whether a lost machine hurts, how much data moves, what the deadline looks like — so between them they bracket most of what a Slurm site actually runs. Every section from here answers the same two questions: what does this mean for the simulation, and what does it mean for the sweep?

	The simulation	The sweep
Who	A structural-biology lab	A quantitative-finance risk desk
The work	GROMACS molecular dynamics — one long simulation, run for weeks	Monte Carlo scenario evaluation, run nightly
Shape	Tightly coupled: every process talks to its neighbours, constantly	Embarrassingly parallel: hundreds of thousands of tasks, none aware of the others
Appetite	2,000 cores held together for weeks (~1.34M core-hours)	500,000 core-hours a month, in six-hour overnight windows
Interruption	Lose one machine, lose the job — checkpoint, or pay for certainty	Lose one machine, lose one task; Slurm re-runs it
Data	Tens of TB of trajectories out	Kilobytes in, kilobytes out, per task
The deadline	A submission date months away that everything is measured against	Tomorrow morning, before the market opens
Where it must run	One region — latency gates the science	Anywhere — the market decides, continuously

They want opposite things, and that is the point. The simulation can't be scattered across clouds, so it wins by landing in the right place on the right silicon — bought once, held for the run. Section 8 takes it from $45.7k to $19.6k, and from 28 days to six. The sweep is scattered by nature, so it wins by contesting every market, every night. Section 9 takes it from $17k a month to $1,649. One is an argument about buying well; the other is an argument about reach. The same layer underneath serves both, because both are the same question at different scales — which machines should exist right now, and where?

Neither is a customer anecdote. Each is a documented workload shape priced against a dated catalog pull, so every figure here can be checked, re-run, and argued with.

A note on units: this paper says "core" in prose where the price tables say "vCPU". They are the same thing — the unit the clouds actually sell and bill. Prices are quoted per vCPU-hour because that is how they are published; nothing is converted behind your back.

2. The scheduler layer is solved: Slurm — and it now runs on Kubernetes

You do not need to replace Slurm, and this paper will not ask you to. Slurm is the de-facto standard scheduler of world HPC — thirty years of queue policy your users and pipelines already speak fluently: who gets machines first, who has used more than their share, which jobs can slip into the gaps. Two upstream developments matter for what follows.

Slurm already knows how to burst — one cloud at a time. Slurm can define nodes that don't exist yet. When work needs them, it runs a script your site supplies to create the machines; when they sit idle, another script hands them back (SchedMD, power_save). This is the mechanism underneath every cloud vendor's Slurm tooling. The catch is in the words your site supplies: that script talks to one cloud's API, with one cloud's credentials, images, and networking. Bursting to a second cloud means writing, operating, and debugging a second script — and the choice between the two is a human reading price lists.

Slurm now runs natively on Kubernetes — per its own maintainers. SchedMD's Slinky project ships an official operator that runs Slurm's own daemons — the controller, the accounting database, the REST endpoint, the worker agents — as ordinary Kubernetes pods, with worker pools that scale, drain, and shrink to zero

(slurm.schedmd.com/slinky, SlinkyProject/slurm-operator). It reached v1.0 in late 2025 — and in December 2025 NVIDIA acquired SchedMD outright, pledging continued open-source, vendor-neutral development of Slurm and moving commercial support under NVIDIA Enterprise Support; NVIDIA reports running Slinky in production at the scale of eight-thousand-plus GPUs (NVIDIA). Nebius's open-source Soperator proves the same shape independently — and, usefully, keeps the operator layer a two-source market. The direction is not controversial anymore: Slurm's queue semantics on top, Kubernetes' machine lifecycle underneath.

That convergence is the hinge of this paper. Once a Slurm worker is just a pod, "where do Slurm's nodes come from?" becomes a Kubernetes question — and that is a question a cross-cloud capacity layer can answer.

One thing Slurm's own multi-cluster feature does not solve: federation lets sibling clusters inside one organization share a queue (SchedMD). It is not a price mechanism, and it never decides where capacity should exist. Federation answers "which of my clusters runs this job?" This paper answers "which market builds my cluster today?" They compose; they don't compete.

3. What Slurm assumes — and deliberately does not solve

Read Slurm's documentation closely and a boundary appears. Slurm decides which job runs when, on the machines it has been told about. Everything below that line is somebody else's problem — which today means yours:

Slurm optimizes	Slurm assumes someone else provides
Which job runs next — priority, fair-share, gap-filling	Which machines exist, and where
Queues, limits, accounting	Which cloud, region, and price each machine has
Job dependencies and arrays	The script that buys them — one per cloud, yours to write
Handing machines to jobs	Spot reclaims: noticing, draining, replacing
—	Cross-cloud networking, images, credentials, quotas

This is the seam, and it is where the money is. Slurm can be brilliant with the machines it has; if those machines were bought in the wrong region at the wrong price — or capped by one region's quota while a deadline burns — the queue is being optimized on top of a bill that was already lost. The regional spreads and the spot floor of section 1 live below the scheduler, in the layer that decides what to buy. Our two workloads hit this seam from opposite sides: the simulation needs one right region, chosen once and held for weeks; the sweep needs the right markets, chosen again every night. Slurm answers neither — on purpose.

That is not a flaw in Slurm. It is the layering that has let Slurm outlive every hardware generation it has ever scheduled: the scheduler works in seconds against the fleet it has, while buying machines works in minutes against a moving market. The rest of this paper is about running the first on the second.

4. The capacity layer: one decision over every region of every cloud

Multicloud operates the capacity market the scheduler assumes. The mechanism, briefly — the full story is the companion Architecture Whitepaper:

• A live catalog. Roughly 265,000 priced offers across ~3,900 machine types — every region and zone of AWS, Google Cloud, and Azure, spot and on-demand, each carrying how likely it is to be reclaimed and how fast its actual silicon measures. That last part matters more in HPC than anywhere: "32 cores" is not a unit of work, and the catalog knows which 32 cores do more arithmetic per dollar.
• One placement decision. The scheduler adds up what's waiting and buys the cheapest machines that qualify — from anywhere in that catalog. Qualify means whatever you declared: enough memory, fast enough silicon, a GPU class, allowed locations, a spend cap. Spot where your constraints tolerate interruption, on-demand where they don't. The simulation and the sweep differ only in what they declare — same decision, same catalog, opposite answers.
• Interruptions as weather. Every machine runs an agent listening for its cloud's reclaim warning. On notice: drain the work, buy the replacement from the catalog — often in another zone or region entirely — and have it serving in about a minute. Machines are replaced, never repaired; none of them holds anything you can't lose.
• One network, one cluster. Machines in different clouds join a single encrypted network under one standard Kubernetes control plane. A cluster spanning AWS, Google Cloud, and Azure is still one cluster — one config file, one scheduler's view, no federation project to run. Each customer gets their own: own control plane, own private networks in every cloud it touches, own certificate authority, nothing shared with anyone.
• Your accounts, your bill. Machines are provisioned into dedicated cloud accounts you own. Your negotiated discounts, committed-use plans, and credits still apply; Multicloud charges for the platform, never a markup on machines.

The operating model is deliberate: like a power plant behind a wall socket, the machinery is ours to run and what reaches you is standard. For a Slurm shop the socket has two sides — ordinary Kubernetes for your platform team, and ordinary Slurm for your users, who go on submitting jobs exactly as they did and never learn a single word from this section.

Because it is ordinary Kubernetes, Slurm isn't "integrated" here in any special sense. SchedMD's own operator just installs, from the same charts it uses anywhere else. Slurm's workers become pods; the pool's only job is to make sure the right machines, bought well, exist underneath them.

5. Doorway one: extend the cluster you already run (early access)

Nothing migrates. For a site already running Slurm, the entry point is Extend your cluster: your environment gains reach into the pool, and Slurm sees nothing more exotic than a new queue.

Never beg for quota or wait for capacity. Never overpay for it. And never migrate to get there.

The sweep goes through this doorway first, and that is the reason to start there: a nightly, re-runnable queue is the lowest-stakes thing a site owns. Point one queue at the pool, leave everything else exactly where it is, and the difference lands in your own accounting inside a week. The simulation can follow through the same door once the burst queue has proven itself, pinned to a single region (section 7).

The shape is a burst partition — a queue named burst. Your existing machines keep carrying the baseline exactly as configured today; the new queue maps to capacity the pool buys on demand and hands back when idle. Users opt in the way they already express everything else, by naming the queue when they submit. Your policy still governs: limits, priorities, fair-share, and accounting all apply to burst jobs like any others.

Two wirings, depending on where your control plane lives:

• Slurm on Kubernetes (Slinky or Soperator). Your cluster gains one virtual node backed by the pool — a node that isn't a machine, but a doorway to thousands. It is fenced off, so nothing lands on it by accident; point your burst workers at it and they run on the pool while still showing up in your dashboards and your queue listings exactly like local ones. The connection is outbound-only and encrypted. No foreign machine ever joins your cluster.
• Classic Slurm (bare metal or VMs). Your burst nodes are the same create-them-when-needed nodes Slurm has always supported — except the script asks the pool for capacity instead of calling one cloud's API. Same five-line contract, pointed at every cloud at once. The workers register with your controller over the encrypted network, and hand the machines back when the queue goes quiet.

# slurm.conf — the burst partition, classic wiring
NodeName=mc-burst-[001-256] State=CLOUD CPUs=32 RealMemory=63000 Feature=multicloud
PartitionName=burst Nodes=mc-burst-[001-256] State=UP MaxTime=24:00:00
ResumeProgram=/opt/multicloud/resume.sh     # asks the pool, not one cloud
SuspendProgram=/opt/multicloud/suspend.sh
SuspendTime=600

Only one thing changes, and it changes at exactly the moment your cluster fills up. Instead of the queue deepening behind fixed capacity — or behind one cloud's quota — the waiting work becomes a buy signal. The pool takes the cheapest qualifying machines on the whole board, joins them in about a minute each, and releases them when they go idle.

Spill-over is the doorway, not the ceiling. The same mechanism runs from "5% overflow during deadline week" to "the baseline is now the small half," one config edit at a time, with no migration event anywhere along the way.

6. Doorway two: Slurm as a front door — one URL, your SSO (design-partner program)

The second doorway is for teams with no cluster to extend — or no appetite to keep operating one. Multicloud stands up a complete Slurm environment on the pool and serves it to your users the way modern software is served: a public URL, TLS, and your identity provider. This is the doorway for the simulation's lab — a group whose scarcest resource is graduate-student time, not core-hours, and which should never have to staff a cluster administrator to run a simulation. It is a reference architecture assembled from standard parts, run for you as a design-partner engagement:

• The cluster. An isolated, standard Kubernetes cluster built from cheapest-qualifying capacity, with SchedMD's own operator running Slurm's controller, accounting database, REST endpoint, and login sessions. Sized by demand; costs nothing when nobody is using it.
• The door for humans: Open OnDemand. The standard open-source HPC web portal, from the Ohio Supercomputer Center — file management, job submission, interactive shells — with first-class single-sign-on support. Users go to hpc.yourdomain.com, log in with the company account they already have, and are submitting jobs in a browser. No VPN, no jump host, no SSH keys pasted into a wiki.
• The door for pipelines. Slurm's REST API behind the same login, for CI systems and scripted submission. The gateway checks the caller's identity and swaps it for the credential Slurm expects, which is the standard way to front it.
• The edge. DNS, certificates and their renewal, and the login proxy are the platform's own serving machinery — the same parts that front Multicloud's own surfaces, assembled for your tenancy as part of the engagement.

The pitch here is Simplicity at HPC scale: a research group gets what a national lab's user services team provides — portal, single sign-on, accounting, a queue with real capacity behind it — without hiring anyone to run the platform underneath. And because the substance is plain Slurm on plain Kubernetes in accounts you own, the door swings both ways. Ask for the keys and operate it yourself whenever you like. The Simplicity is optional, and it doesn't lock.

7. Two shapes of HPC work — and where each should land

Everything so far bought cheap machines. Where those machines should sit is a property of the work itself, and HPC has two classic shapes with opposite answers — the two the simulation and the sweep were picked to represent. The platform expresses both with the same two knobs: how tightly a job's machines must stay together, and which places are allowed at all.

Shape one: tightly coupled — the simulation. Molecular dynamics, fluid dynamics, weather: one model split across many machines that must stay in lockstep, exchanging results every step. The whole job runs at the speed of the slowest conversation, so the network between machines decides how far it scales. That ladder is steep. Specialist supercomputer fabrics answer in well under a microsecond, the clouds' best networking sits around 5–10 microseconds, and ordinary networking is slower again. An encrypted link between clouds, across the public internet, is a terrible place to run this, and this paper will not pretend otherwise. The honest answer is every machine in one region — often one building — talking over the cloud's own network.

The cross-cloud advantage doesn't disappear. It moves up a level: from the packets to the purchase. Tell the pool to keep the job together and it picks the region the way it picks everything else — the cheapest one, anywhere, that has the capacity and meets your constraints — then keeps every machine of that job inside it. Section 8 prices what that one choice is worth.

Shape two: embarrassingly parallel — the sweep. Monte Carlo runs, genomics pipelines, rendering frames, parameter studies: thousands of independent tasks that never talk to each other. (The name is a genuine term of art, and slightly unfair — this is the easy kind of parallel.) No grouping, no location rules. Every machine is bought on its own merits, wherever qualifying capacity is cheapest right now. Interruption is cheap by nature: lose a machine, lose one task's progress, and Slurm simply runs it again.

One cluster holds both shapes at once — a region-pinned queue for the simulation, a roaming scavenger queue for the sweep — because the rules attach to groups of machines, not to the cluster. Your users see two queue names. Underneath them sit two different answers to "where should machines exist?"

8. The simulation, priced: tightly coupled, pinned to one region

The first workload, in full. GROMACS is the workhorse of molecular dynamics, and it does scale — a 2025 peer-reviewed study drove it to 65,536 cores at better than 90% efficiency, though on a purpose-built supercomputer fabric (Kutzner et al., J. Comput. Chem.). The lab wants 2,000 cores held together for a multi-week run: 63 machines of 32 cores each, all in one region, talking over the cloud's own network.

So the placement question is a single line — which region? — and it is worth real money. Same catalog pull as section 1; cheapest qualifying 32-core/64-GB machine per sampled region, on-demand:

Cloud / region	Instance	$/vCPU-hr	2,000 vCPU × 1 hr
AWS ap-south-1 (Mumbai) — chosen	c6g.8xlarge	$0.0213	$42.60
Google Cloud us-central1	n4a-highcpu-32	$0.0325	$65.00
AWS us-east-1	c6g.8xlarge	$0.0340	$68.00
Azure eastus	Standard_D32ps_v6	$0.0351	$70.20
AWS eu-central-1 (Frankfurt)	c6g.8xlarge	$0.0388	$77.60
Azure westeurope	Standard_E32ps_v6	$0.0551	$110.20

Best to worst is a 61% spread for the same hour of the same-sized cluster. Even holding the machine type fixed — AWS c6g.8xlarge, identical hardware — Mumbai is 45% cheaper than Frankfurt. Across a four-week run, the chosen region bills about $28.6k and the worst row bills $74.1k. No grant proposal in history has a line item for that.

Three honesty notes, before anyone quotes the numbers:

• Spot is a dial here, not a default. A tightly coupled job dies with its slowest machine, so putting all 63 on reclaimable capacity multiplies the risk. The catalog scores that risk per zone, so the choice is explicit rather than hopeful: runs that can checkpoint ride low-risk spot and restart when unlucky; runs that can't run on-demand and take the regional spread — which is what this example's 45–61% already is, before any spot at all.
• Processor architecture is a constraint you declare. The cheapest rows here are ARM chips. GROMACS builds cleanly on ARM; code that doesn't is one constraint away from an Intel-and-AMD-only view of the catalog, at a somewhat higher floor.
• These are point-in-time list prices (live catalog, 2026-07-22). The existence of the spread is durable; its layout moves. Which is the argument for deciding by live query instead of by whichever region a spreadsheet picked two years ago.

Data doesn't complicate the choice, because the simulation's storage is cloud-neutral. Inputs stage in from Cloudflare R2 at no data cost into any region (section 10); results write out once and are readable everywhere afterwards, forever, for nothing. The region decision stays a pure compute-price decision, which is exactly what it should be.

Buying arithmetic, not cores — the third arbitrage axis

Everything above priced cores. But nobody wants cores; they want arithmetic, and the two are not the same market. A core is a unit of rental. Arithmetic is the unit of work. The catalog measures how fast each processor model actually is and prices every offer per unit of measured compute — and that ranking is not the price-per-core ranking. Same region, same day, same 32-core size (AWS ap-south-1, on-demand, 2026-07-22):

Instance	$/hr	Multi-core bench	$/vCPU-hr	$/1k-bench-pt-hr
c6g.8xlarge (Graviton2) — cheapest per vCPU	$0.682	11,616	$0.0213	$0.0587
c6a.8xlarge (AMD Milan)	$0.748	8,609	$0.0234	$0.0869
c7g.8xlarge (Graviton3)	$0.785	17,683	$0.0245	$0.0444
c8g.8xlarge (Graviton4) — cheapest per benchmark	$0.864	21,502	$0.0270	$0.0402
c6i.8xlarge (Intel Ice Lake)	$1.360	13,791	$0.0425	$0.0986

The winner of the section-8 table, c6g.8xlarge, is the genuine price-per-core champion — and a trap. Per unit of arithmetic delivered, c8g.8xlarge is 31.5% cheaper, despite costing 27% more per core-hour. Shopping by the obvious number overpays by 46%. This is not a one-region quirk: the same swap is worth 36.6% in us-east-1 and 36.9% in eu-central-1, where the obvious number overpays by 58%.

What it does to the simulation: re-ranking by arithmetic leaves the region order untouched — Mumbai still wins — but changes the machine inside every AWS region. Restated in the currency the lab actually cares about, the four-week run's $28.6k becomes about $19.6k — another 31.5% off a bill that was already the best region's. And the same work now arrives on ~34 machines instead of 63: fewer participants, less chatter between them, a straight win for a job whose speed is set by its slowest conversation. (If the job is rigidly sized at 2,000 cores, the saving shows up as calendar instead.)

Two refinements, both expressible as constraint floors:

• Work that runs on one core at a time ranks differently. Serial stages, and codes licensed per core (chip design is the classic), are buying single-core speed, not throughput. Pick the wrong shape and it costs between 14.5% and 39.3%, depending on which way you get it wrong. Both kinds of speed are first-class constraints: declare the shape of your work and the catalog query does the rest.
• On spot the distortion is bigger, not smaller — see section 9.

Honesty notes: these scores are the catalog's own index, measured per processor model — a signal for picking machines, not a benchmark of your code, and only directional when comparing ARM against x86. Run your own kernel before committing a run. Coverage grows continuously but isn't total: Azure's newest ARM line has scores for its smaller sizes in this pull (section 9 puts one to work), while its largest sizes are still awaiting samples — including two rows in this section's regional table. Unscored machines are never guessed at. They compete on price and size alone, and stay out of arithmetic-ranked picks until real numbers land.

9. The sweep, priced: embarrassingly parallel, pinned to nothing

The second workload, in full. The risk desk's nightly run is hundreds of thousands of independent scenarios, each a few minutes long, needing no checkpoints because a lost task is simply run again. Appetite: 500,000 core-hours a month, kilobytes in and out per task, no task ever aware of another. Where the simulation had to choose one market and commit, the sweep contests all of them at once — every machine bought on its own merits.

The cheapest qualifying spot capacity anywhere, same catalog day, ranked by price per core:

Cloud	Region	Instance	$/vCPU-hr	Risk band
Azure	newzealandnorth	Standard_F16ams_v6	$0.00208	0–5%
Google Cloud	us-west4	e2-highcpu-8	$0.00336	0–5%
Google Cloud	northamerica-northeast2	n2-highcpu-8	$0.00356	0–5%
AWS	ca-west-1	c6g.2xlarge	$0.00373	10–15%
Azure	centralindia	Standard_D8pls_v6	$0.00377	>20%
AWS	us-east-2	c6gd.2xlarge	$0.00384	>20%
Google Cloud	asia-south2	n2-highcpu-8	$0.00388	0–5%
Azure	centralindia	Standard_D8pls_v5	$0.00393	0–5%

Read that table twice. The floor sits in a region most teams have never priced in their lives. The fifteen cheapest rows span AWS, Google Cloud, and Azure across ten regions on four continents. And price rank does not track risk rank — two rows priced within a whisker of each other differ fourfold in how likely they are to be taken back.

Which is why a scavenger fleet should spread across these rows rather than pile onto rank #1: concentrating in one spot pool is exactly how a single reclaim wave eats a night's run. Spreading across markets is precisely what a single-cloud setup cannot do, and what one decision over the whole board does by default.

What it adds up to over the 500,000 core-hour month:

Scenario	$/vCPU-hr	Monthly	vs. on-demand
(a) Single-cloud on-demand (AWS us-east-1, c6g)	$0.0340	$17,000	—
(b) Single-cloud spot — best AWS-only row	$0.00373	$1,863	−89.0%
(c) Cross-cloud spot — top-5 blend, all clouds	$0.00330	$1,649	−90.3%

One honesty note on row (c): two of the five blended rows are Azure — priced live in the catalog, provisioning in final validation. Restricted to the clouds provisioning live today (AWS and Google Cloud), the top-5 blend prices at $0.00367/vCPU-hr — $1,836 a month, −89.2% — with the Oceania and India floors joining the blend as Azure validation completes.

The honest reading: the big jump is on-demand → spot, and any single cloud gives you most of it — if your team builds and operates the reclaim handling itself. Going cross-cloud is the last 11.5% off the best single-cloud spot bill, plus the part that won't fit in a table: the resilience of not being in one market.

The arithmetic lens from section 8 bites harder here, because a sweep is pure throughput. Re-pricing this table by measured compute keeps the same #1 — New Zealand wins both rankings — and then re-sorts the board in both directions. Upward: Azure's newest ARM row looks 81% above the floor on sticker price, but only 40% above it per unit of arithmetic — fresh silicon that the price-per-core lens mispriced in the buyer's favour, jumping it to #2. Downward: the oldest silicon in the table (Google doesn't even guarantee which generation of Intel you get) drops behind AWS's ARM row — and the legacy machines' mild-looking 61–79% price premiums balloon to 148–279% more per unit of arithmetic delivered. Shopping these same rows by measured compute rather than sticker price buys between two and a half and nearly four times more work per dollar.

The whole-workload ledger: cheaper and faster are the same knob

Budgets get approved as totals and deadlines as dates, so here are both workloads restated whole. One property of an elastic pool deserves its own line first: the bill is machine-hours × rate, so the size of the fleet cancels out. Double the machines and the days halve; the bill doesn't move. Time stops being a consequence and becomes a knob — which is why each table below carries a clock column next to the bill column, and why the fastest rows and the cheapest rows turn out to be the same rows.

The simulation (section 8): a fixed amount of arithmetic to get through. "Default" is what most teams do — a US region, cores as the unit:

Strategy	Fleet	Wall-clock	Total bill
Default: c6g, us-east-1, on-demand	~63 nodes	28 days	$45.7k
+ region arbitrage — c6g, ap-south-1	~100 nodes	17.5 days	$28.6k (−37%)
+ benchmark arbitrage — c8g, ap-south-1	~79 nodes	12 days	$19.6k (−57%)
+ deadline-first — c8g, ap-south-1	~158 nodes	6 days	$19.6k (−57%)

Read the last two rows together: doubling the fleet halved the calendar and left the bill untouched. Fleet sizes here are choices, not consequences — any fleet size on the same row costs the same, bounded only by how far the code scales (the cited GROMACS study holds >0.9 parallel efficiency to 512 nodes; the largest row above is 158). The worst sampled region, for reference, ran the same run at $74.1k.

The sweep (section 9): 500,000 core-hours a month; call the overnight window six hours, which today's on-demand fleet just fills:

Strategy	Nightly fleet	Window	Monthly bill
Today: on-demand, one region	~2,800 vCPU	6 h	$17,000
Cross-cloud spot, same fleet	~2,800 vCPU	6 h	$1,649 (−90%)
Cross-cloud spot, 10× fleet	~28,000 vCPU	~36 min	$1,649
Same bill as today, 10× the scenarios	~28,000 vCPU	6 h	≈$16.5k — 10× the science

The symmetry is the point: the same science for a tenth of the money, or ten times the science for the same money, and the clock comes free either way. Caveats stay on the table. A fleet ten times the size needs ten times the available capacity — which is exactly what spreading across the markets in section 9's table buys you. Interruptions requeue, which this shape of work shrugs off. And every figure is the 2026-07-22 snapshot, recomputable live.

When one cloud's spot market tightens — a reclaim wave, a capacity crunch — a single-cloud fleet stalls. A fleet with somewhere else to go re-lands on the next-cheapest rows and keeps running. On spot, that handling is the whole difference between a price and a bill: noticing the warning, draining the work, requeuing it, buying the replacement. It is the pool's job, not a script your team maintains once per cloud.

10. The data plane: Cloudflare R2 and the end of data gravity

The quiet force pinning HPC to one cloud isn't compute. It's data. Clouds charge nothing to put data in and roughly $0.09–0.12 per GB to take it out again, so 30 TB of inputs sitting in one cloud's storage means every burst somewhere else opens with a $2,700–3,600 invoice — every time. Results are worse: they pile up inside whichever cloud ran the job and leave at the same rates.

This is data gravity, and it quietly cancels every bit of placement freedom the sections above just bought. It is also not an accident of physics.

So this architecture keeps the data somewhere neutral: Cloudflare R2, which speaks the same storage API everyone already uses and charges nothing to read data out, at any volume (R2 pricing, checked 2026-07-22):

	Cloudflare R2	Typical hyperscaler object store
Storage (standard)	$0.015 /GB-month	~$0.02–0.023 /GB-month
Reads (class B)	$0.36 /million	~$0.40 /million
Writes (class A)	$4.50 /million	~$5 /million
Egress to internet	$0 — always	~$0.09–0.12 /GB (per GiB on Google Cloud) after ~100 GB free
Egress to another cloud's compute	$0	same as internet egress

Your tooling doesn't change — same commands, same libraries, one endpoint URL swapped. The pattern that fits HPC is pull, compute, push: copy inputs to the machine's local disk when the job starts, work against local disk, write results back at the end. This is a library, not a shared filesystem: scratch space stays local, and writes favour fewer, larger files so the per-operation charges stay in the noise.

What zero egress changes, per shape:

• The simulation (tens of TB of results): inputs land in any region of any cloud for nothing — the store charges nothing to send, the clouds charge nothing to receive. Where last year's data happens to live no longer constrains where this year's run runs.
• The sweep (kilobytes per task): a fleet scattered across ten regions reads the same scenario library, freely. There is no "wrong cloud" for a task to land in — which is exactly what lets every machine be bought on its own merits.
• Both: results stay readable forever — by the next stage of the pipeline in a different cloud, by analysis back at the lab, by a collaborator's laptop — with no exit toll on top. Your data stops being a hostage of wherever it happened to be computed.

The honest asymmetry: writing results out of the cloud that produced them still pays that cloud's exit fee once. Neutral storage can't waive another company's toll. The economics rest on paying it once, on results (usually a fraction of the input volume), instead of paying it again and again on inputs and re-reads. The repeat charges are the trap, and they go to zero.

Caveats that belong in the open: this store keeps no version history (put provenance in your naming or a manifest), its cold tier charges for retrieval, and its per-operation pricing punishes millions of tiny files. None of that bites the pull-compute-push pattern. All of it would bite anyone who tries to mount it like a filesystem.

11. Why this stack, on this pool

Each advantage here is structural — a consequence of where the platform sits, not a bolt-on:

1. One integration instead of one per cloud. Multi-cloud Slurm today means running a separate provisioning toolchain for every cloud — separate scripts, images, permissions, quotas — plus a human deciding between them. Here the whole board answers one signal, and the decision is a catalog query.
2. The whole board, in one decision. Per-cloud tooling optimizes inside the cloud it belongs to. Nothing in that toolbox prices the choice section 8 showed was worth 45–61% before spot even enters: which cloud and region the cluster should exist in at all.
3. Spot survival is built in, not rebuilt per site. Noticing the reclaim, draining, requeuing, replacing — risk-scored when buying, healed when losing. The very burden that keeps most HPC fleets paying on-demand prices is the part that arrives as platform.
4. Arithmetic, not core counts. HPC buys work done, so the catalog prices machines by measured compute and treats speed floors as first-class constraints. "Cheapest that qualifies" therefore means cheapest per unit of work, not per marketing core. In this paper's own tables that is worth another 31–37% inside the already-cheapest region (section 8), and it exposes spot picks priced 148–279% above the true floor (section 9).
5. Every customer gets their own cluster. Own control plane, own private networks, own certificate authority, own encrypted fabric. For the pharma, chip-design, and defense-adjacent work that dominates commercial HPC, that isolation is a structural given rather than a compliance addendum.
6. Your accounts, no markup, both doors open. Machines are bought at cost in accounts you own, the scheduler on top is unmodified open-source Slurm, and the cluster underneath is standard Kubernetes. Adopt it as a service; walk away with a working cluster. The exit is part of the product.

The field, by name

Running Slurm in the cloud is a solved problem many times over, and this paper does not pretend otherwise. Here is the field as it actually stands in mid-2026 — good products, named, with the axis that separates this architecture from each:

Route	Your Slurm survives?	Who picks cloud · region · machine	Spot lifecycle	Reach	What the tool costs (compute always extra, at each cloud's rates)
Single-cloud builders — AWS ParallelCluster, Azure CycleCloud, Google Cluster Toolkit	Yes — you operate everything	You · you · you, in config files	Requeue hooks; policy is yours to build	One cloud	Free tools — plus the platform team that runs them
Managed single-cloud Slurm — AWS PCS, Azure CycleCloud Workspace, Google Cluster Director	Yes — vendor runs the control plane	Cloud is fixed · you · you, as static node groups	Requeue; healing varies by vendor	One cloud	PCS: controller $0.58–6.54/hr + $0.08/instance-hr ($0.64 on P/Trn); CCWS and Cluster Director: free
GPU clouds with Slurm included — CoreWeave SUNK, Nebius, Lambda, Crusoe	Yes — theirs, managed	Their fleet · their DCs · contract shapes	Reserved contracts; no spot market to play	Their fleet	Bundled into committed $/GPU-hr on their fleet — the compute is the price
HPC SaaS portals — e.g. Rescale, with its own Slurm burst connector	Portal-mediated	Multi-cloud, from their offer sheet, at their rates	Varies	Many clouds, their platform	Their core-hour rates, commitment-priced — platform, compute, and licenses bundled; list not public
Spot-rescue overlays — MemVerge, Exostellar	Yes — untouched	Wherever you already are	Checkpoint or live-migrate off reclaims — genuinely good	Per cloud	Commercial license, vendor-quoted
Client-side job brokers — SkyPilot	No — replaces the front end your users speak	Per job: cloud, region, SKU, price-aware	Job-level auto-recovery	Many clouds, your accounts	OSS free (you operate it); commercial platform pricing undisclosed (launched Jul 2026)
This paper's stack	Yes — unmodified, either doorway	The platform: live catalog, benchmark floors, risk scores	Detect · drain · requeue · replace, operated	Every region of AWS, Google Cloud, Azure — your accounts	Flat platform fee, sized under your audited savings — never per node, never a markup

Tool prices are published list where public (PCS, the free tools) and vendor-model descriptions where not (Rescale, the GPU clouds' bundles, SkyPilot's platform), all as of 2026-07-22 — the meter arithmetic below shows what the shapes of these prices do to a real workload, which matters more than the rates themselves.

Read the third column all the way down. In every row but the last, the answer to which cloud, which region, which machine is a human being or a config file. Everywhere else on this table the list of machines is an input. Here it is an output.

Credit where it is due. SkyPilot's broker really does price-shop clouds per job — but it asks your users to stop speaking Slurm, it prices cores rather than measured arithmetic, and between jobs it hands the standing platform back to you. The GPU clouds' Slurm offerings are genuinely good and their contract rates often beat hyperscaler list, so if you are training on GPUs, price them; this paper's economics are about the CPU-heavy batch estate, where a single-fleet vendor has no market to shop on your behalf. And Rescale's catalogue of licensed simulation software, managed end to end, is real value this stack does not replicate — its compute simply prices as a platform's committed core-hours rather than as your own accounts at cost.

What the meters and the walls do to section 9's sweep

Numbers, because they are published. AWS PCS — the most complete managed-Slurm product in the field — adds two fees on top of the machines (published rates, us-east-1, 2026-07-22): a controller at $0.58–6.54/hr depending on cluster size, and $0.08 per machine per hour for management.

Now run section 9's sweep through that meter. The cheapest AWS row on the board is a small machine at about $0.030/hr — so the $0.08 management fee is 2.7 times the price of the thing it manages. Across the month, the fees (≈$5,000 of machine metering plus $2,346 for a controller that never sleeps) cost more than the $1,863 of compute they meter. The identical work on the cross-cloud pool bills $1,649, all in. To be fair: on the big on-demand machines of section 8's simulation the same meter fades to a defensible 9–12%. It is priced for the traditional centre of HPC, and it lands hardest on exactly the scavenger economics this kind of work depends on.

Azure's and Google Cloud's tools charge no fee at all. Their cost is the wall. Each sees one cloud, so what you pay is whatever your cloud's slice of section 9's table happens to hold that day: the floor when your cloud has it, 61–79% above when it doesn't — and it rotates. The fee-free tools also leave every actual decision — region, silicon, which spot pool, how to spread — sitting in your team's spreadsheet, which is what the third column above is really about. The comparison this paper cares about isn't fees. It is that no tool in the table, free or metered, shops the whole board — by measured arithmetic, with the reclaim handling operated for you, underneath the Slurm you already run.

12. Adopting it: Audit → Extend → Serve

The same three phases as every Multicloud engagement, each with standalone payoff, mapped to an HPC estate:

1. Audit (read-only, days, zero risk). Point the Advisor at what you run today. It computes the comparison no single-cloud tool can: what your actual job mix would have cost on the best cloud, region, machine, and pricing model — corrected for real processor speed and for data charges. It measures; it changes nothing. What sections 8 and 9 did for the simulation and the sweep, it does for whatever you actually run.
2. Extend (early access — start with one partition). Attach the burst partition behind your existing Slurm cluster (section 5). Pick the lowest-risk queue you have — the overnight sweep, the student partition, the deadline overflow — and watch cost and reclaim behavior in your own accounting. The dial runs from 5% overflow to most of the fleet, one slurm.conf edit at a time.
3. Serve (design-partner program). Stand up the front door of section 6 — portal, SSO, REST — for the teams that should never have to think about the machinery. When you want the keys, the pool underneath is a standard cluster in your accounts: promotion to self-operated is a configuration step, not a migration.

The savings pay for it. A flat platform fee, priced under your audited savings — never per-node, never a percentage of savings, never a markup on machines. The audit comes first precisely so the fee is justified by a number you measured yourself.

Where the two workloads end up

	The simulation	The sweep
Placement	Affinity: cloud + region — one market, chosen by live price and capacity, held for the run	Affinity: none — every node purchase contests the whole board, re-decided continuously
Pricing model	On-demand, or low-risk spot where it can checkpoint	Spot, diversified across markets; reclaims requeue
The bill	$45.7k → $19.6k (−57%): region arbitrage, then benchmark arbitrage	$17,000/mo → $1,649/mo (−90%): on-demand → cross-cloud spot blend
The calendar	28 days → 6 days, at the same $19.6k — fleet size is a knob, not a consequence	6 h → ~36 min at 10× fleet, same bill — or 10× the scenarios for today's money
Doorway	Served front door (section 6) — the lab runs science, not a cluster	Extend your cluster (section 5) — one burst partition, everything else untouched
What it needed from the platform	The right market, bought once	Every market, bought again every night

Read across the last two columns and the paper's claim is visible in one line: the simulation and the sweep want opposite things from a capacity layer, and neither is served by picking a cloud in advance. One needs the best single market that exists on the day it starts; the other needs all of them, continuously. The layer that can answer both questions is the same layer, and it is the one Slurm has always assumed someone else would build.

Whichever one you recognise, start the same way. If you run something like the sweep — an overnight queue, a rendering farm, a genomics pipeline — it is the cheapest thing you own to prove: the Audit prices it read-only in days, and one burst partition then tests the number for real with your baseline untouched. If you run something like the simulation — a one long coupled job with a date attached — the Audit re-prices your next run against every region on the board before you commit a single node-hour, and tells you what the calendar costs at each fleet size.

What's built, what's early, what's reference architecture

Piece	Status
Catalog, scheduler, spot-aware placement, reclaim healing, cross-cloud fabric	In production (AWS and Google Cloud provisioning live; Azure provisioning in final validation — Azure pricing already live in the catalog)
Benchmark floors, location rules, placement affinity, spend caps	Built
Advisor (read-only savings counterfactual)	Built — install is a Helm chart
Extend your cluster (virtual node, burst tier)	Early access — design-partner phase
Burst partition for classic Slurm (ResumeProgram → pool)	Reference architecture — built with design partners on the mechanisms above
Hosted Slurm front door (Slinky + Open OnDemand + slurmrestd behind OIDC)	Reference architecture — standard upstream parts on the platform's serving stack, assembled per engagement
Slurm, Slinky, Open OnDemand, Cloudflare R2	Upstream projects and third-party services, used unmodified at arm's length (Slurm and Slinky are stewarded by SchedMD — an NVIDIA subsidiary since December 2025, with continued open-source, vendor-neutral development pledged)

Sources

• Slurm elastic computing / power save: slurm.schedmd.com/power_save.html
• Slurm federation: slurm.schedmd.com/federation.html
• Slurm REST API and JWT auth: slurm.schedmd.com/rest.html
• Slinky / slurm-operator: slurm.schedmd.com/slinky.html · github.com/SlinkyProject/slurm-operator · NVIDIA on Slinky
• NVIDIA's acquisition of SchedMD (announced December 2025; open-source, vendor-neutral continuation pledged): nvidia.com/en-us/software/slurm and contemporaneous press
• Soperator: github.com/nebius/soperator
• The field (section 11), per vendor documentation as of 2026-07-22: AWS PCS + pricing (rates cross-checked against the AWS Price List API) · AWS ParallelCluster · Azure CycleCloud Workspace for Slurm · Google Cluster Director (no service charge; underlying resources only) · CoreWeave SUNK · Nebius Soperator · Lambda Managed Slurm · Crusoe Managed Slurm · Rescale · SkyPilot · MemVerge SpotSurfer · Exostellar
• Open OnDemand OIDC: osc.github.io/ood-documentation
• GROMACS scaling: Kutzner et al., J. Comput. Chem. 2025 — doi:10.1002/jcc.70059
• Cloudflare R2 pricing and S3 API: developers.cloudflare.com/r2/pricing · S3 API compatibility
• Cloud egress rates: AWS/Azure/Google Cloud published bandwidth pricing (tiered; ~100 GB/month free; verify against live calculators — figures cited are typical first-tier rates as of 2026-07)
• Compute prices and CPU benchmark index: Multicloud live catalog, pulled 2026-07-22 (reproducible via the catalog API with the query parameters held in this paper's source notes; benchmark scores are the catalog's per-CPU-model index, pooled across clouds)
• Platform: the Multicloud Architecture Whitepaper — multicloud.io