Solution
We are paying for accelerators that sit idle
Reserved or owned capacity goes unused while work queues elsewhere — and preempted jobs lose their state.
Idle accelerators are the most expensive thing in most AI estates, because the cost is committed whether the work happens or not. It is also the problem most often misdiagnosed as needing more capacity.
Low utilisation is almost always a scheduling and allocation problem rather than a hardware one. When capacity is divided statically between teams, one team queues while another’s allocation sits unused — and both are being paid for.
What is usually going on
Capacity carved up per team and never shared
Static allocation guarantees idle time. Pooling with fair-share queueing and internal priorities recovers a surprising amount without buying anything.
Whole accelerators given to work that needs a fraction
Common for development, notebooks and smaller inference services. Partitioning or sharing changes the economics substantially.
People hold nodes they are not using
When engineers cannot rely on getting capacity back, they keep it. That behaviour is rational, and it is expensive — the fix is a queue people trust, not a policy memo.
Interruptible work paying uninterruptible prices
Batch and training jobs can usually tolerate interruption if checkpointing and automatic restart are handled properly. Many teams pay full rates because the tooling cannot recover, not because the workload demands it.
Preemption loses state instead of pausing it
A spot instance reclaimed mid-run should cost you minutes, not a day. If a checkpoint was written to the node that just disappeared, or nothing finds replacement capacity without a human, the discount was never real.
Utilisation is not actually measured
Allocation and utilisation are very different numbers. Most teams know the first and assume the second.
What we look at, in order
- 01 Measure real utilisation against allocation, per team and per workload
- 02 Review how capacity is divided and whether it can be pooled safely
- 03 Assess fair-share queueing, internal preemption and accelerator partitioning
- 04 Identify which workloads could safely run on interruptible capacity
- 05 Size what you actually need before renewing any commitment
What you receive
- Measured utilisation baseline with the gap quantified
- Scheduling and allocation design, including queue policy
- Recommendation on partitioning and interruptible capacity
- Right-sized capacity plan ahead of your next renewal
Delivered as a written assessment. More on how we work.
Products relevant to this
Not a shortlist for you specifically — that depends on constraints this page cannot know. These are the options an engineering team addressing this problem will encounter first.
Run:ai
L4 · GPU pooling and fractional sharing
A Kubernetes-based orchestration layer that pools accelerators across teams and allocates fractions of one to a workload.
- Fit
- Shared clusters where several teams compete for fixed capacity.
- Catch
- Assumes you already run Kubernetes, and run it well.
SkyPilot
L4 · Multi-cloud workload orchestration
An open-source layer that runs the same job on whichever cloud, specialist GPU provider or Kubernetes cluster has capacity.
- Fit
- Teams using more than one compute provider, or wanting to.
- Catch
- You still need accounts and quota with every provider it places work on.
Kubernetes batch scheduling
L4 · Batch scheduling
Add-ons giving Kubernetes the queueing and gang-scheduling behaviour AI workloads need and plain Kubernetes does not provide.
- Fit
- Organisations already standardised on Kubernetes.
- Catch
- You assemble a scheduler from components rather than buying one.
Other problems
Tell us what you are trying to solve.
Describe your current setup and the problem in your own words. We reply within 2 working days with an honest assessment of whether we can help.