Solution
We cannot see where our AI spend actually goes
The invoice arrives as one number. Nobody can attribute it to a team, a feature or a customer.
AI infrastructure spend is unusually hard to attribute. It arrives across GPU instances, hosted model APIs, storage, data movement and a handful of platform tools, none of which agree on how to label anything. The result is a large number that finance cannot break down and engineering cannot defend.
This matters beyond tidiness. Without attribution you cannot tell which feature is profitable, which team is driving growth in spend, or whether a proposed optimisation is worth the engineering time. Every other cost decision on this list depends on getting this right first.
What is usually going on
No consistent tagging across providers
Hyperscalers, specialist GPU clouds and model APIs all label resources differently. Without one scheme applied everywhere, allocation is guesswork.
Shared clusters with no internal metering
When several teams use one cluster, the invoice shows a total. Attributing it needs usage measured at the scheduler, not the invoice.
Model API spend outside the cloud bill
Hosted model usage often sits on a separate invoice, sometimes on a credit card, and rarely lands in the same reporting as infrastructure.
Cost measured per resource, not per outcome
Cost per GPU-hour is easy and not very useful. Cost per thousand requests, per active user or per completed task is what informs a decision.
Reporting nobody reads
A dashboard that is not tied to a target and an owner changes nothing. The reporting cadence matters more than the tooling.
What we look at, in order
- 01 Inventory every source of AI spend, including the ones outside the main cloud account
- 02 Define one allocation scheme — team, environment, workload, customer — and apply it consistently
- 03 Instrument usage at the scheduler and gateway so shared capacity can be attributed
- 04 Agree the unit metric the business will actually be managed on
- 05 Set the reporting cadence and name an owner for the number
What you receive
- A complete map of where AI spend originates today
- Tagging and allocation scheme, documented and ready to apply
- Recommended metering points and tooling, with trade-offs
- Unit-economics definition and a reporting template
Delivered as a written assessment. More on how we work.
Products relevant to this
Not a shortlist for you specifically — that depends on constraints this page cannot know. These are the options an engineering team addressing this problem will encounter first.
LiteLLM
L4 · Inference gateway
A gateway presenting one consistent interface across many hosted model providers and your own self-hosted models.
- Fit
- Applications calling more than one model provider, or planning to.
- Catch
- Adds a network hop, and sits on the critical path of every request.
Run:ai
L4 · GPU pooling and fractional sharing
A Kubernetes-based orchestration layer that pools accelerators across teams and allocates fractions of one to a workload.
- Fit
- Shared clusters where several teams compete for fixed capacity.
- Catch
- Assumes you already run Kubernetes, and run it well.
AWS, Google Cloud and Azure
L3 · Hyperscale cloud
The three large general-purpose clouds, each offering accelerated compute alongside everything else you already run.
- Fit
- Estates where the data, identity and compliance posture already live.
- Catch
- Generally the highest hourly rate, and quota is often the real limit.
Other problems
Tell us what you are trying to solve.
Describe your current setup and the problem in your own words. We reply within 2 working days with an honest assessment of whether we can help.