At a glance
- What an AI agent really costs is more than the build: usage, hosting, monitoring, review and maintenance recur for as long as it stays in production.
- An operating cost model with eleven cost lines — from model usage to internal owner time — makes those recurring costs visible before you commit budget.
- Usage-based cost scales with request volume and tokens per request; the method below estimates it from volume and your provider's own rate, without naming a rate.
- Human review time is a cost line to size like any labor cost, not a gap that closes on its own.
- Several cost lines can be small in year one and grow afterward: retraining, integration upkeep and security review all depend on time in production.
Why the build can become the smaller number over time
It's easy to build an AI project budget around one number: what it costs to design, integrate and launch the system. That number is real, and it's a one-time cost. What an AI agent really costs, though, includes what happens after launch: every request it answers, every output someone checks, every upstream system it stays connected to carries a cost that recurs for as long as the system runs.
Generative AI workloads make this harder to see coming than typical software cost. A Google Cloud post on AI cost controls puts it plainly: generative AI can make cloud costs difficult to predict, since a single short prompt can run complex operations and generate significant cost [1], and traditional metrics like requests per second no longer help estimate the bill [1]. A system that looks inexpensive at pilot volume can carry a different total once it's in production, because the bill moves with usage, not with a project plan.
- Build: design, integration and testing — priced once, before launch.
- Run: usage, hosting, monitoring, review and maintenance — priced for as long as the system stays in production.
The operating cost model
AI cost doesn't sit neatly inside one budget category. The FinOps Foundation's framework for AI workloads puts it directly: AI cost and usage is not only new to many organizations and very granular, but also tends to transcend technology category boundaries [2]. The table below turns that into line items: eleven cost lines, what drives each one, how to estimate it, and whether it's one-time or recurring.
| Cost line | Driver (what makes it grow) | How to estimate it | One-time or recurring |
|---|---|---|---|
| Model / API usage | Request volume; tokens per request; chained model calls | Monthly requests × average input/output tokens × your provider's rate for the model tier you'll run in production | Recurring |
| Hosting and infrastructure | Concurrency, latency targets, self-hosted vs. managed | Price compute, storage and networking for the serving tier as for any production service | Recurring (setup is one-time) |
| Monitoring and observability | Number of production endpoints; alerting granularity | Price logging, tracing and anomaly detection as its own line; check your cloud provider's billing for how it's metered | Recurring |
| Evaluation runs | How often you re-test output quality, and dataset size | Compute cost per run × run frequency, plus reviewer time on results | Recurring |
| Human review time | Share of outputs checked, and how deep the check goes | Reviewed volume × minutes per review × loaded hourly rate | Recurring |
| Incident response | How often the system errs in a way that needs a person, and diagnosis cost | Expected incidents per period × average person-hours to resolve one | Recurring |
| Retraining / prompt maintenance | How quickly the task, data or underlying model changes out from under you | Engineering hours per update cycle × cycles per year | Recurring |
| Integration upkeep when source systems change | How often connected systems (ERP, CRM, document stores) change schema, API or vendor | Hours per upstream change × expected changes per year | Recurring |
| Security reviews | The system's risk classification and your policy's audit cadence | Review hours per cycle × cycles per year | Recurring (a first review held during the build is one-time) |
| Vendor / license | Seat count, usage tier and contract terms for any dependent third-party tool | The vendor's quoted fee × seats or units needed | Depends on the contract |
| Internal owner time | Coordination and oversight the system needs from a named internal owner | Hours per week × loaded rate × weeks per year | Recurring |
Estimating usage-based costs
Usage-based cost is easy to get wrong, because it scales with adoption instead of staying fixed. One provider's pricing page shows the shape of it: a regional-inference option is priced as a multiplier on input and output tokens [3], the same page discounts batch processing [3], and it lists more than a dozen separately priced models [3]. No rate is quoted here; check your provider's current pricing page for the number that applies to you.
Because rates vary by provider, model tier and contract [3], and change over time, the method below is written in variables, not numbers.
- Estimate monthly request volume
How many times the system will be called each month, based on current process volume or a realistic adoption forecast, not the optimistic case.
- Estimate input and output tokens per request, separately
Average input tokens (prompt, context, retrieved documents) and average output tokens, kept separate, since combining them hides which side drives the cost.
- Get your provider's current rate for each token type
Look up the input and output rate for the model tier you'll actually run in production — rates differ by tier and change over time.
- Combine volume, tokens and rate
Multiply monthly requests by tokens per request by the matching rate, separately for input and output, then add the two together.
- Add auxiliary calls
Price any calls beyond the main model the same way: retrieval, embeddings, a routing or classification model, and tool calls.
- Build a range, not a point estimate
Run the calculation at a low, base and high volume — usage is one of the hardest inputs to predict before real traffic exists.
- Reconcile against real usage after launch
Compare actual spend to your estimate every month once live, and update the model. An estimate is a starting point, not a forecast to lock in.
Human review is a cost line
It's tempting to model human review as a placeholder that shrinks to zero once the system matures. We recommend budgeting review as a permanent line: the question is how much review to keep, not whether.
Size it the way you'd size any labor cost, using parameters you set from your own operation, not figures this article can hand you:
- Reviewed share: what fraction of outputs get checked — every output, a sample, or only flagged ones.
- Minutes per review: how long a competent reviewer actually takes, measured, not guessed.
- Escalation rate: how often a review becomes a longer investigation instead of a quick approval.
- Reviewer seniority: whether it needs the person who does the work today, or someone more junior.
Where the review sits in the workflow is a governance decision, not just a cost one — our guide to human-in-the-loop AI governance covers designing the checkpoint. This section only prices the time it takes once designed.
Costs that appear in year two
Several line items can be small in year one and grow afterward, because their triggers occur with time in production: connected systems change, the task or data drifts from what the system was tuned on, and the audit cadence your policy sets catches up with it. Budgeting only for month one misses these.
This isn't specific to one vendor. Using the technical-debt framework from software engineering, research on ML systems finds it common to incur massive ongoing maintenance costs in real-world systems [4]. In the cost model above, that maintenance shows up in the retraining and integration-upkeep lines.
| Cost line | Why it can grow after year one | What triggers it |
|---|---|---|
| Retraining / prompt maintenance | Tuned against the task and data at launch; both can drift | A new document type, a policy change, or a shift in request mix |
| Integration upkeep | Connected systems don't stay static | An ERP upgrade, a CRM migration, or a vendor API change |
| Security reviews | A first review scheduled during the build is part of build cost; recurring reviews aren't | Your audit cycle, or a new data source feeding the system |
| Incident response | Edge cases surface as volume and input variety grow | Traffic beyond what was tested, or use in a new part of the business |
Internal owner time is easy to leave out of the first-year budget: someone must track the above, decide when a retrain is warranted, and own the relationship with the teams whose systems you integrated with. AI operations and enablement is specifically about building that ownership into your own team, rather than leaving it with an outside vendor by default.
Putting TCO next to value
None of the cost lines above matter in isolation; they matter next to what the system returns. The companion piece to this article, how to calculate AI ROI before you build, covers the value side: status quo cost, recoverable portion, and payback. Put the two together: payback period is build cost divided by (recovered value per period minus the recurring operating cost this article covers).
Run cost isn't a footnote to that calculation; it's a term in it. A strong recovery estimate paired with a heavy operating cost model can still be a weak investment once run cost is netted out — which is why we rank AI opportunities by cost and time-to-value, not just by upside, inside a 360° AI Blueprint.
When this cost model doesn't apply
- The system is small, low-volume and deterministic (rules-based logic, no model calls) — budget it like conventional software instead.
- You're evaluating a single prototype with no production traffic — monitoring and incident-response cadence scale with production use and shouldn't be sized yet.
- A vendor-hosted point solution already bundles usage, hosting and monitoring into one subscription — track the vendor's price instead of modeling each line.
- The workflow changes so often any estimate would be stale within a quarter — revisit it on a fixed schedule instead of treating it as one-time work.
Frequently asked questions
What is the TCO of an AI agent?
The total cost of ownership of an AI agent is the one-time build cost (design, integration and testing) plus every recurring cost it carries once live: model or API usage, hosting, monitoring, evaluation, human review, incident response, retraining and internal owner time. The operating cost model above lists each line with how to estimate it.
What are the hidden costs?
The hidden costs are the ones a build quote doesn't show, because they appear once the system is running: human review time, evaluation runs, incident response, integration upkeep when a connected system changes, security-review cycles, and internal owner time to coordinate it. Budget for them from day one.
How much does AI maintenance cost per year?
There's no single figure: it depends on your engineering hours per update cycle, how often the task or data drifts, and how often connected systems change. Estimate it as hours per retraining or integration-upkeep cycle × expected cycles per year, at your own team's loaded rate — the year-two table above lists the triggers to watch.
How do you forecast LLM API costs?
Estimate monthly request volume, then average input and output tokens per request separately, then apply your provider's current published rate for the model tier you'll run in production. Multiply volume by tokens by rate for each token type, add auxiliary calls like retrieval or routing, and build a low, base and high range rather than one number.
Build vs buy: which has lower TCO?
It depends which cost lines a vendor bundles into its fee versus which you'd carry yourself. A bought point solution can fold usage, hosting and monitoring into one subscription; a built system spreads those across lines you control. Compare both with the same operating cost model, line by line, not a subscription price against a build estimate.
Related reading
- The value side of this model: how to calculate AI ROI before you build.
- How we rank opportunities by cost and time-to-value: 360° AI Blueprint.
- Building the internal ownership this cost model assumes: AI operations and enablement.
- What to check before you commit budget: 12 questions for an AI readiness assessment.
- Questions worth asking any vendor whose fees become a cost line: questions to ask an AI consulting firm.
Sources
- Google Cloud, "Detect early and enforce firmly with Google Cloud's enhanced cost controls for AI spend" (2026-07-28) (accessed 2026-09-28)
- FinOps Foundation, "FinOps for AI" (FinOps Framework technology category, retrieved 2026-09-28) (accessed 2026-09-28)
- Anthropic, "Pricing" (claude.com, retrieved 2026-09-28) (accessed 2026-09-28)
- D. Sculley et al., "Hidden Technical Debt in Machine Learning Systems", Advances in Neural Information Processing Systems 28 (NeurIPS 2015) (accessed 2026-09-28)
More in AI strategy
AI Agents vs. RPA for Hotel Back-Office Operations: When Bots Break and What Replaces Them
AI agents vs RPA for hotel back-office operations: a decision table for OTA statement matching, folio audit and multi-property AP, plus how to phase bots.
Hotel Management CompaniesWhat a 360 AI Blueprint Costs for a Food & Beverage Manufacturing Back Office
What drives food and beverage manufacturing AI cost in the back office: a table of plants, co-packers, deductions and traceability scope, plus how to budget.
Food and Beverage ManufacturersAI Implementation Cost for Multi-Rooftop Dealer Groups: What Changes When You Add a Store
AI implementation cost for multi-rooftop dealer groups: a driver table showing how DMS count, OEM portals, build type and upkeep move cost per added store.
Auto Dealerships