FinOps for AI

FinOps for AI is the FinOps operating model applied to AI spend. Engineering, product, finance and procurement share timely data on what AI costs and produces, and decide together what is worth paying for. In the FinOps Foundation’s words, it “focuses on addressing the cost complexity, faster development cycle, spend unpredictability, and the need for a greater degree of policy and governance to support innovation through allocation, forecasting, and optimization decisions that align consumption, investment, and business value.”^[1]

What is new is the meter: tokens, GPU hours, capacity paid around the clock, and AI credits in SaaS. Tokens are the pieces of text, image or audio a model reads and writes, priced per million and by type. In one line: put the cost of each useful AI outcome in front of the people who choose the model, the prompt and the capacity, early enough to change them.

For FinOps, engineering, product, finance and leaders (routes by role); example prices are illustrative.

AI spend breaks four habits formed on virtual machines.

1. The unit hides multipliers. Output tokens often cost “three to five times more than input tokens”, and stateless APIs resend the history each turn.^[8] Reasoning tokens are invisible yet “billed as output tokens”.^[9] Anthropic’s tokenizer for Claude 4.7 and later “produces approximately 30 percent more tokens for the same text than earlier models”.^[10]
2. Usage compounds. Agent multipliers “show up only on the invoice”,^[11] and unit cost “is not a fixed price. It is a distribution.”^[8]
3. New spenders, channels and lifetimes. Product, marketing and sales create AI spend, often on SKUs that cannot be tagged natively.^[12] From June 2026 all GitHub Copilot plans bill by “the model and the number of tokens consumed”.^[13] Provisioned capacity is charged “whether or not the deployment is handling requests”,^[14] yet “You cannot commit for three years to something with a six-month shelf life.”^[6]
4. Value is unproven. In the State of FinOps 2026, 98% of respondents (self-selected FinOps practitioners) manage AI spend, up from 31% in 2024. Their top AI challenges are visibility, allocation and ROI. One said: “Is your AI providing value? No one can answer that question yet.”^[15]

Per-token pricing made design a cost decision; GPU scarcity made capacity a commitment. Traceable milestones:

• 2023: FrugalGPT reports LLM API fees that “can differ by two orders of magnitude” and proposes model cascades.^[16]
• 2024–2025: Foundation working groups map AI services (December 2024), infrastructure choices and forecasting.^[12][17][18]
• 2026: AI is a technology category of the Framework;^[1] on 3 June the Linux Foundation announces its intent to launch the Tokenomics Foundation;^[4] from July the working draft of FOCUS (the FinOps Open Cost and Usage Specification) gains model and token properties, not yet released.^[7]

The evidence is mostly Foundation guidance, provider documentation and a few papers; no peer-reviewed study of the practice was found, so this page omits unsupported savings figures.

An LLM API counts each call’s tokens by type and multiplies by a price per million; images and audio become tokens too, usually at higher prices.^[8]

• The cost of one model call is (uncached input + cache reads × r + cache writes × w) × input price + output × output price, times a tier factor (batch, flex, priority) and any context-length or regional modifier. Typically r = 0.1 and w = 1.25 (October 2026).^[10][19] Most APIs count hidden reasoning inside output, but not all (conventions).
• The cost of an outcome adds capacity paid whether used or not, retrieval, tools and people, divided by successful outcomes.
• Call growth: the Tokenomics Foundation’s informal “Big-T” notation writes consumption as T(n · k · a), “requests × model calls per request × agent depth”, from T(1), a cache hit, to T(∞), endless loops.^[11]

%%{init: {"flowchart": {"wrappingWidth": 170, "nodeSpacing": 28, "rankSpacing": 34}}}%%
flowchart TB
  O["Outcome: one resolved case, document or decision"] --> N["n: requests per outcome"]
  N --> K["k: model calls per request: retries, tool turns, reasoning steps"]
  K --> A["a: agent depth: sub-agents that call models"]
  A --> T["Tokens per call: uncached input, cache read, cache write, output with reasoning"]
  R["Retrieved context: chunks × chunk size; images resent each turn"] --> T
  T --> P["Price per token type × tier × context length × region"]
  P --> S["Cost per successful outcome"]
  C["Capacity: provisioned units, GPU-hours, vector store; paid whether used or not"] --> S
  H["People: review, escalation, operations"] --> S
  classDef core fill:#e8f3f2,stroke:#16756f,color:#122029
  classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029
  classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029
  classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029
  class O,S decision
  class N,K,A,R risk
  class T,P core
  class C,H neutral

• Cache read and write; context window creep: reusing a stored prompt prefix at a discount after paying a premium to store it; input growing as each turn resends the history.^[8]
• Service tier: one model priced by speed: batch, flex, priority or fast, reserved (comparison).
• Provisioned throughput: capacity reserved for a term in provider units (PTUs on Azure; model units or reserved tokens per minute on Bedrock);^[14][20][21] spillover sends the excess to pay-as-you-go.^[22]
• Idle allocated and unallocated capacity: deployed but unused; reserved but assigned to no model.^[23]

%%{init: {"flowchart": {"wrappingWidth": 150, "nodeSpacing": 30, "rankSpacing": 50}}}%%
flowchart TB
  FO["FinOps: allocation rules, commitments, cadence"] -->|"generic mechanics"| FA["FinOps for AI: unit economics, capacity and budget decisions"]
  MO["ModelOps: approved models, risk tiers"] -->|"allowed models"| FA
  ML["MLOps and KubernetesOps: training jobs, GPU pools"] -->|"GPU-hours, utilisation"| FA
  LL["LLMOps: routing, caching, evaluations"] <-->|"quality results and cost targets"| FA
  FA -->|"demand forecasts, unit cost"| PR["ProcurementOps: contracts"]
  FA -->|"usage and cost data"| GR["GreenOps: energy and carbon"]
  classDef core fill:#e8f3f2,stroke:#16756f,color:#122029
  classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029
  class FA core
  class FO,MO,ML,LL,PR,GR neutral

The six FinOps principles apply unchanged (FinOps). This Atlas proposes six AI-specific ones:

1. Price the outcome, not the token: “Consider the total cost to get a successful outcome.”^[8]
2. Quality floor first: benchmark quality against cost “for their specific use case, not against generic benchmarks”.^[24]
3. Bound the multipliers: OWASP’s controls against “Denial of Wallet” include input validation, rate limits, timeouts and limits on queued and total actions;^[25] step, retry and output caps are this Atlas’s agent version.
4. An owner on every call: multi-agent allocation lacks a “generally accepted” framework,^[1] so tag at the call.
5. Commit only to what you will still use: each unit above break-even, for no longer than you keep the model.
6. Fund in stages and count people: set “the limits of both costs and time” before a pilot,^[12] and include human review in outcome cost.

The Foundation recommends an “AI Investment Council or similar body to regularly approve, evaluate, and track the impact of AI projects”.^[1]

RACI for recurring decisions (this Atlas’s proposal). R = responsible, A = accountable, C = consulted, I = informed.

The Foundation describes Crawl (validate feasibility, minimal cost, fail fast), Walk (integrate, budgets split) and Run (core processes, higher non-functional requirements) for AI initiatives,^[12] and investment statuses from proof of concept to scaling.^[1] The gates below are this Atlas’s proposal.

%%{init: {"flowchart": {"wrappingWidth": 170, "nodeSpacing": 26, "rankSpacing": 34}}}%%
flowchart TB
  I["Intake: value hypothesis, outcome unit, quality floor"] --> P1["Proof of concept: time box and cost cap"]
  P1 --> G1{{"Gate 1: does it work at all?"}}
  G1 -->|yes| P2["Proof of value: real traffic, cost per successful outcome"]
  G1 -->|no| X["Stop and record the lesson"]
  P2 --> G2{{"Gate 2: value exceeds cost with a margin?"}}
  G2 -->|no| X
  G2 -->|yes| SC["Scale: capacity model, budgets, forecast"]
  SC --> RN["Run: weekly unit cost, monthly review"]
  RN -->|"model, price or volume change"| RE["Re-evaluate cost and quality"]
  RE --> RN
  RN -->|"value gone or model retired"| RT["Retire: release capacity, revoke keys"]
  classDef core fill:#e8f3f2,stroke:#16756f,color:#122029
  classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029
  classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029
  class I,P1,P2,SC,RN,RE core
  class G1,G2 decision
  class X,RT risk

The cadence (this Atlas’s proposal) is faster than general FinOps because the Foundation expects “shorter forecasting windows” for AI^[1] and suggests weekly or monthly forecasting.^[18]

The architecture is a cost pipeline joined to request telemetry and business outcomes, with controls acting on the gateway and the providers (figure below). Its inputs:

• Provider cost APIs: OpenAI’s report daily buckets by project, line item or key;^[26] Anthropic’s report USD cents, typically within five minutes, but leave out Priority Tier costs (October 2026).^[27]
• Cloud billing exports: FOCUS 1.4 has no AI columns, so tokens appear through SKU, ConsumedUnit and ConsumedQuantity.^[1]
• Everything else: SaaS invoices, gateway logs or OpenTelemetry GenAI spans,^[28] GPU telemetry, outcome events.

Trust boundary: prompts can contain personal data, and the cost pipeline needs only counts and metadata; agree retention with PrivacyOps.

%%{init: {"flowchart": {"wrappingWidth": 120, "nodeSpacing": 16, "rankSpacing": 40}}}%%
flowchart TB
  B["Provider invoices and cost APIs"] --> N["Normalise usage conventions; price with a versioned rate card"]
  F["Cloud billing exports in FOCUS"] --> N
  L["Gateway logs and GenAI spans"] --> N
  G["GPU telemetry and cluster allocation"] --> N
  O["Outcome events from business systems"] --> N
  N --> R{{"Reconciles to invoices?"}}
  R -->|yes| A["Attribute to team, feature and use case; idle and unallocated lines"]
  R -->|no| X["Investigate the gap: usage conventions, unpriced lines, price changes"]
  X -->|"fix parser or rate card"| N
  A --> U["Unit cost, forecasts, anomalies, utilisation"]
  U --> D["Weekly reviews and the AI investment council"]
  D --> C["Controls: gateway budgets and caps, routing, provider spend limits, capacity purchases"]
  classDef core fill:#e8f3f2,stroke:#16756f,color:#122029
  classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029
  classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029
  classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029
  class N,A,U core
  class R,D decision
  class X risk
  class B,F,L,G,O,C neutral

Happy path. The product owner records the outcome unit, quality floor and volume; engineering and FinOps estimate cost per outcome for two or three models; the platform issues a key or profile per feature with metadata and a budget; the gateway logs every call and the product emits outcome events; the pipeline joins them daily and reconciles monthly; the weekly review acts on the largest lever; model and price changes trigger a re-decision.

Exception path. At 80% of a period budget the gateway alerts the product owner; at 100% it applies the feature’s agreed action (degrade to a smaller model, queue in a batch tier, or reject with a clear error). The owner explains or fixes the cause, and the breach is recorded; the runbook follows this path.

Provisioned capacity buys predictable latency. In the Foundation’s words it “can sometimes cost more per token than shared tiers”, for bursty traffic “may be a trap”, and without latency requirements “can solely be used to reduce costs”, bought like a savings plan to a coverage target.^[23]

1. Prove steady demand: four or more weeks of hourly tokens by type (Bedrock’s Reserved tier also counts cache writes).^[21]
2. Normalise the price and buy unit by unit: break-even utilisation = hourly unit price ÷ (tokens per hour per unit × pay-as-you-go price per token). Buy a unit only if its own utilisation, the traffic it would serve beyond the units already bought, clears break-even (Lab 3).
3. Load-test your mix: throughput per PTU varies by model,^[14] and a vendor’s figure “is best treated as a rough estimate”.^[23]
4. Match the term to the model’s life (October 2026): Bedrock Provisioned Throughput runs without commitment, one or six months (customised models need it unless base model and Region allow on-demand deployment: some Nova and Llama models in two US Regions);^[20] Bedrock’s Reserved tier one or three months;^[21] Azure reservations one month or one year, with the advice to “always create deployments first, then purchase the Azure Reservation”;^[14] Google sells a “fixed-cost, fixed-term subscription” with weekly and monthly prices.^[29] AWS reservations are model-specific, the Foundation reports;^[23] Azure’s are “a financial discount applied to PTU billing meters”, covering any supported model in scope.^[14]
5. Plan the overflow: Azure spillover^[22] and Bedrock’s Reserved tier, which “automatically overflows to the Standard tier”,^[21] send excess to pay-as-you-go; elsewhere build it into the gateway. Forecast spilled tokens.
6. Report idle and unallocated capacity; separate preparer and approver; file the decision record. Do not resize production with traffic: on Azure, scaling down “releases capacity permanently”.^[14]

OpenCost defines workload cost for GPUs as “max(request, usage)”.^[30] Kubernetes requests GPUs through limits (requests, if set, must be equal),^[31] so a pod holding a GPU pays for all of it, unless MIG slices (hardware isolation) or time-slicing (“no memory or fault-isolation between replicas”) divide it.^[32] That is the device-plugin path; with Dynamic Resource Allocation, stable since Kubernetes v1.35, Pods can share a device through one claim, so allocate by claim (KubernetesOps).^[31]

Multi-agent workloads have no accepted allocation framework;^[1] this Atlas proposes charging every call in a run to the feature that started it, via a propagated root request ID (Lab 1).

Usage conventions differ. OpenTelemetry’s gen_ai.usage.input_tokens “SHOULD include all types of input tokens, including cached tokens”, with reasoning inside output;^[28] OpenAI matches.^[19] Anthropic’s input_tokens counts only tokens after the last cache breakpoint.^[34] One parser for both misprices (Lab 1). Reasoning differs too: OpenAI and Anthropic count it inside output (OpenAI’s example: 1,186 output tokens, 1,024 of them reasoning); Gemini reports thinking tokens separately and prices “the sum of output tokens and thinking tokens”.^[9][35][36] Draft FOCUS properties ModelDeveloper, ModelFamily, ModelId, ModelVersion, TokenCacheAction and TokenDirection arrive in SkuPriceDetails with “No new columns”;^[7] name your fields to match.

Automate the guardrails (budgets, caps, step limits, spend-rate alerts, re-evaluation after model releases); keep people on decisions that trade cost against quality or commit money.

Four artefacts; all values are placeholders. 1. A request metadata contract: the fields every call must carry, and those it must not.

Example status: YAML syntax-checked; a design artefact, not tied to one product.

# ai-request-metadata.yaml: owner fields every model call must carry (version 1)
version: 1
required:
  team: "Owning team ID from the service catalogue, e.g. support"
  feature: "Product feature, e.g. assistant"
  use_case: "Funded use case ID from the investment register, e.g. UC-0142"
  environment: "prod | staging | dev"
  budget_id: "Budget the call draws from, e.g. support-assistant-2026q4"
  root_request_id: "ID of the user request the call serves; joins calls to outcomes"
optional:
  agent_step: "Step number inside an agent run"
  outcome_event: "Business event that marks success, e.g. case_resolved"
forbidden:
  - prompt_text   # never in the cost dataset; content stays in application logs
  - user_email    # use a pseudonymous user ID if per-user cost is needed
mapping:
  gateway_header_prefix: "x-ai-meta-"   # e.g. x-ai-meta-team: support
  otel_span_attributes:                 # OpenTelemetry GenAI conventions (Development)
    - gen_ai.request.model
    - gen_ai.usage.input_tokens         # includes cached tokens
    - gen_ai.usage.cache_read.input_tokens
    - gen_ai.usage.cache_write.input_tokens
    - gen_ai.usage.output_tokens        # includes reasoning tokens
enforcement: "Gateway rejects calls without required fields; dev environments only warn"

2. Provider-side attribution on Amazon Bedrock. Application inference profiles let you “track usage and costs when invoking a model” and accept tags;^[37] activate the keys as cost allocation tags first (FinOps). Tags see only calls made through the profile, so use the bedrock:InferenceProfileArn IAM condition key to allow the model only through it; for cross-Region routing, copy from a system-defined (cross-Region) profile instead of a foundation model.^[37]

Example status: syntax-checked with bash -n; not executed (needs Bedrock access).

# Create an application inference profile that carries cost allocation tags (AWS CLI v2).
# Placeholders: REGION and MODEL_ID; use a model your organisation has approved.
PROFILE_ARN=$(aws bedrock create-inference-profile \
  --inference-profile-name support-assistant-prod \
  --description "Support assistant, production" \
  --model-source copyFrom="arn:aws:bedrock:REGION::foundation-model/MODEL_ID" \
  --tags key=team,value=support key=feature,value=assistant \
         key=budget_id,value=support-assistant-2026q4 \
  --query inferenceProfileArn --output text)

# Check the profile by its ARN (the identifier is the profile ID or ARN, not its name).
# Applications then invoke the model with this ARN as the model ID.
aws bedrock get-inference-profile --inference-profile-identifier "$PROFILE_ARN"

3. A gateway budget with LiteLLM. The proxy tracks spend per key, user and team, but checks budgets against its database: without one “none of them cap anything” and max_budget “fails open”.^[38] Keys that bypass the gateway are not capped either, so add provider-side spend limits. Also:^[38] the proxy-wide budget covers “all calls on the proxy”, so reaching it would stop every team (our inference; keep it above the team budgets); checks trust a Redis counter that after a restart “can come back lower than the spend already recorded” unless fail_closed_budget_enforcement reads the database (more load; unverifiable requests get a 503); batch jobs are costed only on completion.

Example status: syntax-checked (YAML, bash -n); not run against a proxy (LiteLLM 1.105.1, PyPI 2026-10-14).

# config.yaml for the LiteLLM proxy: central spend tracking and a global ceiling.
model_list:
  - model_name: large-default
    litellm_params:
      model: provider/large-model-id      # placeholder: provider prefix and model ID
      api_key: os.environ/PROVIDER_API_KEY
  - model_name: small-default
    litellm_params:
      model: provider/small-model-id
      api_key: os.environ/PROVIDER_API_KEY
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL   # required: without it, budgets fail open
  fail_closed_budget_enforcement: true    # check spend in the database if Redis is stale
litellm_settings:
  max_budget: 20000        # USD per period for ALL calls; keep above the team budgets
  budget_duration: 30d

# A team with its own monthly budget; keys issued to the team draw from it.
curl -sS -X POST "${LITELLM_URL:?}/team/new" \
  -H "Authorization: Bearer ${LITELLM_MASTER_KEY:?}" \
  -H "Content-Type: application/json" \
  -d '{"team_alias": "support-assistant", "max_budget": 2500, "budget_duration": "30d"}'

4. A versioned rate card. Prices change, so price usage from a dated file, not code constants.

Example status: JSON syntax-checked; values illustrative.

{
  "rate_card_version": "2026-10-17",
  "currency": "USD",
  "unit": "per 1M tokens",
  "source_note": "Illustrative. Copy real prices from provider pages, with URL and date.",
  "models": {
    "large-v1": {"input": 4.0, "output": 20.0, "cache_read": 0.1, "cache_write": 1.25},
    "small-v1": {"input": 0.4, "output": 2.0, "cache_read": 0.1, "cache_write": 1.25}
  },
  "tier_factors": {"standard": 1.0, "batch": 0.5, "flex": 0.5, "priority": 1.75},
  "stacking_rule": "Check per provider: some stack cache and batch discounts, some do not"
}

The tier factors echo published examples (October 2026): OpenAI and Google discount batch by 50%, OpenAI prices flex at batch rates,^[40][29] and Bedrock lists Flex at 50% off and Priority at a “75% premium” for many models.^[41] Anthropic’s multipliers “stack with” the batch discount;^[10] Google’s “don’t stack”.^[29]

Objective. Price calls in two usage conventions, reconcile and explain a gap, find unowned spend, and compare cost per call with cost per successful outcome. Prerequisites. Python 3.8+, standard library; no cost or risk. Data. Fourteen illustrative calls: inclusive rows count cached tokens inside input (OpenTelemetry, OpenAI), exclusive rows report them separately (Anthropic).^[28][19][34] Each call carries the root_request_id of the user request it served (metadata contract); a second file records each request’s outcome. In the support assistant a small model triages each conversation first.

Example status: verified by execution on 2026-10-18 (Python 3.8.10). Data and prices are illustrative.

Save this as calls.csv:

call_id,root_request_id,style,model,tier,team,feature,input,cache_read,cache_write,output
r01,c1,inclusive,small-v1,standard,support,assistant,900,0,0,40
r02,c1,inclusive,large-v1,standard,support,assistant,14200,12000,0,420
r03,c2,inclusive,small-v1,standard,support,assistant,950,0,0,35
r04,c2,inclusive,large-v1,standard,support,assistant,15100,12000,0,610
r05,c2,inclusive,large-v1,standard,support,assistant,16300,15100,0,450
r06,c3,inclusive,small-v1,standard,support,assistant,920,0,0,38
r07,c3,inclusive,large-v1,standard,support,assistant,13900,0,12000,380
r08,c4,inclusive,large-v1,standard,support,assistant,41800,12000,0,2900
r09,e1,exclusive,large-v2,standard,sales,email-drafts,1800,6000,0,900
r10,e2,exclusive,large-v2,standard,sales,email-drafts,2100,0,6000,1100
r11,e3,exclusive,large-v2,standard,sales,email-drafts,1500,6000,0,850
r12,i1,exclusive,large-v2,batch,finance,invoice-extract,52000,0,0,6000
r13,i2,exclusive,large-v2,batch,finance,invoice-extract,48000,0,0,5500
r14,,inclusive,large-v1,standard,,,30000,0,0,8000

Save this as outcomes.csv:

root_request_id,outcome
c1,resolved
c2,resolved
c3,escalated
c4,resolved
e1,accepted
e2,rejected
e3,accepted
i1,accepted
i2,accepted

Save this as lab1_price.py:

"""Lab 1: price, reconcile, attribute and join LLM calls to outcomes (stdlib)."""
import csv
import statistics
import sys
from collections import defaultdict

# Illustrative rate card: USD per 1M tokens; cache multipliers relative to input price.
RATES = {
    "large-v1": {"input": 4.00, "output": 20.00, "read": 0.10, "write": 1.25},
    "small-v1": {"input": 0.40, "output": 2.00, "read": 0.10, "write": 1.25},
    "large-v2": {"input": 3.00, "output": 15.00, "read": 0.10, "write": 1.25},
}
TIER = {"standard": 1.0, "batch": 0.5}                 # e.g. a 50% batch discount
# The provider's cost report for the same period (illustrative).
INVOICED = {"large-v1": 0.6084, "small-v1": 0.00133, "large-v2": 0.35343}
SUCCESS = {"resolved", "accepted"}
INT_FIELDS = ("input", "cache_read", "cache_write", "output")


def split_input(r):
    """(uncached, cache_read, cache_write) input tokens in either provider convention."""
    if r["style"] == "inclusive":   # input already contains the cached tokens
        uncached = r["input"] - r["cache_read"] - r["cache_write"]
    else:                           # input excludes them; they are reported separately
        uncached = r["input"]
    if uncached < 0:
        raise ValueError(f"{r['call_id']}: cached tokens exceed input; wrong style?")
    return uncached, r["cache_read"], r["cache_write"]


def price(r):
    rate = RATES[r["model"]]
    uncached, read, write = split_input(r)
    tokens_usd = (uncached + read * rate["read"] + write * rate["write"]) * rate["input"]
    tokens_usd += r["output"] * rate["output"]
    return tokens_usd / 1e6 * TIER[r["tier"]]


with open("calls.csv", newline="") as f:
    calls = [dict(r, **{k: int(r[k]) for k in INT_FIELDS}) for r in csv.DictReader(f)]
with open("outcomes.csv", newline="") as f:
    outcome = {r["root_request_id"]: r["outcome"] for r in csv.DictReader(f)}
for c in calls:
    c["usd"] = price(c)

# 1. Reconcile computed cost with the provider's cost report, model by model.
by_model = defaultdict(float)
for c in calls:
    by_model[c["model"]] += c["usd"]
for model, usd in sorted(by_model.items()):
    gap = INVOICED[model] - usd
    status = "ok" if abs(gap) < 0.0005 else f"INVESTIGATE: invoice {gap / usd:+.0%}"
    print(f"{model:9s} computed {usd:.4f} | invoiced {INVOICED[model]:.4f} | {status}")

# 2. Attribution: how much has no owner?
total = sum(c["usd"] for c in calls)
unowned = sum(c["usd"] for c in calls if not c["team"])
print(f"Total {total:.4f} USD; unattributed {unowned:.4f} ({unowned / total:.1%})")

# 3. Per call versus per outcome: join every call to the outcome of the request that
#    started it (root_request_id), then divide by successful outcomes.
groups = defaultdict(list)
for c in calls:
    groups[(c["team"] or "UNKNOWN", c["feature"] or "UNKNOWN")].append(c)
print("team     feature          calls  median_call  roots  per_root  wins  per_success")
for (team, feature), cs in sorted(groups.items()):
    usd = sum(c["usd"] for c in cs)
    roots = {c["root_request_id"] for c in cs if c["root_request_id"]}
    wins = sum(outcome.get(root) in SUCCESS for root in roots)
    per_root = f"{usd / len(roots):.4f}" if roots else "n/a"
    per_win = f"{usd / wins:.4f}" if wins else "n/a"
    median = statistics.median(c["usd"] for c in cs)
    print(f"{team:8s} {feature:15s}  {len(cs):5d}  {median:11.4f}  {len(roots):5d}"
          f"  {per_root:>8s}  {wins:4d}  {per_win:>11s}")

# 4. The classic mistake: one parser for both conventions bills cached tokens twice.
inclusive = [c for c in calls if c["style"] == "inclusive"]
wrong = sum(price(dict(c, style="exclusive")) for c in inclusive)
right = sum(c["usd"] for c in inclusive)
print(f"Inclusive rows read as exclusive: {wrong:.4f} instead of {right:.4f} "
      f"({wrong / right - 1:+.0%})")

Steps. Save the three files in an empty folder and run python3 lab1_price.py in it.

Expected output (actual output of the run on 2026-10-18):

large-v1  computed 0.6084 | invoiced 0.6084 | ok
large-v2  computed 0.3213 | invoiced 0.3534 | INVESTIGATE: invoice +10%
small-v1  computed 0.0013 | invoiced 0.0013 | ok
Total 0.9311 USD; unattributed 0.2800 (30.1%)
team     feature          calls  median_call  roots  per_root  wins  per_success
UNKNOWN  UNKNOWN              1       0.2800      0       n/a     0          n/a
finance  invoice-extract      2       0.1181      2    0.1181     2       0.1181
sales    email-drafts         3       0.0207      3    0.0284     2       0.0425
support  assistant            8       0.0209      4    0.0824     3       0.1099
Inclusive rows read as exclusive: 0.8622 instead of 0.6098 (+41%)

Validation. (a) large-v1 and small-v1 reconcile; the large-v2 report is 10% higher because those calls ran with a 1.1× data-residency multiplier (Anthropic charges one for US-only inference on Claude 4.6 and later)^[10] missing from the rate card. (b) The unlabelled call r14 is 30.1% of spend and serves no outcome. (c) The assistant’s median call costs 0.0209, a conversation 0.0824 (two calls on average, triage included; long calls such as r08 lift the mean) and a resolved conversation 0.1099, because the escalated conversation’s cost is spread over the three that succeeded: per call understates the unit cost five times. (d) Reading inclusive rows with the exclusive rule bills cached tokens twice: the log runs 41% above the truth.

Troubleshooting. A ValueError about cached tokens means a wrong style. INVESTIGATE means a missing row, a mid-period price change or an unpriced line: longer-lived cache writes, a regional or long-context multiplier, a separate thinking field. Teardown. Delete the folder.

Objective. See why conversation cost grows faster than turns, and when caching pays. Prerequisites. Python 3.8+. Assumptions. A stateless API resends the history each turn; turns arrive within the cache lifetime; each conversation starts with a cold cache (no prefix shared across users) except in the line after the table; write 1.25× and read 0.1×, as Anthropic and OpenAI publish for most current models (some newest models read at 0.05× or 0.025×, October 2026).^[10][19] Real caches have minimum lengths and can miss.^[42]

Example status: verified by execution on 2026-10-18 (Python 3.8.10). Prices are illustrative.

"""Lab 2: context-window creep and prompt-caching break-even (Python 3.8+, stdlib)."""
S, U, B = 3000, 150, 350          # system prompt, user turn, assistant reply (tokens)
P_IN, P_OUT = 4.00, 20.00         # illustrative USD per 1M input / output tokens
READ, WRITE = 0.10, 1.25          # cache read, 5-minute cache write (x input price)


def conversation(turns, cached, warm_prefix=False):
    """Total cost of one conversation in which every turn resends the full history."""
    usd = 0.0
    for k in range(1, turns + 1):
        prompt = S + (k - 1) * (U + B) + U          # input tokens sent at turn k
        if not cached:
            usd += prompt * P_IN
        elif k == 1 and not warm_prefix:
            usd += prompt * WRITE * P_IN            # cold: turn 1 writes it all
        else:
            # Reused part: the previous turn's prompt, or at turn 1 a system prompt that
            # other users' traffic has already cached.
            reused = S if k == 1 else S + (k - 2) * (U + B) + U
            usd += (reused * READ + (prompt - reused) * WRITE) * P_IN
        usd += B * P_OUT
    return usd / 1e6


def total_input(turns):
    return sum(S + (k - 1) * (U + B) + U for k in range(1, turns + 1))


print("turns  input_tokens  input_share  no_cache_usd  cached_usd  saving")
for t in (1, 5, 10, 20, 40):
    plain, cached = conversation(t, False), conversation(t, True)
    share = total_input(t) * P_IN / 1e6 / plain
    print(f"{t:5d}  {total_input(t):12,d}  {share:11.0%}  {plain:12.4f}  {cached:10.4f}"
          f"  {1 - cached / plain:6.0%}")
warm = conversation(1, True, warm_prefix=True)
print(f"1 turn, system prompt already cached by other users: {warm:.4f}"
      f" ({1 - warm / conversation(1, False):.0%} saving)")

# Closed form: T*S + T*U + (U+B)*T*(T-1)/2 grows with the square of the turns.
t = 40
assert total_input(t) == t * S + t * U + (U + B) * t * (t - 1) // 2

# Break-even: a cached block of n reads costs WRITE + n*READ instead of (1 + n) uncached.
for write in (1.25, 2.0):
    n = 0
    while write + n * READ >= 1 + n:
        n += 1
    print(f"write multiplier {write}: caching pays from read number {n}")

Steps. Save as lab2_context.py and run python3 lab2_context.py.

Expected output (actual output of the run on 2026-10-18):

turns  input_tokens  input_share  no_cache_usd  cached_usd  saving
    1         3,150          64%        0.0196      0.0227    -16%
    5        20,750          70%        0.1180      0.0670     43%
   10        54,000          76%        0.2860      0.1268     56%
   20       158,000          82%        0.7720      0.2614     66%
   40       516,000          88%        2.3440      0.5906     75%
1 turn, system prompt already cached by other users: 0.0089 (54% saving)
write multiplier 1.25: caching pays from read number 1
write multiplier 2.0: caching pays from read number 2

Validation. (a) From 20 to 40 turns, input tokens more than triple: total input T·S + T·U + (U+B)·T(T−1)/2 grows with the square of the turns. The Foundation’s paper says “exponentially”;^[8] quadratic is accurate, and still severe. (b) Output costs five times more per token, yet input is 82% of cost by turn 20. (c) One-turn conversations cost 16% more with caching when cold, but 54% less when other users keep the system prompt cached, as scenario A assumes. (d) The break-even matches Anthropic’s statement: one read for a 1.25× write, two for 2×.^[10]

Troubleshooting. Low real savings usually mean a changing prefix or traffic too sparse for the cache lifetime. Teardown. Delete the file.

Objective. Size provisioned capacity unit by unit, find the break-even, and stress a self-hosting case. Prerequisites. Python 3.8+. Assumptions (illustrative). 9M tokens per hour in weekday business hours, 1.5M at night, 2M at weekends; pay-as-you-go at 6.00 per million; a provisioned unit of 2.5M tokens per hour for 10.00 per hour, with spillover; GPUs of 3M tokens per hour at 4.00 per hour, at least two, plus 1,500 a week of people. Self-hosting assumes an open-weights model meets the quality floor and on-demand GPUs at this price every hour (reserved GPUs are paid all week).

Example status: verified by execution on 2026-10-18 (Python 3.8.10). All prices and throughputs are illustrative.

"""Lab 3: pay-as-you-go, provisioned units with spillover, or self-hosted GPUs?"""

# One week of demand in millions of tokens per hour (illustrative profile).
def demand(hour_of_week):
    day, hour = divmod(hour_of_week, 24)
    if day < 5 and 8 <= hour < 18:
        return 9.0          # weekday business hours
    return 2.0 if day >= 5 else 1.5

HOURS = range(168)
PAYG = 6.00                 # blended USD per 1M tokens on the shared tier (illustrative)
UNIT_CAP, UNIT_USD = 2.5, 10.00   # provisioned unit: capacity (1M tokens/hour), USD/hour
GPU_CAP, GPU_USD = 3.0, 4.00      # self-hosted GPU: capacity at target batch, USD/hour
MIN_GPUS, OPS_WEEK = 2, 1500.0    # availability floor; people USD per week

total = sum(demand(h) for h in HOURS)
print(f"Weekly demand {total:.0f}M tokens; peak {max(map(demand, HOURS)):.1f}M/hour")
print(f"Pay-as-you-go only: {total * PAYG:9,.0f} USD/week = {PAYG:.2f} per 1M")

print("units  served_by_units  utilisation  marginal_util  weekly_usd  usd_per_1M")
best, served_before = None, 0.0
for units in range(0, 5):
    cap = units * UNIT_CAP
    served = sum(min(demand(h), cap) for h in HOURS)
    spill = total - served          # overflow billed on the shared tier
    usd = units * UNIT_USD * 168 + spill * PAYG
    util = served / (cap * 168) if units else 0.0
    marginal = (served - served_before) / (UNIT_CAP * 168) if units else 0.0
    print(f"{units:5d}  {served:15.0f}  {util:11.0%}  {marginal:13.0%}  {usd:10,.0f}"
          f"  {usd / total:10.2f}")
    best = min(best or (usd, units), (usd, units))
    served_before = served
print(f"Cheapest: {best[1]} unit(s) at {best[0]:,.0f} USD/week")

# Break-even utilisation of one provisioned unit against pay-as-you-go:
print(f"A unit pays for itself above {UNIT_USD / (UNIT_CAP * PAYG):.0%} utilisation")

# Self-hosted: GPUs scale per hour between a floor and the peak; people cost is fixed.
print("gpu_capacity  people_usd  gpu_hours  weekly_usd  usd_per_1M")
for gpu_cap in (GPU_CAP, GPU_CAP / 2):          # measured vs. pessimistic throughput
    for ops in (OPS_WEEK, OPS_WEEK * 2):
        gpu_hours = sum(max(MIN_GPUS, -(-demand(h) // gpu_cap)) for h in HOURS)
        usd = gpu_hours * GPU_USD + ops
        print(f"{gpu_cap:12.1f}  {ops:10,.0f}  {gpu_hours:9.0f}  {usd:10,.0f}"
              f"  {usd / total:10.2f}")

Steps. Save as lab3_capacity.py and run python3 lab3_capacity.py.

Expected output (actual output of the run on 2026-10-18):

Weekly demand 651M tokens; peak 9.0M/hour
Pay-as-you-go only:     3,906 USD/week = 6.00 per 1M
units  served_by_units  utilisation  marginal_util  weekly_usd  usd_per_1M
    0                0           0%             0%       3,906        6.00
    1              326          78%            78%       3,630        5.58
    2              451          54%            30%       4,560        7.00
    3              576          46%            30%       5,490        8.43
    4              651          39%            18%       6,720       10.32
Cheapest: 1 unit(s) at 3,630 USD/week
A unit pays for itself above 67% utilisation
gpu_capacity  people_usd  gpu_hours  weekly_usd  usd_per_1M
         3.0       1,500        386       3,044        4.68
         3.0       3,000        386       4,544        6.98
         1.5       1,500        536       3,644        5.60
         1.5       3,000        536       5,144        7.90

Validation. (a) The first unit runs at 78%, above the 67% break-even, and saves 7%; a second would serve only 30% of its capacity, so stop at one. (b) Four units, sized to the peak, cost 72% more than pay-as-you-go. (c) Break-even is 10.00 ÷ (2.5 × 6.00) = 67%. (d) Self-hosting wins only if throughput and people cost hold; halve one or double the other and it loses. Throughput depends on batching and the serving stack (vLLM’s authors reported 2–4 times earlier systems’ throughput),^[43] so measure it on your traffic.

Troubleshooting. If many units look best, check the profile has quiet hours. Teardown. Delete the file.

All three are illustrative; the facts they use are sourced.

A. Resent history and the policy prompt dominate (Lab 2), then escalations.

B. Small models waste a whole GPU unless partitioned (allocation). Judge efficiency by DCGM SM activity, the share of time multiprocessors had work (0.8 is “necessary, but not sufficient”), not by nvidia-smi, which counts time “during which one or more kernels was executing”.^[39]

C. Try prompts first, the Foundation advises; for tuning it asks about data preparation (“often underestimated and sometimes exceeding the compute cost”), retraining frequency and serving: serving your own tuned model means “taking on the operational costs of inference infrastructure permanently”.^[24] Flow: price the prompt tokens tuning would remove; add training (the 6ND estimate, about 200 for an illustrative 8B run) per retrain, plus data preparation; then price serving. On Bedrock a customised model may need Provisioned Throughput (playbook);^[20] one unit at Lab 3’s illustrative 10.00 an hour costs 1,680 a week, about eight times that training run. Pitfall: approving the cheap training line and missing the permanent serving line.

The Foundation lists cost per inference, cost per token, utilisation, anomaly cost, ROI and time to value.^[1] Manage by the first row below.

Worked example (illustrative, from the judgement call). Per 100 conversations, (9.13 tokens + 73.00 escalations) ÷ 85.4 resolved = 0.96 per resolved conversation. Tokens are 11% of it: a 20% token cut saves about 2%; one more escalation costs 5.00, almost three times as much.

Leading: metadata coverage, cache-read share, p95 tokens per request, step-limit hits. Lagging: cost per successful outcome, forecast accuracy, value ratio. Vanity trap: “tokens saved” without the quality effect. Data limits: provider cost data arrives in buckets,^[26][27] so real-time control needs gateway logs.

Regulatory duties belong to ModelOps and ComplianceOps; their cost belongs to the use case.

Training cost, to an order of magnitude. Training needs about 6 × parameters × tokens FLOPs: Kaplan and colleagues write C = 6NBS, where “the factor of 6 accounts for the forward and backward passes”.^[44] Divide by peak FLOPs times model FLOPs utilisation (MFU), the observed share of peak throughput; PaLM reported 46.2%,^[45] Meta 38–43% for Llama 3 405B, trained on 15.6T tokens with 3.8×10²⁵ FLOPs.^[46]

Example status: verified by execution on 2026-10-18 (Python 3.8.10); the price per GPU-hour is illustrative.

"""Order-of-magnitude training cost: FLOPs ~ 6 x params x tokens (Kaplan et al., 2020)."""

def train_cost(params, tokens, peak_flops, mfu, usd_per_gpu_hour):
    flops = 6 * params * tokens
    gpu_hours = flops / (peak_flops * mfu) / 3600
    return flops, gpu_hours, gpu_hours * usd_per_gpu_hour

# Check against a published run: Llama 3 405B, 15.6T tokens, reported 3.8e25 FLOPs.
print(f"Llama 3 405B: {6 * 405e9 * 15.6e12:.2e} FLOPs by the 6ND rule")

# Illustrative full fine-tune: 8B params, 2B tokens, H100 SXM at 40% MFU, 3 USD/hour.
DENSE_BF16 = 989e12          # H100 SXM dense BF16 peak; 1,979 TFLOPS assumes sparsity
for label, peak in (("dense peak", DENSE_BF16), ("sparse peak (wrong)", 2 * DENSE_BF16)):
    flops, hours, usd = train_cost(8e9, 2e9, peak, 0.40, 3.00)
    print(f"{label:20s} {flops:.2e} FLOPs  {hours:6.1f} GPU-hours  {usd:6.0f} USD")

Llama 3 405B: 3.79e+25 FLOPs by the 6ND rule
dense peak           9.60e+19 FLOPs    67.4 GPU-hours     202 USD
sparse peak (wrong)  9.60e+19 FLOPs    33.7 GPU-hours     101 USD

The rule reproduces Meta’s figure. The second line is a common error: NVIDIA lists 1,979 BF16 teraFLOPS for the H100 SXM “With sparsity”, and its architecture post pairs each dense figure with a sparse one twice as large;^[47] using it halves the estimate. The formula omits evaluations, restarts (Meta recorded 466 job interruptions in 54 days, 419 unexpected),^[46] data preparation and people.

Capacity. GPU capacity is often bought ahead: AWS Capacity Blocks reserve GPU instances up to eight weeks ahead, and SageMaker AI Savings Plans are their own commitment type.^[48] Interruptible capacity suits training only with checkpointing (MLOps). In serving, batching sets throughput per GPU, so optimising only for single-request latency can raise cost.^[24][43]

Retrieval and multimodal inputs. Retrieval-augmented generation (RAG) adds retrieved text to every call: input grows with chunks per query × chunk size. Retrieving snippets “dramatically reduces” tokens compared with whole documents, but the Foundation calls RAG infrastructure costs “frequently underestimated in initial proposals”: embeddings for every document, “re-done when documents change” or the embedding model is upgraded, “vector database storage and query costs”, and pipeline upkeep.^[24] Give the vector store its own idle line, price re-indexing as a recurring cost driven by document churn, and count retrieved tokens per outcome. An image in the history is “silently re-billed” on every later turn.^[8]

Sustainability. Energy is layer zero of the Tokenomics Foundation’s seven-layer stack, whose “lower three layers set what AI costs”.^[33] Carbon accounting belongs to GreenOps; idle reserved capacity wastes energy as well as money.

Monthly: request-log cost reconciles with every provider report (Lab 1), and attribution covers every channel. Quarterly: a test key pushed past its budget and an agent past its step limit both stop, and last quarter’s capacity and funding decisions are checked against results.

A research agent went live on Thursday. At 02:10 on Saturday the gateway alerts: spend at 12 times baseline, 70% of the month’s budget gone, nobody using the product.

%%{init: {"flowchart": {"wrappingWidth": 180, "nodeSpacing": 26, "rankSpacing": 32}}}%%
flowchart TB
  A["Alert: feature spend rate 12 times baseline"] --> B["Triage: which key, agent run and model; calls per request"]
  B --> C{{"Is the spend serving real users?"}}
  C -->|no| D["Contain: set the feature budget to zero; revoke or rotate the feature's key"]
  C -->|yes| E["Throttle and page the product owner"]
  D --> F["Verify in gateway logs that spend stopped or fell to the throttled rate"]
  E --> F
  F --> G["Root cause: no step limit; a failing tool triggered retries with full context"]
  G --> H["Fix: step, retry and output caps; circuit breaker; test the limit"]
  H --> I["Review: incident cost, re-forecast, add a control test"]
  classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029
  classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029
  classDef core fill:#e8f3f2,stroke:#16756f,color:#122029
  class A,G risk
  class C decision
  class B,D,E,F,H,I core

1. Triage (10 minutes): one run made 3,000 calls, each resending 40,000 tokens: the T(∞) class.^[11]
2. Contain (5 minutes): a provider-side spend limit is the backstop if the gateway fails.
3. Verify in gateway logs; provider cost data arrives in buckets.^[27][26]
4. Root cause: OWASP’s “unbounded consumption”.^[25]
5. Fix and review: back-off on retries; drive a test agent into the limit; add it to quarterly control tests.

• One API key for the company: cheap to start, impossible to allocate later.
• Comparing price per token across models: tokenizers, reasoning and output length differ;^[10][9] compare cost per successful outcome.
• Cost targets without quality floors, and pilots without caps or end dates.
• Half-truth: AI is “simply another bucket of spend”. A respondent quoted in the State of FinOps 2026 (labelled Financial Services) wrote that AI “is simply another bucket of spend that requires the same discipline and governance as any other technology”; the report concludes that “AI is not rewriting FinOps, but it is testing how flexible the operational framework can become”.^[15] The discipline carries over; the unit economics do not.

Build the practice when AI spend is material or growing fast, several teams or providers are involved, capacity commitments are on the table, agents run in production, or leadership asks whether AI pays. Do not build a separate practice for small AI spend with one provider: add AI lines and caps to existing FinOps reviews (not all AI spend needs its own Scope, the Foundation notes).^[1] Hidden costs: metadata discipline, evaluation sets, review time.

How a workload consumes models decides most of its cost. The tree is this Atlas’s proposal; thresholds come from Lab 3.

%%{init: {"flowchart": {"wrappingWidth": 160, "nodeSpacing": 24, "rankSpacing": 34}}}%%
flowchart TB
  Q0{{"Does the task need a model call at all?"}} -->|no| R0["Rules, search or a cached answer"]
  Q0 -->|yes| Q1{{"Is an answer needed within seconds?"}}
  Q1 -->|no| R1["Batch or flex tier"]
  Q1 -->|yes| Q2{{"Steady demand, and the model (or a model-independent pool) kept for the term?"}}
  Q2 -->|no| R2["Pay-as-you-go with caching, caps and routing"]
  Q2 -->|yes| Q3{{"Next unit above break-even utilisation, or a latency guarantee worth its premium?"}}
  Q3 -->|no| R2
  Q3 -->|yes| Q4{{"An open-weights model meets the quality floor and you can run GPUs?"}}
  Q4 -->|no| R3["Provisioned units, each above break-even or worth its latency premium; bursts spill over"]
  Q4 -->|yes| R4["Compare self-hosting with provisioned at measured throughput"]
  classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029
  classDef core fill:#e8f3f2,stroke:#16756f,color:#122029
  class Q0,Q1,Q2,Q3,Q4 decision
  class R0,R1,R2,R3,R4 core

Self-hosting needs all of these (Atlas checklist): utilisation above the managed break-even; an open-weights model that passes your evaluations; a team already running GPUs; and one of the Foundation’s reasons to own a model: privacy, a specialised domain or “very high inference volumes”.^[24] The tree puts self-hosting behind the managed break-even on purpose (Atlas reasoning): it adds fixed people cost, a GPU floor and throughput risk (Lab 3).

This Atlas’s proposal; the Foundation assesses maturity per capability, not for AI.

Quick wins: batch tiers, stable prefixes, output and step caps, closed unowned keys.

Quick start

• Every AI channel listed: cloud AI services, direct APIs, marketplaces, SaaS credits, GPUs
• Every key has an owner, a feature and a budget; unowned keys have a removal date
• Every pilot has a time box, a cost cap and a value hypothesis

Launch readiness for an AI feature

• Outcome event emitted; quality floor and evaluation set agreed; cost per outcome estimated
• Output, step, retry and input-size limits configured and tested
• Owner metadata on every call; budget alert at 80% and a defined action at 100%
• Stable, cacheable prompt prefix; non-urgent paths on batch or flex tiers

Capacity purchase preflight (playbook)

• Each unit above break-even on four weeks of hourly data; load test with the real token mix
• Term within the model’s life; overflow configured; Azure deployment before reservation; preparer is not approver

AI use-case funding request (Atlas template)

Use case ID / name:          UC-____ / ______________________
Owner (product) / engineer:  ______________ / ______________
Outcome unit:                e.g. "support case resolved without a person"
Value per outcome:           ____ (method and source: ______________________)
Baseline:                    cost per outcome of the current process: ____
Quality floor:               metric ____ >= ____ on evaluation set ____
Candidate models and tiers:  1. ________  2. ________  (approved by ModelOps: Y/N)
Estimated cost per outcome:  model 1: ____  model 2: ____  (tokens/call x calls/outcome)
Expected volume:             ____ outcomes per month; growth assumption ____
Stage requested:             proof of concept / proof of value / scale
Cost cap and time box:       ____ per month until ____ (YYYY-MM-DD)
Kill criteria:               stop if ______________________________________
Controls:                    output cap __ / step limit __ / budget __ / alert at 80%
Data classification:         ____ ; residency needs: ____ ; prompts logged: Y/N
Decision:                    fund / fund with changes / decline    Date: __________

Provisioned capacity decision record

Decision ID / date:          CAP-____ / YYYY-MM-DD
Provider, model, region:     ______________________________________________
Unit and quantity:           ____ units (PTU / model units / reserved TPM)
Term and renewal:            ____ months; auto-renew: Y/N; end date: ________
Demand evidence:             weeks of data __; floor __ tokens/h; peak __ tokens/h
Load test result:            __ tokens/h per unit at the real input/output mix
Normalised price:            __ per 1M tokens at __% expected utilisation
Last unit's own utilisation: __% (buy only if above break-even)
Pay-as-you-go comparison:    __ per 1M tokens; break-even utilisation __%
Overflow plan:               spillover / overflow tier / gateway fallback
Model-change risk:           model can / cannot be changed during the term
Waste lines reported:        idle allocated / unallocated
Prepared by / approved by:   ______________ / ______________
Review date and result:      ________ ; utilisation achieved __%

AI cost forecast (Atlas template). It applies the Foundation’s “quantity * rate = cost” approach per driver; the Foundation’s outcome is forecasts “within 5% of actuals per month”.^[18] Use mean, not median, tokens and take the range from low and high scenarios.

Feature / use case: ____________   Period: ______   Rate card version: ______
Driver                            Low     Expected  High    Source
Successful outcomes               _____   _____     _____   product plan
Requests per successful outcome   _____   _____     _____   logs, last 4 weeks
Mean tokens per request, by type  _____   _____     _____   logs (mean, not median)
Price per 1M tokens, by type      _____   _____     _____   rate card; announced changes
= Token cost (product / 1M)       _____   _____     _____
+ Provisioned units x hours       _____   _____     _____   capacity decision records
+ Retrieval, tools, vector store  _____   _____     _____   invoices; re-index plan
+ People (escalations x cost)     _____   _____     _____   escalation rate
= Forecast                        _____   _____     _____
Re-forecast when: a driver moves more than __% in a week; a new model, price or tier
Last period's variance: volume __  tokens per outcome __  price __  capacity __

Monthly AI cost review agenda (45 minutes)

1. Reconciliation and coverage (5 min): gaps to invoices; unowned spend and owners
2. Unit costs (15 min): cost per successful outcome by feature; p95 outliers; quality
3. Capacity (10 min): utilisation, spillover, idle and unallocated; expiries in 60 days
4. Forecast (10 min): variance by volume, tokens per outcome and price; re-forecast
5. Decisions and owners (5 min): actions with dates; items for the council

Consumption options compared (October 2026):

Do we need a separate FinOps-for-AI team? Usually not: survey respondents run lean central teams with a wider scope.^[15] Add AI skills and an investment council; dedicate an analyst once AI is a top line.

Should we charge AI costs back? Show back by feature for a quarter first; charge back only when attribution is reliable and the rules for shared and idle capacity are accepted (FinOps).

Why does our token log not match the invoice? A log above the invoice usually counts cached or reasoning tokens twice; an invoice above it has unpriced lines or a price change (failures, Lab 1).^[28][34][36]

Will provisioned capacity give us a discount? Only for units above break-even utilisation and, where reservations are model-specific (Bedrock), only while you keep the model (playbook).

How do we treat AI inside SaaS tools? As SaaS spend with an AI meter: budgets, burn by model and value per active user. Since 1 June 2026 every GitHub Copilot plan includes “GitHub AI Credits”, with budgets at enterprise, cost-centre and user level and a choice to cap spend or allow more usage at published rates;^[13] the licence is no longer the whole cost.

• Billing data for AI. FOCUS draft model and token properties (reasoning tokens still an open request)^[7] are work the Tokenomics Foundation says it will help fund.^[4]
• Price direction. Epoch AI found the price of a fixed benchmark level fell 9 to 900 times a year (median 50) up to early 2025, excluding reasoning models, which generate far more tokens.^[50] The Linux Foundation asserted in June 2026, without data, that per-token costs “have leveled off” and “new model token prices are rising”.^[4] Both can hold: a fixed capability gets cheaper while the newest costs more, so forecast them separately.
• Routing research. FrugalGPT reported “up to 98% cost reduction” and RouteLLM cuts of “over 2 times in certain cases”, on the authors’ 2023–2024 benchmarks.^[16][51] In the Tokenomics Foundation’s release-candidate survey, 86% were evaluating or using a router.^[52] Treat published savings as hypotheses to test.

Routes by role. Beginners: the first part, Lab 1. Engineers: labs, reference implementation, failures. FinOps and finance: operating model, metrics, capacity playbook, templates. Leaders: problem, decision tree, judgement call.

• Decide by cost per successful outcome, people included.
• Bound the multipliers (calls, agent depth, retrieved context, output, retries) before negotiating rates.
• Tiers before commitments: batch and flex, then caching, then provisioned units that each clear break-even.
• Test every control’s preconditions: gateway databases, direct keys, provider limits.
• Re-evaluate on change: a new model, price or tokenizer is a cost event. Fund in stages with kill criteria.

Situation (illustrative). A support assistant runs four-turn conversations on a large model: a 12,000-token cached prefix, 1,500 new input and 400 output tokens per turn, earlier turns resent from cache. About 70% of conversations are “easy”; a small model costs a tenth as much per token. An escalation to a person costs 5.00; the large model escalates 8% of easy and 30% of hard conversations, and a pilot suggests 11% for the small model on easy ones. Option A, route: token cost falls by over 60%. Option B, keep one model: simpler, quality kept.

Example status: verified by execution on 2026-10-18 (Python 3.8.10). All numbers are illustrative.

"""Judgement call: route easy conversations to a small model, or keep one large model?"""
TURNS, PREFIX, NEW_IN, OUT = 4, 12_000, 1_500, 400     # tokens per turn
MODELS = {"large": (4.00, 20.00), "small": (0.40, 2.00)}  # USD per 1M in/out
READ, WRITE = 0.10, 1.25                                # cache read and write multipliers
ROUTER_USD, HUMAN_USD = 0.0004, 5.00                    # per conversation (illustrative)
EASY, HARD_FAIL = 0.70, 0.30                            # easy share; hard escalation rate


def ai_usd(model):
    """One conversation; each turn resends the history, read from cache after turn 1."""
    p_in, p_out = MODELS[model]
    usd = 0.0
    for k in range(1, TURNS + 1):
        history = (k - 1) * (NEW_IN + OUT)              # earlier turns, resent
        cached = PREFIX + max(0, history - OUT)         # prefix (warm) + earlier prompts
        written = PREFIX + history + NEW_IN - cached    # last reply + new input
        usd += (cached * READ + written * WRITE) * p_in + OUT * p_out
    return usd / 1e6


def per_100(route, easy_fail_large=0.08, easy_fail_small=0.11):
    easy_model = "small" if route else "large"
    ai = 100 * (EASY * ai_usd(easy_model) + (1 - EASY) * ai_usd("large"))
    ai += 100 * ROUTER_USD if route else 0.0
    easy_fail = easy_fail_small if route else easy_fail_large
    escalations = 100 * (EASY * easy_fail + (1 - EASY) * HARD_FAIL)
    return ai, escalations * HUMAN_USD


for label, route in (("one large model", False), ("route easy to small", True)):
    ai, people = per_100(route)
    print(f"{label:20s} AI {ai:5.2f}  escalations {people:6.2f}"
          f"  total {ai + people:6.2f}")

saving = per_100(False)[0] - per_100(True)[0]
points = 100 * saving / HUMAN_USD / (100 * EASY)
print(f"AI saving per 100 conversations: {saving:.2f} USD")
print(f"Routing wins only if easy-case escalations rise by less than {points:.1f} points")

one large model      AI  9.13  escalations  73.00  total  82.13
route easy to small  AI  3.42  escalations  83.50  total  86.92
AI saving per 100 conversations: 5.71 USD
Routing wins only if easy-case escalations rise by less than 1.6 points

Decision: not yet. Routing cuts token cost by 63% but raises total cost by 6%: three extra escalations per 100 easy conversations cost more than all the tokens saved. It wins only if the small model’s easy-case escalation rate is within about 1.6 points of the large model’s, so the next step is a better evaluation (a larger sample, a sharper “easy”, a tuned prompt), not a launch.

Uncertainty. The 3-point gap comes from a small pilot whose confidence interval may include 1.6. Cheaper human handling or a better router favour routing; low volume does not: routing “saves money until it breaks your cache”.^[6] Why the attractive option misleads: it optimises the visible line (tokens, 11% of cost) at the expense of the dominant one (people).

You should now be able to explain FinOps for AI against FinOps, LLMOps and tokenomics; calculate call and outcome costs; design metadata, budgets and caps; decide how a workload consumes models; and evaluate savings against quality.

AI did not change the economics of FinOps: price times quantity still makes the bill. What changed is where the quantity comes from: design choices that multiply quietly (resent history, retrieved context, agent calls, hidden reasoning, idle reservations), made by engineers and product owners in an afternoon. If you remember one thing: measure the cost of each successful outcome, people included, and bound the multipliers before you buy capacity.

FinOps Foundation material (CC BY 4.0) is quoted briefly with attribution. Evidence ledger: research/finops-for-ai-ledger.md. Re-check quarterly and on provider releases: prices, tier factors, cache and residency multipliers, tokenizers, capacity terms, cost-API scope, LiteLLM budgets, the FOCUS draft.

• 2026-10-17: first draft (managing-author pass); term status confirmed as emerging; Labs 1–3, the training estimate and the judgement-call model executed with Python 3.8.10.
• 2026-10-17: r1 revision: Bedrock custom models, reasoning-token conventions, marginal capacity rule, retrieval costs; Lab 1 rebuilt around outcomes; Labs 2–3 and the judgement call re-run.
• 2026-10-18, post-score polish: Anthropic multipliers scoped by model; forecast template; scenario C expanded, D moved to the FAQ; runbook and self-hosting gate clarified; re-check note; code wrapped.