ModelOps

ModelOps is the governance and lifecycle management of every model an organisation relies on: machine-learning models, scoring rules, optimisation models, applications built on large language models (LLMs) and agents, built or bought. For each model it answers: what is it for, how risky is it, who allowed its use, and is it still fit for that use? The working parts: an inventory, a tiering rubric, independent validation where it matters, approval with conditions, review and retirement.

Gartner defines ModelOps as “focused primarily on the governance and life cycle management of a wide range of operationalized artificial intelligence (AI) and decision models, including machine learning, knowledge graphs, rules, optimization, linguistic and agent-based models”.^[1] Banks run the governance core under an older name, model risk management (MRM), which US supervisors set out in supervisory guidance in 2011 and replaced in April 2026.^[2][3]

Organisations accumulate models faster than they track them: credit scores, fraud rules, vendor anti-money-laundering scenarios, an LLM support assistant, a refund agent. Nobody can list them all or say who approved each use. US guidance warns that “even a fundamentally sound model producing accurate outputs consistent with the model’s design objective can exhibit high model risk if it is misapplied or misused”.^[8] Engineering quality does not remove that risk; deciding where a model may be used, and checking, does.

• Not MLOps with a new name. This Atlas’s editorial distinction: MLOps engineers the lifecycle (pipelines, registry, drift detection); ModelOps governs the portfolio (which uses are allowed, at what risk, on whose authority). Gartner’s own “core capabilities” include engineering items such as CI/CD integration and rollback,^[1] so no definition enforces this.
• Not a committee that approves everything, nor legal advice: audit evidence belongs to ComplianceOps, and interpreting the law to lawyers.
• Not only for banks: hospitals, airlines and home buyers run consequential models (use cases).
• “Model” means different things. US guidance excludes “deterministic rule-based processes and software where there are no statistical, economic, or financial theories underpinning their design or use” and leaves generative and agentic AI out of scope;^[8] the UK PRA asks banks to consider applying MRM to material, complex rules;^[9] Canada’s OSFI includes AI/ML methods.^[10] Your governance scope can be wider than any regulator’s; record both.

The practice is older than the name: US guidance SR 11-7 (2011) asked banks for “effective challenge” of models.^[2] The label appeared in IBM’s 2019 paper with an engineering meaning,^[4] alongside model cards and datasheets;^[11][12] since 2023 AI regulation has carried governance beyond banking (methods and regulation). The date of Gartner’s first definition is unknown (cited snapshot: May 2025).^[1]

Mental model: a model holds a licence, not a launch date. The licence names a holder (owner), a scope (approved uses), conditions, an expiry and an issuer who did not build the model. ModelOps issues, renews, restricts and revokes licences.

Also: FinCrimeOps runs AML scenarios governed here; ProcurementOps negotiates vendor terms; AIOps is AI for IT operations.

flowchart TB
  ML["MLOps: ML pipelines"]
  LL["LLMOps: LLM apps"]
  DE["DecisionOps: decision logic"]
  J1(["`**Build and run**
  send candidates, results
  receive approvals, conditions`"])
  MO["ModelOps: inventory, tier, validate, approve, review, retire"]
  J2(["`**Set limits and prove**
  send appetite, obligations
  receive inventory, findings`"])
  RI["RiskOps: appetite"]
  PR["PrivacyOps: lawful data"]
  CO["ComplianceOps: evidence"]
  ML & LL & DE <--> J1
  J1 <--> MO
  MO <--> J2
  J2 <--> RI & PR & CO
  classDef core fill:#e8f3f2,stroke:#16756f,color:#122029
  classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029
  classDef flow fill:#ffffff,stroke:#5b7a87,stroke-dasharray:4 3,color:#122029
  class MO core
  class ML,LL,DE,RI,PR,CO neutral
  class J1,J2 flow

No official ModelOps manifesto exists; this charter is this Atlas’s proposal (one-page version). Mission: every model we rely on is known, owned, used within approved limits, and retired on purpose.

1. Inventory before policy. You cannot govern what you have not listed.
2. Rigour follows risk. US, UK and Canadian supervisors all ask for scrutiny proportional to a model’s risk.^[8][9][10]
3. Independence where it matters. Validators “should therefore not be part of any model development activities, nor have a stake in whether or not a model is approved” (PRA).^[9] Tier 3 may need only a peer review.
4. Approve a use, not a model. “Using a model beyond its intended purpose introduces additional uncertainty and risk.”^[8] Approvals name uses, limits and an expiry date.
5. Evidence lives with the model, in records that release automation reads and developers cannot edit.
6. Bought is not exempt (playbook).
7. Retire on purpose, with notice and retained records.

Non-goals: approving every spreadsheet; replacing developers’ testing; legal advice.

The minimum viable set: an inventory with OSFI’s minimum fields (ID, name and use, risk rating, owner, developer, origin);^[10] a one-page tiering rubric; a validation checklist per tier and model type; approval records with uses, conditions and expiry (templates); change classes; review dates fed by MLOps or LLMOps monitoring; an exceptions register; and model cards with intended use, evaluation by group and limitations.^[11]

OSFI names the core roles: the owner coordinates development, deployment and monitoring; the developer designs the method; the reviewer assesses “conceptual soundness, inputs, methodology, assumptions, and performance”; the approver has “the authority to approve the use of a model”; the user relies on outputs.^[10] Large firms add a model risk (MRM) function;^[9] the ML platform team implements gates.

R = responsible, A = accountable, C = consulted, I = informed (an Atlas template). One person may hold the MRM and approver roles, never the developer and validator roles for a tier-1 model. Consultants can validate; US guidance calls for “maintaining proper oversight and integrating that work”.^[8]

OSFI’s lifecycle (design, review, deployment, monitoring, decommission) is “not necessarily sequential”;^[10] release automation needs explicit states in the inventory.

stateDiagram-v2
  %% transitions listed in the order that lays the diagram out without crossing edges
  [*] --> Proposed
  Proposed --> InValidation: tiered
  Suspended --> InValidation: redeveloped
  Approved --> UnderReview: due
  Proposed --> Rejected
  InValidation --> Provisional: urgent need
  Restricted --> Approved: fixed
  InValidation --> Rejected: unacceptable
  Approved --> Restricted: breach
  Approved --> Retired: replaced
  InValidation --> Approved: accepted
  Provisional --> Approved: validated
  Suspended --> Retired
  Restricted --> Suspended: not fixed
  Provisional --> Suspended: expired
  UnderReview --> Approved: fit
  UnderReview --> Restricted: narrowed
  Retired --> [*]

Provisional exists because US guidance accepts urgent use before validation completes, with controls (see the judgement call).^[8] Retired is not a deletion: OSFI asks for retained models and documents, stakeholder alerts and monitoring of “downstream effects”.^[10]

• Intake triage (weekly): a temporary tier within days, as OSFI allows (“a temporary risk rating that is confirmed in the model review or model approval stage”).^[10]
• Model risk committee (monthly): tier-1 approvals, exceptions, retirements.
• Portfolio report (quarterly): tiers, overdue reviews, exceptions, findings and aggregate risk from “reliance on common assumptions, data, or methodologies”.^[8]
• Annual self-assessment, which the PRA expects banks to update “at least annually”.^[9]

The architecture is a control plane around existing delivery systems: it records, decides and enforces; pipelines, registries and monitoring stay with MLOps and LLMOps.

flowchart TB
  subgraph SRC["Discovery sources"]
    direction LR
    REG["Model registries"]
    CLD["Cloud AI and API usage"]
    PRO["Procurement and vendors"]
  end
  INT["Intake form"] --> INV[("Inventory: owner, tier, uses, dates")]
  SRC -->|"reconcile weekly"| INV
  INV --> VAL["Validation workspace: reports, findings"]
  VAL --> APR["Approval record: uses, conditions, expiry"]
  APR --> GATE{"Release gate checks tier and approval record"}
  GATE -->|"pass"| PROD["Production serving"]
  GATE -->|"fail"| BLK["Blocked, owner notified"]
  PROD -->|"monitoring results"| TRG["Review triggers"]
  TRG --> INV
  INV --> RPT["Portfolio reports to committee"]
  classDef core fill:#e8f3f2,stroke:#16756f,color:#122029
  classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029
  classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029
  classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029
  class INT,INV,VAL,APR,TRG,RPT core
  class GATE decision
  class BLK risk
  class REG,CLD,PRO,PROD neutral

Trust boundaries. Whoever can move a production alias, or write the evidence a gate reads, can bypass every approval. In MLflow, creating a version, setting a version tag and setting an alias all need can_update on a registered model (from 3.17, a model-version grant overrides it for versions), and admins bypass all permissions (MLflow 3.17.1).^[14][15] So the boundaries are structural, as in the MLOps reference implementation: developers register into a candidates model; one promotion identity edits the production model; the gate reads tier and approval from the inventory, which only the MRM function and approvers change; admin accounts are break-glass. Alternative: a GRC or AI governance platform as the inventory (criteria).

%%{init: {"sequence": {"actorMargin": 30, "width": 120}}}%%
sequenceDiagram
  participant M as MRM function
  participant O as Owner
  participant V as Validator
  participant A as Approver
  participant P as Release gate
  O->>M: Intake form
  M-->>O: Tier 2 (temporary)
  O->>V: Evidence, access
  V-->>O: Findings
  alt Urgent, validation open
    V->>A: Interim view, exception
    A->>P: Provisional, expiry
  else Normal path
    O->>V: Fixes
    V->>A: Final report
    A->>P: Approval, conditions
  end
  Note over A,P: Gate checks the record
  Note over M,O: Next review date set

The PRA asks for “a risk-based materiality and complexity rating” per model; complexity can include “interpretability, explainability, transparency, and the potential for designer or data bias”.^[9] OSFI asks for “clear, measurable criteria for each risk dimension”.^[10] This Atlas’s starting rubric, used in the reference implementation:

A matrix of the two ratings gives the tier (decision tree); tier 4 means “inventory only”. Acceptance: two reviewers tier the same ten models independently and agree on at least eight.

Validation covers conceptual soundness (is the design fit for purpose?), outcomes analysis (do outputs match reality?) and ongoing monitoring.^[8] By model type:

Depth by tier. The PRA expects validation rigour and intensity “commensurate with the associated model tier”; OSFI’s rating drives the “frequency, intensity, and scope of model review”.^[9][10] This Atlas’s starting scopes:

Each finding gets a severity, owner and due date; high-severity findings block approval or become conditions.

The PRA expects policy to set procedures for “when a material change is made to a model” and “the materiality criteria to be used to identify the potential impact of prospective model changes”, with testing of “material changes over a period of time in dynamic models” that compares outputs before and after with actual outcomes.^[9] OSFI re-rates models after “a material change in the model’s use, data, or infrastructure”.^[10] This Atlas’s starting classes:

A pre-approved change envelope (this Atlas’s proposal) is approved once, like a use: fixed features, method and population, limits on score shift and segment performance, a retraining cadence. A version inside it reaches the Lab 3 gate with an approval entry written by the MRM function’s automation, not the developer. Re-check the envelope at each periodic review: small steps add up to the drift the PRA warns about.

Explanations serve audiences. US applicants refused credit must get reasons that are “specific and indicate the principal reason(s) for the adverse action” (Regulation B); the CFPB withdrew its 2022 circular on complex algorithms on 12 May 2025, but the regulation stands (as of October 2026).^[16][17] SHAP assigns “each feature an importance value for a particular prediction”;^[18] Cynthia Rudin argues that “explainable black boxes should be avoided in high-stakes decisions”.^[19] Make the choice between explaining and an interpretable model explicit at tier 1.

Fairness is measured by group, then judged. Model cards propose evaluation “across different cultural, demographic, or phenotypic groups”;^[11] Fairlearn’s MetricFrame computes such metrics.^[20] Three cautions:

1. Metrics conflict. Kleinberg and colleagues show that calibration within groups and balance for the positive and negative classes cannot all hold: “except in highly constrained special cases, there is no method that can satisfy these three conditions simultaneously”;^[21] Chouldechova shows one criterion “may lead to considerable disparate impact when recidivism prevalence differs across groups”.^[22] Policy names the leading metric per use.
2. The four-fifths rule is a screen. Under US employment-selection guidelines, a group selection rate below four-fifths of the highest “will generally be regarded by the Federal enforcement agencies as evidence of adverse impact”; smaller differences can still matter, larger ones may not with small numbers.^[23] Elsewhere it is, in this page’s view, a heuristic.
3. Measurement needs group data, often sensitive. The EU’s new Article 4a lets providers of high-risk systems “exceptionally process special categories of personal data” for bias detection under strict safeguards, and extends this to deployers and to other AI systems, but “does not create any obligation” to do so.^[24] The lawful basis belongs to PrivacyOps.

New York City bars automated employment decision tools unless the tool “has been subject to a bias audit within one year of the use of the tool”.^[25]

• Vendor models. US guidance expects validation “either by internal or outside parties”, plus monitoring and outcomes analysis;^[8] OSFI rates external models “on a standalone basis”.^[10] Contract for change notices (ProcurementOps).
• Foundation models behind an API. Approve the use and record the model version. Providers retire models on their own schedule: OpenAI promises at least six months’ notice for generally available models (barring safety or compliance concerns), less for previews;^[26] Anthropic gives “at least 60 days’ notice” for publicly released models, and partner platforms “set their own retirement schedules” (as of October 2026).^[27] A replacement is a material change.
• Agents. Singapore’s framework (January 2026) asks for “placing limits on agents’ powers” and checkpoints “at which human approval is required”.^[28] Autonomy raises the tier; tool permissions become approval conditions.

Separate what binds you from what helps you. Status as of October 2026; this is orientation, not legal advice.

EU details. Prohibitions apply from 2 February 2025 (those added by the Digital Omnibus, Regulation (EU) 2026/1744, from 2 December 2026), GPAI rules from 2 August 2025, transparency rules from August 2026.^[31][24] Credit scoring of natural persons is an Annex III use, “with the exception of AI systems used for the purpose of detecting financial fraud”; profiling Annex III systems are always high-risk.^[13]

Start with jobs, then products (as of October 2026):

Automate the mechanical: reconciliation with discovery sources, linting records (Lab 1), review reminders, refusing releases without an approval record (Lab 3), report drafts. Keep people for tier overrides, validation judgements, approvals and exceptions. An LLM can draft documentation but cannot provide effective challenge.

The core artefact is an inventory with a policy and a linter: one JSON file holds the tiering policy and a record per model with OSFI’s six minimum fields (building blocks) plus dates;^[10] a standard-library script computes tiers, review dates and findings. Assumptions (illustrative): a financial company with eight models of six types; owners are named people (fake IDs); review intervals of 12, 24 and 36 months are this Atlas’s starting values (the PRA asks for a frequency “consistent with the model tier”).^[9]

Example status: verified by execution (Python 3.8.10, 2026-10-17) with the script below.

{
  "as_of": "2026-10-17",
  "policy": {
    "tier_matrix": {"H": {"H": 1, "M": 1, "L": 2},
                    "M": {"H": 2, "M": 2, "L": 3},
                    "L": {"H": 3, "M": 3, "L": 4}},
    "people_impact_floor": 2,
    "review_months": {"1": 12, "2": 24, "3": 36},
    "warn_days": 60,
    "provider_warn_days": 180
  },
  "models": [
    {"id": "credit-pd-retail", "type": "ml", "origin": "internal",
     "use": "default risk for retail loan approval", "owner": "a.rossi",
     "developer": "credit-analytics", "materiality": "H", "complexity": "H",
     "people_impact": true, "status": "production", "recorded_tier": 1,
     "approved": "2025-11-03", "last_review": "2025-10-20", "monitoring": true},
    {"id": "fraud-rules-cards", "type": "rules", "origin": "internal",
     "use": "block suspicious card payments", "owner": "k.mensah",
     "developer": "fraud-strategy", "materiality": "H", "complexity": "L",
     "people_impact": false, "status": "production", "recorded_tier": 3,
     "approved": "2024-06-10", "last_review": "2024-06-01", "monitoring": true},
    {"id": "markdown-optimiser", "type": "optimisation", "origin": "internal",
     "use": "set clearance discounts", "owner": "l.chen",
     "developer": "pricing-science", "materiality": "M", "complexity": "M",
     "people_impact": false, "status": "production", "recorded_tier": 2,
     "approved": "2025-03-15", "last_review": "2025-03-01", "monitoring": true},
    {"id": "aml-vendor-scenarios", "type": "ml", "origin": "vendor",
     "use": "flag transactions for AML review", "owner": "p.novak",
     "developer": "vendor (example-vendor)", "materiality": "M", "complexity": "H",
     "people_impact": true, "status": "production", "recorded_tier": 2,
     "approved": "2026-04-01", "conditions_until": "2026-09-30",
     "last_review": "2026-03-20", "monitoring": true},
    {"id": "support-assistant", "type": "llm-app", "origin": "third-party-api",
     "use": "answer customer questions", "owner": "s.haddad",
     "developer": "customer-ai", "materiality": "M", "complexity": "H",
     "people_impact": false, "status": "production", "recorded_tier": 2,
     "approved": "2026-05-12", "last_review": "2026-05-01", "monitoring": true,
     "provider_retirement": "2027-01-14"},
    {"id": "refund-agent", "type": "agent", "origin": "internal",
     "use": "issue refunds up to a limit", "owner": "s.haddad",
     "developer": "customer-ai", "materiality": "M", "complexity": "H",
     "people_impact": true, "status": "production", "recorded_tier": 2,
     "approved": null, "last_review": null, "monitoring": false},
    {"id": "churn-propensity", "type": "ml", "origin": "internal",
     "use": "target retention offers", "owner": "",
     "developer": "marketing-analytics", "materiality": "L", "complexity": "M",
     "people_impact": false, "status": "production", "recorded_tier": 3,
     "approved": "2024-02-01", "last_review": "2024-01-15", "monitoring": true},
    {"id": "branch-forecast-sheet", "type": "spreadsheet", "origin": "internal",
     "use": "forecast branch staffing", "owner": "j.berg",
     "developer": "finance-planning", "materiality": "L", "complexity": "L",
     "people_impact": false, "status": "production", "recorded_tier": 4,
     "approved": null, "last_review": null, "monitoring": false}
  ]
}

Example status: verified by execution (Python 3.8.10, 2026-10-17); the output below is the actual output.

"""Lint a model inventory: compute risk tiers, review due dates and blocking findings.

Usage: python3 modelops_inventory.py model-inventory.json
Standard library only. The tier matrix, review intervals and thresholds live in the
inventory's "policy" block (this Atlas's illustrative policy, not a regulatory rule).
Exit code 1 if any ERROR (unowned, unapproved or out-of-conditions use in production).
"""
import calendar
import json
import sys
from datetime import date


def parse(value):
    return date.fromisoformat(value) if value else None


def add_months(d, months):
    m = d.month - 1 + months
    year, month = d.year + m // 12, m % 12 + 1
    return date(year, month, min(d.day, calendar.monthrange(year, month)[1]))


def tier_of(model, policy):
    tier = policy["tier_matrix"][model["materiality"]][model["complexity"]]
    if model.get("people_impact"):
        tier = min(tier, policy["people_impact_floor"])
    return tier


def check(model, policy, today):
    findings = []
    tier = tier_of(model, policy)
    live = model["status"] == "production"
    if not model.get("owner"):
        findings.append(("ERROR", "no accountable owner"))
    for field in ("use", "developer"):
        if not model.get(field):
            findings.append(("WARN", f"missing inventory field {field}"))
    if tier != model.get("recorded_tier"):
        findings.append(("WARN", f"recorded tier {model.get('recorded_tier')}, "
                                 f"rubric says {tier}: re-tier"))
    if tier == 4:
        return tier, None, findings          # inventory only: owner and attestation
    if live and not model.get("approved"):
        findings.append(("ERROR", "in production without approval"))
    until = parse(model.get("conditions_until"))
    if live and until and until < today:
        findings.append(("ERROR", f"approval conditions expired {until}"))
    due = None
    last = parse(model.get("last_review"))
    if last:
        due = add_months(last, policy["review_months"][str(tier)])
        if due < today:
            findings.append(("WARN", f"review overdue since {due}"))
        elif (due - today).days <= policy["warn_days"]:
            findings.append(("INFO", f"review due {due}"))
    elif live:
        findings.append(("WARN", "never reviewed"))
    if live and not model.get("monitoring"):
        findings.append(("WARN", "no monitoring plan"))
    retire = parse(model.get("provider_retirement"))
    if retire and (retire - today).days <= policy["provider_warn_days"]:
        findings.append(("WARN", f"provider retires the model {retire}: "
                                 "plan and approve a replacement"))
    return tier, due, findings


def main(path):
    with open(path, encoding="utf-8") as fh:
        inv = json.load(fh)
    today, policy = parse(inv["as_of"]), inv["policy"]
    counts = {"ERROR": 0, "WARN": 0, "INFO": 0}
    tiers = {}
    print(f"Model inventory check as of {today}")
    for model in inv["models"]:
        tier, due, findings = check(model, policy, today)
        tiers[tier] = tiers.get(tier, 0) + 1
        print(f"{model['id']:<22} {model['type']:<12} tier {tier}  "
              f"next review {due or '-'}")
        for level, text in findings:
            counts[level] += 1
            print(f"    {level:<5} {text}")
    summary = ", ".join(f"tier {t}: {n}" for t, n in sorted(tiers.items()))
    print(f"{len(inv['models'])} models ({summary})")
    print(f"findings: {counts['ERROR']} ERROR, {counts['WARN']} WARN, "
          f"{counts['INFO']} INFO")
    return 1 if counts["ERROR"] else 0


if __name__ == "__main__":
    if len(sys.argv) != 2:
        sys.exit("usage: python3 modelops_inventory.py <inventory.json>")
    sys.exit(main(sys.argv[1]))

$ python3 modelops_inventory.py model-inventory.json
Model inventory check as of 2026-10-17
credit-pd-retail       ml           tier 1  next review 2026-10-20
    INFO  review due 2026-10-20
fraud-rules-cards      rules        tier 2  next review 2026-06-01
    WARN  recorded tier 3, rubric says 2: re-tier
    WARN  review overdue since 2026-06-01
markdown-optimiser     optimisation tier 2  next review 2027-03-01
aml-vendor-scenarios   ml           tier 2  next review 2028-03-20
    ERROR approval conditions expired 2026-09-30
support-assistant      llm-app      tier 2  next review 2028-05-01
    WARN  provider retires the model 2027-01-14: plan and approve a replacement
refund-agent           agent        tier 2  next review -
    ERROR in production without approval
    WARN  never reviewed
    WARN  no monitoring plan
churn-propensity       ml           tier 3  next review 2027-01-15
    ERROR no accountable owner
branch-forecast-sheet  spreadsheet  tier 4  next review -
8 models (tier 1: 1, tier 2: 5, tier 3: 1, tier 4: 1)
findings: 3 ERROR, 5 WARN, 1 INFO

How to read it. The errors matter most: an agent live without approval, a vendor model used after its conditional approval lapsed, an unowned model. The fraud rules block card payments at scale, so the rubric moves them to tier 2. The assistant’s provider retirement (illustrative date) starts re-approval planning. The spreadsheet stays as tier 4, although US guidance excludes “simple arithmetic calculations, such as those found within spreadsheets”:^[8] governance scope is your choice.

Objective: find and resolve blocking findings. Prerequisites: Python 3.8+; no account or cost; 45 minutes.

1. Save the reference JSON as model-inventory.json and the script as modelops_inventory.py.
2. Run python3 modelops_inventory.py model-inventory.json; echo $?.
3. Record four decisions: give churn-propensity a named owner; set refund-agent to "suspended" until approved; extend the AML vendor’s conditions_until to "2026-12-31" only if your approver signs an exception; set fraud-rules-cards to recorded_tier: 2.
4. Run again, then add five of your own models.

Expected output (checked by execution). Step 2: the reference output, exit code 1. Step 3: 0 ERROR, 2 WARN, 1 INFO (the overdue fraud review and the provider retirement remain), exit code 0. Validation: two colleagues agree with every tier. Troubleshooting: KeyError means a rating other than H, M or L; ValueError: Invalid isoformat string, a date not in YYYY-MM-DD. Teardown: delete the folder, or commit the inventory.

Objective: produce the fairness evidence a validator reviews, and see two metrics disagree. Prerequisites: Python 3.8+; synthetic counts, no personal data; 45 minutes. Real credit data lacks outcomes for declined applicants; the synthetic file assumes them known.

Example status: verified by execution (Python 3.8.10, 2026-10-17); the output below is the actual output.

{
  "model": "credit-pd-retail v7",
  "window": "synthetic backtest; outcomes of declined applicants assumed known",
  "min_group_size": 100,
  "thresholds": {"impact_ratio": 0.8, "tpr_gap": 0.05},
  "groups": {
    "A": {"applicants": 2400, "approved": 1440, "good": 1900, "good_approved": 1330},
    "B": {"applicants": 1100, "approved": 495, "good": 820, "good_approved": 455},
    "C": {"applicants": 60, "approved": 33, "good": 45, "good_approved": 31}
  }
}

"""Disaggregated outcome report for a binary approve/decline model.

Usage: python3 fairness_report.py outcomes.json
Per group: approval rate, impact ratio against the highest-rate group, true positive
rate (approval rate among applicants who later repaid) and default rate among the
approved. Flags are prompts for investigation, not verdicts of (un)fairness.
"""
import json
import sys


def main(path):
    with open(path, encoding="utf-8") as fh:
        data = json.load(fh)
    limit, th = data["min_group_size"], data["thresholds"]
    rows = {}
    for name, g in data["groups"].items():
        rows[name] = {
            "n": g["applicants"],
            "rate": g["approved"] / g["applicants"],
            "tpr": g["good_approved"] / g["good"],
            "bad": (g["approved"] - g["good_approved"]) / g["approved"],
        }
    usable = {k: v for k, v in rows.items() if v["n"] >= limit}
    top_rate = max(v["rate"] for v in usable.values())
    top_tpr = max(v["tpr"] for v in usable.values())
    print(f"{data['model']} | {data['window']}")
    print(f"{'group':<6}{'n':>6}{'approval':>10}{'impact':>8}{'TPR':>7}"
          f"{'defaults':>10}  flags")
    for name, r in rows.items():
        flags = []
        if r["n"] < limit:
            flags.append(f"n<{limit}: too small to conclude")
            impact = "-"
        else:
            ratio = r["rate"] / top_rate
            impact = f"{ratio:.2f}"
            if ratio < th["impact_ratio"]:
                flags.append("impact ratio below screen")
            if top_tpr - r["tpr"] > th["tpr_gap"]:
                flags.append(f"TPR gap {top_tpr - r['tpr']:.3f}")
        print(f"{name:<6}{r['n']:>6}{r['rate']:>10.1%}{impact:>8}{r['tpr']:>7.1%}"
              f"{r['bad']:>10.1%}  {'; '.join(flags) or 'none'}")
    return 0


if __name__ == "__main__":
    if len(sys.argv) != 2:
        sys.exit("usage: python3 fairness_report.py <outcomes.json>")
    sys.exit(main(sys.argv[1]))

$ python3 fairness_report.py outcomes.json
credit-pd-retail v7 | synthetic backtest; outcomes of declined applicants assumed known
group      n  approval  impact    TPR  defaults  flags
A       2400     60.0%    1.00  70.0%      7.6%  none
B       1100     45.0%    0.75  55.5%      8.1%  impact ratio below screen; TPR gap 0.145
C         60     55.0%       -  68.9%      6.1%  n<100: too small to conclude

Steps: run the script; set group B to "approved": 560, "good_approved": 510 (a lower cut-off) and rerun. Expected output (checked by execution): B’s impact ratio becomes 0.85 and passes the screen, its true-positive-rate gap falls to 0.078 but still exceeds 0.05, and its default rate among the approved rises from 8.1% to 8.9%. Validation: you can name the policy’s primary metric for this use and the cost in defaults. Troubleshooting: ZeroDivisionError means a group with no approvals or no good outcomes. Teardown: delete the files.

Objective: promote a version only when the inventory holds its tier’s evidence, and make sure nobody else can. Prerequisites (the control depends on them):

• An MLflow server you administer, with authentication on (--app-name basic-auth, default permission READ). Written against the MLflow 3.17.1 docs; permissions changed in 3.13 (roles) and 3.17 (DENY; version-level grants).^[14][15]
• credit-pd-retail-candidates, editable by data scientists, and credit-pd-retail, created first by the promotion identity, the only non-admin with edit rights on it (creating a registered model makes the creator its manager).^[14] No data scientist is an MLflow or workspace admin, or holds EDIT or MANAGE on registered_model:* or registered_model_version:* (with default READ, the version grant alone edits the production model’s versions).^[15]
• Only the MRM function and approvers can change tiers and approvals in the inventory (a GRC workflow, or a repository whose approval files need their review). The lab uses a local copy.

The gate uses the stricter of the recorded and rubric tiers, refuses tiers outside the policy, and accepts a provisional approval only with an exception ID, approver, monitoring plan and an expiry at most 14 days out, renewed at most once. Method names follow the MLflow client reference.^[39]

Example status: syntax-checked; logic executed with a stub client; not run against MLflow.

"""Promote an approved candidate version into the production registered model.

Usage: python3 promotion_gate.py <inventory.json> <model-id> <version> [alias]
The tier and the approval come from the inventory (changed only by the MRM function and
approvers), never from registry tags that developers can write. On a pass, version
<version> of "<model-id>-candidates" is copied into "<model-id>", the approval is
recorded as tags on the copy and the alias (default "champion") is moved.
Run it as the promotion identity: the only identity with EDIT on the production model.
"""
import json
import sys
from datetime import date

REQUIRED = {  # this Atlas's illustrative policy: approval evidence per risk tier
    1: ["validation_report", "committee_approval", "monitoring_plan"],
    2: ["validation_report", "approver", "monitoring_plan"],
    3: ["approver"],
    4: [],  # inventory only
}
PROVISIONAL = ["exception_id", "approver", "monitoring_plan", "date",
               "conditions_until"]
MAX_PROVISIONAL_DAYS, MAX_RENEWALS = 14, 1


def effective_tier(record, policy):
    """The stricter of the recorded tier and the rubric's tier (1 is strictest)."""
    tier = policy["tier_matrix"][record["materiality"]][record["complexity"]]
    if record.get("people_impact"):
        tier = min(tier, policy["people_impact_floor"])
    return min(tier, record.get("recorded_tier") or tier)


def missing_evidence(record, version, tier, today):
    """Return a list of problems; an empty list means the gate passes."""
    if tier not in REQUIRED:
        return [f"tier {tier} is not in the policy"]
    if tier == 4:
        return []
    approval = record.get("approvals", {}).get(str(version))
    if not approval:
        return [f"no approval record for version {version}"]
    decision = approval.get("decision")
    if decision == "provisional":
        needed = PROVISIONAL
    elif decision in ("approved", "approved_with_conditions"):
        needed = REQUIRED[tier]
    else:
        return [f"decision is {decision}"]
    problems = [f"missing {key}" for key in needed if not approval.get(key)]
    until = approval.get("conditions_until")
    if decision == "approved_with_conditions" and not until:
        problems.append("conditional approval without conditions_until")
    if until and date.fromisoformat(until) < today:
        problems.append(f"conditions expired {until}")
    if decision == "provisional" and until and approval.get("date"):
        days = (date.fromisoformat(until) - date.fromisoformat(approval["date"])).days
        if days > MAX_PROVISIONAL_DAYS:
            problems.append(f"provisional for {days} days (max {MAX_PROVISIONAL_DAYS})")
        if approval.get("renewals", 0) > MAX_RENEWALS:
            problems.append(f"renewals {approval['renewals']} (max {MAX_RENEWALS})")
    return problems


def main(inventory_path, model_id, version, alias="champion"):
    with open(inventory_path, encoding="utf-8") as fh:
        inv = json.load(fh)
    record = {m["id"]: m for m in inv["models"]}.get(model_id)
    if record is None:
        print(f"REFUSED {model_id}: not in the inventory")
        return 1
    tier = effective_tier(record, inv["policy"])
    problems = missing_evidence(record, version, tier, date.today())
    if problems:
        print(f"REFUSED {model_id} v{version} (tier {tier}): " + "; ".join(problems))
        return 1
    from mlflow import MlflowClient  # imported late: the logic is testable without MLflow

    client = MlflowClient()
    copy = client.copy_model_version(f"models:/{model_id}-candidates/{version}", model_id)
    for key, value in record.get("approvals", {}).get(str(version), {}).items():
        client.set_model_version_tag(model_id, copy.version, key, str(value))
    client.set_registered_model_alias(model_id, alias, copy.version)
    print(f"PROMOTED {model_id}-candidates v{version} -> {model_id} "
          f"v{copy.version} @{alias} (tier {tier})")
    return 0


if __name__ == "__main__":
    if len(sys.argv) not in (4, 5):
        sys.exit("usage: promotion_gate.py <inventory.json> <model-id> <version> [alias]")
    sys.exit(main(*sys.argv[1:]))

Merge this approvals object into the credit-pd-retail record of model-inventory.json; give version 8 today’s date and a conditions_until 30 days later.

Example status: syntax-checked; used in the stub run.

{
  "approvals": {
    "7": {"decision": "approved", "date": "2026-10-09",
          "validation_report": "VR-2026-031", "committee_approval": "MRC-2026-10-08",
          "monitoring_plan": "MP-credit-pd-v7"},
    "8": {"decision": "provisional", "date": "2026-10-17",
          "conditions_until": "2026-11-16", "exception_id": "EX-2026-044",
          "approver": "cro-delegate", "monitoring_plan": "MP-credit-pd-v8",
          "renewals": 0},
    "9": {"decision": "pending", "validation_report": "VR-2026-037"}
  }
}

Steps: as the promotion identity, run python3 promotion_gate.py model-inventory.json credit-pd-retail 7, then versions 8 and 9; set version 8’s conditions_until to ten days after its date and rerun it. Expected output (stub run, 2026-10-17): v7 PROMOTED credit-pd-retail-candidates v7 -> credit-pd-retail v1 @champion (tier 1), exit 0; v8 REFUSED ... provisional for 30 days (max 14) and v9 REFUSED ... decision is pending, exit 1; v8 after the fix PROMOTED ... v2, exit 0. Validation: (1) a data scientist’s attempt to create a version in, tag or move an alias of credit-pd-retail must fail, or the gate is advisory; (2) with recorded_tier 3 and no committee_approval on v7, the gate still applies tier 1 and refuses. Troubleshooting: a permission error on the copy means the promotion identity lacks rights on one of the two models. Teardown: delete the test versions, alias and approvals block.

The Epic Sepsis Model, “implemented at hundreds of US hospitals”, was validated externally at Michigan Medicine on 38,455 hospitalisations: an area under the ROC curve of 0.63 (0.5 is chance), 67% of sepsis patients missed, and alerts for 18% of all hospitalisations; the authors concluded that it “has poor discrimination and calibration”.^[40] Lesson: validate a vendor model on your population; alert burden is part of its risk.

Obermeyer and colleagues found that a widely used care-management algorithm predicted health-care costs rather than illness, so “At a given risk score, Black patients are considerably sicker than White patients”; remedying this would raise the share of Black patients receiving extra help from 17.7% to 46.5%.^[41] Lesson: conceptual soundness asks whether the target is the right proxy.

In November 2021 Zillow wound down Zillow Offers; its CEO said “the unpredictability in forecasting home prices far exceeds what we anticipated” and that continuing “would result in too much earnings and balance-sheet volatility”.^[42] It reported a $304 million inventory write-down “as a result of purchasing homes at higher prices than our current estimates of future selling prices”.^[43] Lesson (this page’s reading, not Zillow’s): when a model drives large exposure, approval conditions need exposure caps, enforced in DecisionOps.

In Moffatt v. Air Canada (British Columbia Civil Resolution Tribunal, 14 February 2024), the airline’s chatbot said bereavement fares could be claimed retroactively; they could not. The tribunal called the suggestion that the chatbot is “a separate legal entity that is responsible for its own actions” “a remarkable submission”, held: “It makes no difference whether the information comes from a static page or a chatbot”, and ordered Air Canada to pay C$812.02.^[44] Lesson: customer-facing LLM applications need an owner and approved uses.

Owned share, review currency and exception age lead; incidents and audit findings lag. Review monthly in the MRM function, quarterly with senior management.

From the reference implementation (eight models, all in production):

• Owned share: 7 ÷ 8 = 87.5% (churn-propensity has no owner).
• Review currency: of seven tier 1–3 models, fraud-rules-cards is overdue and refund-agent never reviewed: 5 ÷ 7 = 71%.
• Unapproved or lapsed use: 2 (the agent; the AML vendor model).
• Monitoring coverage for tiers 1–3: 6 ÷ 7 = 86%.

After the Lab 1 fixes, owned share is 100% and unapproved or lapsed use 0. Review currency rises to 5 ÷ 6 = 83% only because the suspended agent left the denominator; the fraud rules are still overdue. Vanity traps: “models approved this quarter”, “share of models with a model card” (empty cards count).

Auditability. The approval records are the trail; collecting evidence across systems belongs to ComplianceOps.

Illustrative calculation (assumed effort, not a benchmark): initial validation takes 120 hours for tier 1, 40 for tier 2 and 8 for tier 3; a periodic review a third of that. For 15 tier-1, 60 tier-2 and 150 tier-3 models reviewed every 12, 24 and 36 months: 15 × 40 + 60 × 13.3 ÷ 2 + 150 × 2.7 ÷ 3 ≈ 600 + 400 + 135 = 1,135 review hours a year; ten new tier-2 models add 400. At about 1,500 productive hours per validator that is one to 1.5 validators; all 225 at tier-1 depth would take 9,000 hours, six people. Compute and token spend belong to FinOps for AI, carbon to GreenOps; ModelOps’s own cost driver is hand-made evidence.

• Internal audit evaluates whether the practices “are rigorous and effective” without redoing validation.^[8]
• Validate the tiering itself: the PRA expects “periodic validation, or other objective and critical review” of it.^[9]
• Reconcile quarterly: trace twenty sampled production endpoints and AI service accounts to the inventory.
• Risk indicators (this Atlas’s thresholds): any unapproved or lapsed use; a tier-1 review 30 days overdue; an exception past expiry; coverage below 95%.

An illustrative scenario: a marketing campaign changes the applicant mix, and the tier-1 credit model breaches two approval conditions: population stability (how far applicants have moved from the development sample, a drift measure from MLOps) and an approval-rate band. OSFI expects contingency plans “for model unavailability, deterioration in model performance, or outright failure”.^[10]

flowchart TB
  D["Monitoring breach on tier-1 model"] --> S{"Harm ongoing to people or money?"}
  S -- "Yes" --> C["Contain: fallback model, tighter cut-off or manual review"]
  S -- "No" --> W["Watch: owner reviews within 2 business days"]
  C --> N["Notify approver, MRM function, users"]
  W --> N
  N --> R{"Cause"}
  R -- "Data or pipeline defect" --> F["Fix with MLOps; backfill decisions"]
  R -- "Population changed" --> O["Overlay with expiry; redevelop"]
  R -- "Used outside approved scope" --> U["Stop the out-of-scope use"]
  F --> V["Validator reviews the fix"]
  O --> V
  U --> V
  V --> A{"Approver decision"}
  A -- "Restore" --> OK["Approved: conditions updated"]
  A -- "Not yet" --> X["Restricted or suspended; exception logged"]
  classDef core fill:#e8f3f2,stroke:#16756f,color:#122029
  classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029
  classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029
  class W,N,F,O,U,V,OK core
  class S,R,A decision
  class D,C,X risk

1. Record the breach against the approval conditions; contain if harm is ongoing.
2. Notify the approver and users; in the EU, once high-risk duties apply, a deployer reports a serious incident to the provider first, then the authorities (Article 26).^[13]
3. Diagnose with MLOps: data defect, drift, or use outside the validated population?
4. Remediate: fix, expiring overlay, or redevelopment (a material change).
5. Re-approve on the validator’s review; add campaigns to the review triggers.

Build an explicit practice when any of these hold: models inform decisions about people’s access to credit, jobs, insurance, care or public services; your supervisor publishes model risk expectations, or you deploy EU AI Act Annex III systems; models are bought or called through APIs faster than anyone reviews them, or agents can act; nobody can list production models with owners.

Keep it minimal, or wait, when a few low-stakes models have clear owners (an inventory and quarterly review suffice); approvals would have no delivery pipeline to enforce them (fix MLOps first); or the “programme” would be a policy with no inventory.

Hidden responsibilities: independent validators, a vendor-review budget, and someone who can say no to a launch.

The tree encodes the policy file’s matrix and override, so hand tiering and the linter agree.

%%{init: {"flowchart": {"wrappingWidth": 130, "nodeSpacing": 24, "rankSpacing": 34}}}%%
flowchart TB
  S["New model or material change"] --> Q1{"Output used in a decision?"}
  Q1 -- "No" --> T4A["Tier 4: inventory only"]
  Q1 -- "Yes" --> QH{"Materiality high?"}
  QH -- "Yes" --> Q3{"Complexity low?"}
  Q3 -- "No" --> T1["Tier 1: independent validation, committee approval"]
  Q3 -- "Yes" --> T2["Tier 2: independent validation, MRM approval"]
  QH -- "No" --> QP{"Affects individuals' access or rights?"}
  QP -- "Yes" --> T2
  QP -- "No" --> QM{"Materiality medium?"}
  QM -- "Yes" --> QC1{"Complexity low?"}
  QC1 -- "No" --> T2
  QC1 -- "Yes" --> T3["Tier 3: peer review, owner approval"]
  QM -- "No, low" --> QC2{"Complexity low?"}
  QC2 -- "No" --> T3
  QC2 -- "Yes" --> T4B["Tier 4: inventory only"]
  classDef core fill:#e8f3f2,stroke:#16756f,color:#122029
  classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029
  classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029
  classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029
  class S neutral
  class Q1,QH,Q3,QP,QM,QC1,QC2 decision
  class T1 risk
  class T2,T3,T4A,T4B core

How much ModelOps? If three or more conditions above hold, staff a part-time MRM function and an independent reviewer; if a supervisor oversees your models, plan a separate validation function.

These levels are this Atlas’s model; no official ModelOps maturity model exists.

Dependencies: a registry to hook into (MLOps); a risk appetite (RiskOps).

• Purpose, decision supported and approved uses in one paragraph; type and origin recorded.
• Owner is a named person; data sources and regulatory scope set; provisional tier within five working days.
• For a change: class recorded; material changes re-tiered.

• Validation covers conceptual soundness, outcomes analysis and a monitoring plan.
• Performance by group, with the primary fairness metric named; explanations fit each audience.
• Limits, conditions and expiry written into the approval record that the release gate reads.
• Vendor or API models: documentation received; change and retirement notices routed to the owner.
• Agents: tool permissions, action limits and human checkpoints listed.

• Used only for approved purposes; consumers current; breaches, overrides and overlays examined.
• Tier still right; change envelope still valid; findings closed or re-dated with reasons.

• Consumers notified with a date; traffic and access stopped; downstream effects watched.
• Model, documentation and approval history retained for the agreed period.
• Third-party contracts and API keys closed (ProcurementOps).

MODEL GOVERNANCE CHARTER — <organisation>            Owner: <name>   Review: yearly
Mission:    Every model we rely on is known, owned, used within approved limits,
            and retired on purpose.
Scope:      ML, rules, optimisation, LLM applications, agents, vendor and API models,
            spreadsheets used in decisions. Regulatory scope recorded per model.
Principles: Inventory before policy · Rigour follows risk · Independence where it
            matters · Approve a use, not a model · Evidence lives with the model ·
            Bought is not exempt · Retire on purpose.
Tiers:      1 committee approval · 2 MRM approval · 3 owner approval · 4 inventory only.
Changes:    material = new approval · bounded = validator + approver ·
            in envelope = automated record · record only = log.
Goals:      Owned share 100% · unapproved or lapsed use 0 · review currency >= 95%.
Decision rights: Approvers per tier: <names>. Exceptions: <who>, maximum <n> days.
Non-goals:  Approving every spreadsheet; replacing developers' testing; legal advice.

MODEL RECORD — <model-id>                                      Date: <YYYY-MM-DD>
Name and description:          Type: ml | rules | optimisation | llm-app | agent | other
Origin: internal | vendor <name> | API <provider, model version>
Owner (person):            Developer:            Users:
Purpose and decision supported:
Approved uses:                         Not approved for:
Materiality (H/M/L) and why:           Complexity (H/M/L) and why:
Affects individuals' access or rights? Y/N    Can act without a person? Y/N
Regulatory scope: bank MRM Y/N · EU AI Act class <...> · other <...>
Data sources and limitations:
Dependencies (models, data, providers):
Tier: proposed <n>  confirmed <n> by <name>
Approval: <date> by <name> · conditions <...> · conditions until <date>
Change envelope: <id, fixed features, limits>   Last material change: <date, what>
Last review: <date>   Next review: <date>   Monitoring plan: <link>
Provider retirement date (if any): <date>

APPROVAL — <model-id> v<version>                 Tier: <n>    Date: <YYYY-MM-DD>
Validation report: <link>        Validator: <name>     Independent of developer: Y/N
Decision: approved | approved_with_conditions | provisional | rejected
Approved uses: <list>                 Prohibited uses: <list>
Conditions: <limits, monitoring thresholds, manual review rates, exposure caps>
Provisional only: exception ID <id> · expiry <= 14 days · renewals 0 | 1
Open findings and due dates: <id, severity, owner, date>
Expires / next review: <date>      Triggers for early review: <list>
Approver: <name, role>         Recorded in the inventory by: <approver or MRM function>

EXCEPTION — <model-id>                                  Requested: <YYYY-MM-DD>
Type: use before validation | use outside approved scope | performance breach
Business need and why it cannot wait:
Limits while the exception lasts: <volume caps, manual review, overlays>
Extra monitoring and reporting: <metric, frequency, recipient>
Expiry date (maximum <n> days): <date>          Renewal allowed: once | never
Compensating controls owner:            Approver:            Logged in register: Y/N

Is ModelOps the same as MLOps? Some use the terms interchangeably;^[5] this Atlas separates building and running (MLOps) from deciding allowed uses (ModelOps).

Does a rules engine or a spreadsheet count as a model? It depends on whose definition (what ModelOps is not); inventory anything whose output drives a decision.

We only call a foundation model through an API. What is there to validate? Your use: task, prompts and retrieval, outcomes on your cases, limits, and the provider’s retirement date (Scenario 5). LLMOps supplies the evaluations; ModelOps decides whether they suffice.

Does every retrained version need a committee? No: retraining inside a pre-approved envelope is recorded automatically; a new method, use or provider model is a material change (change classes).

Can an agent be approved at all? Yes, as a bounded use: named tools, data and actions, limits and human checkpoints.^[28] An agent with open-ended permissions cannot be tiered meaningfully.

• Aggregate model risk. Models sharing a data source, vendor or assumption fail together; US guidance assesses risk “both individually and in aggregate”.^[8] Inventory dependency fields make this measurable.
• Generative and agentic AI in banks. The US agencies plan “a request for information” on model risk, “including generative AI and agentic AI” (none in the Federal Register as of 18 October 2026).^[29]

• Prerequisites: MLOps basics; a confusion matrix; risk vocabulary (RiskOps).
• One day: this page; SR 26-2 (12 pages) and OSFI E-23’s inventory appendix; Lab 1.^[8][10]
• One week: Labs 2 and 3; model cards and datasheets;^[11][12] tier ten real models with a colleague.
• Thirty days: the roadmap’s first month for one business unit; one tier-2 validation report; PRA SS1/23.^[9]
• Specialise: fairness (Kleinberg et al.; Fairlearn’s user guide^[21][45]), EU AI Act work with ComplianceOps, or agent evaluation with LLMOps. Project: inventory, rubric and gate for an open-source model zoo.

Situation (illustrative). A card issuer’s tier-1 fraud model misses a new attack; losses rise by an estimated 200,000 (currency units) a week. A retrained version is ready; full validation takes six weeks, a focused review two. Option A: wait. Option B: deploy now under a provisional approval.

What the guidance allows. US guidance accepts that “certain circumstances (e.g., an urgent business need) may necessitate using the model before validation is completed”, with controls such as “placing limits on model use or more closely monitoring its performance”.^[8] The PRA treats it as a temporary exception supported by senior management.^[9]

The attractive choice and why it can be wrong. Option B fails if “provisional” has no teeth: the model may block legitimate customers in a segment the retraining over-weighted (a harm the loss figure ignores), and the exception is renewed while validation never finishes.

Recommendation. Option B, with conditions: (1) 72 hours in shadow, comparing decline rates by segment with the old model; (2) a score band sent to manual review instead of auto-decline; (3) daily monitoring of declines and complaints by segment; (4) a provisional record with an exception ID, a senior approver, the monitoring plan and a 14-day expiry, renewable once after the focused review, as the Lab 3 gate checks; (5) full validation scheduled. Uncertainty: the loss estimate may be off by half; if one segment’s declines jump in shadow, revert to A with a rule-based patch.

You should now be able to explain the meanings of ModelOps, build an inventory and tier models, design approval and change evidence, evaluate fairness reports, and decide exceptions. Self-check:

1. A team says its assistant needs no review because “it just calls the provider’s model”. What is wrong?

2. Your US bank’s MRM policy covers “all models per SR 11-7”. What should change after April 2026?

3. In Lab 2, raising group B’s approvals lifts its impact ratio from 0.75 to 0.85. Is the model now fair?

4. The inventory shows 400 models, all tier 1, and a two-year validation backlog. What do you do first?

5. An EU bank retrains a third-party credit-scoring system on its own data until its behaviour changes materially. What changes under the AI Act?

ModelOps is a young name for an old duty. Banks learned that a sound model used in the wrong place is a risk, that someone independent must be able to say no, and that control starts with a list. What changed is the portfolio: rules, vendor scores, LLM applications and agents now shape decisions, and no single rulebook covers them.

The useful version is modest: one inventory, an agreed rubric, approvals that name uses and expire, a gate only approvers can satisfy, and validation where the stakes justify it. Organisations that skip it still have model risk; they find it in an audit, a court or a loss.

Each source was opened on its “Checked” date; other dates are the source’s own. Claim-level support is in research/modelops-ledger.md.

Further reading: Google Cloud, Practitioners guide to MLOps (2021), “Model governance”; the AI Act Explorer’s Digital Omnibus view.

Changelog. 2026-10-17, r0: first draft. 2026-10-17, r1 (fact check reviews/modelops-r1.md): gate reads approvals from the inventory; one provisional path; change classes added; three figures redrawn; attributions corrected. 2026-10-18, post-score polish: SR 26-2 exclusion in full; validation depth by tier; MLflow pin and grant preconditions (reviews/modelops-polish-check.md); tier wording; Figures 1–2 relaid.