--- title: "AnalyticsOps" slug: analyticsops category: data term_status: ambiguous evidence_status: verified summary: "AnalyticsOps has no standard definition. On this page it means operating the layer where people consume data: versioned metric definitions, a semantic layer that serves them, and BI content that is certified, monitored and retired, so each important metric has one trusted definition." audience: [Analytics engineers, BI developers, Analytics and data leaders, KPI owners in the business, Beginners] last_verified: 2026-10-18 key_takeaways: - "The label is ambiguous: Open Data Group (renamed ModelOp in 2019) and Teradata used \"AnalyticOps\" for deploying analytic models; an analyst and a BI-testing vendor use the term for the analytics last mile. This page takes the last-mile meaning." - "The core problem is drifting copies of definitions, not broken pipelines: Uber traced 6.53M versus 6.20M sessions for the same city and period to stale filters in one tool's query." - "Treat each important metric as a product with a written definition, a business owner, a version, a certification status and a list of consumers." - "Put definitions where every consumer can reach them: the BI tool's model if there is one tool; a tool-independent semantic layer if several tools, notebooks or AI assistants need the same numbers." - "Change a certified definition like an API: version it, measure its effect, tell consumers, provide a back series. Correct errors promptly and say how big they were." - "Retire content on usage evidence (views and scheduled deliveries) and only after notice. Export usage logs before they age out; certified content can also be unused." related: [dataops, bizops, decisionops, datasecops, productops, revops, modelops, mlops] --- ## What AnalyticsOps is ### In 30 seconds **AnalyticsOps** has no agreed definition. **On this page it means operating the layer where people consume data:** metric definitions, the semantic layer that serves them and the dashboards built on top, so every important number has one trusted definition, an owner and a known freshness, and unused content is retired. [DataOps](../dataops/) makes the tables correct and on time; AnalyticsOps makes "revenue" mean the same in the board pack, the sales dashboard and the spreadsheet. ::: note Four uses of the label 1. **"AnalyticOps" for deploying analytic models.** Open Data Group, renamed ModelOp in 2019,[^odgrebrand] called it "the lynchpin of an organization that connects data science, IT, and business teams";[^modelop] Teradata offered an "AnalyticOps Accelerator", whose Python client points users to a "ModelOps UI".[^aoa] This meaning belongs to [ModelOps](../modelops/). 2. **DataOps-style agility for analytics and BI.** A Ventana Research analyst wrote in 2021 that "The same desire for agility suggests that organizations need to adopt AnalyticOps".[^menninger] 3. **The BI "last mile".** A BI-testing vendor: "Unlike DataOps and DevOps, AnalyticsOps takes into account the last mile of the data journey; the Analytics layer."[^wiiisdom] 4. **Operating an analytics platform.** An Oracle blog post; it refused automated access, so this page does not rely on it. This page combines meanings 2 and 3, an **editorial distinction** of this Atlas. ::: **For** analytics engineers and BI developers, analytics leaders who decide what to certify and retire, business owners of KPIs, and beginners who wonder why two dashboards disagree. ### At a glance | Item | Summary | |---|---| | Term status | **Ambiguous.** No standard, body or certification; four uses, two spellings. A 2021 prediction that one-third of organisations would adopt it by 2024 was never measured, as far as this page found.[^menninger] | | Goal | One trusted definition for each important metric, and a dashboard estate that people use and trust. | | In scope | Metric definitions and lifecycle; the semantic layer; certifying, monitoring and retiring BI content; governed self-service; report freshness; conflicting-number incidents. | | Out of scope | Pipelines and data tests → [DataOps](../dataops/); who may see which rows → [DataSecOps](../datasecops/); which KPIs the company runs on → [BizOps](../bizops/); automated decisions → [DecisionOps](../decisionops/); model lifecycles → [MLOps](../mlops/). | | Prerequisites | SQL; basic dimensional modelling (facts, dimensions, grain); the [DataOps](../dataops/) page. | | Key outputs | Metric catalogue and cards; certified semantic models; change log; usage-based content review; conflict runbook. | ### The problem AnalyticsOps solves When analysis is decentralised, the same metric gets defined many times. Uber's metrics team wrote that "we often observed multiple versions of metrics defined or generated in different channels for the same business logic, which could be inaccurate or misleading". In one example, two internal tools showed "Shopping Sessions" for the same city and period as 6.53M and 6.20M; the cause was one tool's "stale filters in their static query".[^umetric] At Airbnb, when the CEO asked which city had the most bookings, "Data Science and Finance would sometimes provide diverging answers using slightly different tables, metric definitions, and business logic", and "trust from decision makers degraded".[^minerva] No pipeline needs to break for this. Copies of one definition drift apart, and correct data then yields conflicting numbers when a filter, time zone, grain, currency rule or refund treatment differs. The costs: meetings spent reconciling, decisions on the wrong number, and a growing estate of dashboards that is refreshed and paid for but rarely opened. | Operating reality | Before | After | |---|---|---| | Definition of "revenue" | Inside each dashboard's query | One versioned definition with an owner, reused by every tool | | Two numbers disagree | Escalation, side-by-side spreadsheets | Cards compared in minutes; one canonical metric | | Changing a definition | Silent edit; consumers notice later | Pull request, impact list, measured size, notice, back series | | Dashboard estate | Grows indefinitely | Reviewed on usage; archived after notice | | Trust and freshness | "Ask the analyst" | On the content: badge, owner, version, "data as of" | ### What AnalyticsOps is not: naming and boundaries - **Not DataOps renamed.** DataOps asks whether data arrives correct and on time; this page asks whether everyone uses one definition of each number. - **Not a product.** Buying a semantic layer does not decide what "active customer" means; people do. - **Not choosing KPIs.** That is [BizOps](../bizops/); this page governs how KPIs are defined, served and changed. - **Not a central report factory.** Tableau's governance guidance warns against "recreating a 'report factory' delivery model" and advises setting "goals and dates to evolve beyond the centralized governance model".[^tabgov] - **Not operational analytics.** Analysing operational data (logistics, IT events) is a use of analytics, not its operation. ## Where AnalyticsOps comes from and where it sits ### Origins and evolution The practice is older than the label. | When | Milestone | |---|---| | Before the label | Metrics defined once inside one BI tool, for example LookML models kept in Git[^lookml] | | 2019 | "Analytics engineer" named as a role using "version control and continuous integration"[^carroll] | | 2020–2021 | Tool-independent metric platforms: Airbnb's Midas (2020) and Minerva (2021); Uber's uMetric (2021)[^midas][^minerva][^umetric] | | 2021–2022 | The label appears for this meaning, from an analyst and a vendor[^menninger][^wiiisdom] | | 2023 | dbt Labs agrees to acquire Transform, "founded by former Airbnb data scientists", to merge MetricFlow into the dbt Semantic Layer[^transform] | | 2026 | Open Semantic Interchange, in the Apache Incubator, is renamed Apache Ossie (announced July);[^ossie] warehouses offer native semantic objects[^sfsv][^dbxmv] | ### Glossary and mental model | Term | Meaning on this page | |---|---| | **Simple metric (measure)** | An aggregation over one column; dbt's latest spec replaces "measures" with simple metrics.[^dbtspec] | | **Dimension, entity, grain** | An attribute to group or filter by; a join key (customer, order); what one row represents. | | **Additivity** | Whether values can be summed across a dimension. Revenue sums across days; daily distinct customers do not add up to a monthly count, and balances do not sum over time (dbt's `non_additive_dimension`).[^dbtmetrics] | | **Semantic layer** | Metric, dimension and join definitions that tools query instead of raw tables; it generates the SQL.[^mf] | | **Semantic model** | Power BI: the model behind reports; dbt: the semantic annotations on a model. | | **Definition card** | This page's term: a versioned, business-readable definition with owner, status and consumers. | | **Certification** | A reviewer's approval that content meets agreed standards, shown as a badge; tools differ in what can carry one ([table](#what-each-tool-can-certify)). | | **Back series, restatement** | Past values recomputed under a new definition, so charts show no step that is only a definition change; republishing reported numbers with the size of the change. | | **Exposure** | dbt's declaration that a dashboard or application depends on models or metrics.[^exposures] | | **Tier** | This page's criticality class: 1 board, external or regulated; 2 departmental; 3 team and self-service. | | **Definition drift** | One name whose definitions diverge across tools or over time. | | **Freshness promise** | When viewers can expect current data: "X% of data processed in Y".[^srewb] | **Mental model: a metric is an API between data producers and decision makers.** The semantic layer is the interface, the card the contract, dashboards and spreadsheets the clients; a change is compatible (a label) or breaking (a filter, grain or formula). ::: figure Where AnalyticsOps sits between data producers and decision makers ```mermaid flowchart TB BZ["BizOps: which KPIs"] --> DEF DO["DataOps: tested tables"] --> DEF DEF["Definitions with owner and version"] --> SL["Semantic layer"] DS["DataSecOps: access policies"] -.-> SL SL --> CC["Dashboards on certified models"] SL --> SS["Spreadsheets, notebooks"] SL --> AI["AI assistants"] SL --> DC["DecisionOps"] CC --> U["Decision makers"] SS --> U AI --> U classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029 class DEF,SL,CC core class BZ,DO,DS,SS,AI,DC,U neutral ``` Text equivalent: BizOps (which KPIs) and DataOps (tested tables) feed the definitions, which carry an owner and a version. The semantic layer serves them under DataSecOps's policies (dotted) to dashboards on certified models, spreadsheets and notebooks, AI assistants, and DecisionOps; the first three reach decision makers. Green: AnalyticsOps parts. Grey: neighbouring pages, consumers outside the governed path, and decision makers. ::: ### How AnalyticsOps relates to other Ops These splits are this Atlas's editorial boundaries, not industry definitions. | Neighbour | Shared scope | Its unique focus | AnalyticsOps focus | |---|---|---|---| | [DataOps](../dataops/) | Freshness, tests | Pipelines, data contracts, data incidents | Definitions, BI content, conflicting-number incidents | | [BizOps](../bizops/) | KPIs | Which KPIs, operating cadence | One governed definition per KPI and its dashboards | | [DecisionOps](../decisionops/) | Decisions that use metrics | Automated decision logic, audit | The metric definitions those decisions consume | | [DataSecOps](../datasecops/) | Access to data | Row and column policies, masking | Applying them consistently in the semantic layer and exports | | [ModelOps](../modelops/), [MLOps](../mlops/) | The older "AnalyticOps" meaning | Model lifecycle | Metrics; Uber's platform extends "a metric to a machine learning feature"[^umetric] | | [ProductOps](../productops/) | Product metrics | Insight-to-roadmap process | Definitions of activation, retention, engagement | ## Principles and operating model ### A proposed charter and principles No official AnalyticsOps manifesto exists; this charter is **this Atlas's proposal**. **Mission:** every number that drives a decision has one definition, one accountable owner, a known freshness and a visible trust status. 1. **Define once, serve everywhere.** Airbnb's stated vision for Minerva was "define metrics once, use them everywhere".[^minerva] 2. **Definitions are code, reviewed like code.** Minerva enforced "code review, static validation, and test runs";[^minerva] LookML projects are "typically version-controlled together through a Git repository".[^lookml] 3. **The business owns meaning; analytics owns encoding.** Finance decides what "net revenue" includes; an analytics engineer encodes and tests it. 4. **Discipline at the core, flexibility at the edge.** Microsoft's phrase for managed self-service.[^msown] 5. **Change like an API.** Planned changes get notice and a back series; errors are corrected promptly with their measured size, as the UK Code of Practice for Statistics asks of statistics producers.[^cop] 6. **Trust is visible on the content.** Badge, owner, version and freshness appear where people read the number. 7. **Use decides the estate.** What nobody views or receives is archived after notice, certified content included. **Non-goals:** one dashboard for everything; banning spreadsheets; certifying everything (certification is slow; see [Case 2](#case-2-one-metric-platform-plus-certification-airbnb-sourced)). ### Building blocks Seven building blocks recur: a catalogue of definition cards (find a metric before creating one), a semantic layer (compute it one way for every consumer), change control, certification, lineage for impact, usage and delivery telemetry, and freshness signals. The [tooling landscape](#tooling-landscape) maps them to products and [core practices](#core-practices-and-playbooks) to owners. ### Roles, ownership and RACI The **metric owner** (finance, product, sales) is accountable for meaning; the **analytics engineer** encodes and tests; the **BI content owner** builds on certified definitions; the **certifier** reviews (Airbnb designates "Data Architects or Metrics Architects";[^midas] see [what each tool can certify](#what-each-tool-can-certify)); the **platform admin** runs tenant settings, usage export and archiving; data engineers ([DataOps](../dataops/)) own the tables. R responsible, A accountable, C consulted, I informed. | Duty | Metric owner | Analytics engineer | BI content owner | Certifier | Platform admin | |---|---|---|---|---|---| | Define or change a tier-1 metric | A | R | C | C | I | | Certify a metric or content (R prepares; A approves) | C | R | R | A | I | | Approve a breaking definition change | A | R | C | C | I | | Publish a change notice | A | R | I | I | C | | Quarterly content review and archiving | I | C | R | C | A | | Respond to conflicting numbers | A | R | R | C | I | | Export and retain usage and delivery logs | I | I | I | I | R/A | ### Lifecycle of a metric Each state means something to consumers; it lives in the definition's metadata (a `status` key in [Lab 2](#hands-on-labs)). ::: figure Lifecycle states of a governed metric ```mermaid stateDiagram-v2 direction TB [*] --> Proposed Proposed --> Rejected: duplicate of an existing metric Proposed --> Draft: owner named, card written Draft --> Certified: review passed Certified --> Certified: reviewed change Certified --> Deprecated: successor and notice Deprecated --> Retired: notice period ends, no consumers left Rejected --> [*] Retired --> [*] ``` Text equivalent: A Proposed metric that duplicates an existing one is Rejected; otherwise it becomes a Draft once an owner and card exist, and Certified after review. A reviewed change creates a new Certified version. To remove it, the owner names a successor and sends notice (Deprecated); after the notice period, with no consumers left, it is Retired. ::: Dashboards follow a similar path: sandbox, promoted, certified, monitored, archived, deleted; in Tableau's self-governing model, viewers "understand the delineation between certified and ad-hoc, sandbox content states".[^tabgov] ### Operating model and governance - **Tiers set the controls.** Tier 1 (board, investor, regulatory, pricing): certified only, change notice, monthly reconciliation. Tier 2 (department): certified models, lighter review. Tier 3 (team): any model, labelled uncertified. - **Ceremonies.** Weekly triage; monthly certification review; quarterly content review (GitLab's cadence[^gitlab]); a governance review twice a year, as Tableau recommends.[^tabgov] - **Decision rights.** When two departments need different definitions under one name, a metrics council keeps one canonical metric and qualifies the other: "active customers (product, 28-day)". - **Self-service model.** Microsoft describes three content ownership strategies (business-led self-service, managed self-service and enterprise) that most organisations mix, and suggests reserving certification for enterprise and managed self-service content.[^msown] ::: decision Governance proportionality A 50-person company with one BI tool needs a catalogue of its 20 most-used metrics, an owner for each and a quarterly review. Add a semantic layer, a change gate and a metrics council when several tools, teams or external reports depend on the same numbers. ::: ## Architecture and workflow ### Reference architecture A **data plane** (tables, semantic layer, consumers) and a **control plane** (Git, CI, telemetry, review). ::: figure Reference architecture: definitions as code, one semantic layer, many consumers ```mermaid flowchart TB G["Git: definitions and cards"] --> CI["CI: parse, validate, change gate"] CI --> SL["Semantic layer runtime"] T["Tested tables from DataOps"] --> SL SL --> BI["Dashboards on certified models"] SL --> X["Spreadsheets, notebooks"] SL --> AI["AI assistants"] BI --> TM["Usage and freshness telemetry"] X --> TM AI --> TM TM --> R["Catalogue and content review"] R --> G classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029 classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029 class G,CI,R,TM decision class SL,BI core class T,X,AI neutral ``` Text equivalent: Definitions and cards live in Git; CI validates them and runs a change gate before they reach the semantic layer runtime, which reads tested tables from DataOps and serves dashboards on certified models, spreadsheets and notebooks, and AI assistants. Their usage and freshness telemetry feeds the catalogue and content review, whose decisions return to Git. Amber: control plane. Green: governed data plane. Grey: inputs and consumers outside the governed path (tested tables, spreadsheets and notebooks, AI assistants). ::: **Responsibilities.** Git keeps code and card together; CI rejects definitions that fail validation and metric or model-level changes without a version bump and change note ([Lab 2](#lab-2-intermediate-a-governed-metric-and-a-change-gate)); the semantic layer generates the SQL; telemetry answers "is anyone using this?" and "was the data on time?". Model SQL and upstream tables are [DataOps](../dataops/)'s to test. **Trust boundary.** Access rules live in the warehouse or the semantic layer (Cube: "Access control runs at the semantic layer, so the same policies apply to every consumer"[^cube]); direct connections and exports bypass the layer. Policy design belongs to [DataSecOps](../datasecops/). **Trade-offs.** With one BI tool, its own model is the cheapest semantic layer; a transformation-layer one reflects dbt Labs' view that "the metric definition is part of the modeling layer, rather than the business intelligence layer".[^transform] See the [decision framework](#decision-framework). ### End-to-end workflow: a metric change request ::: figure Metric request and change workflow with the breaking-change path ```mermaid flowchart TB RQ["Request: new or changed metric"] --> DUP{"Already in catalogue?"} DUP -- "Yes" --> REUSE["Point to existing metric; close"] DUP -- "No, or a change" --> CARD["Owner approves definition card"] CARD --> CI["Pull request; CI validates and lists impact"] CI --> BRK{"Breaking change?"} BRK -- "No" --> CERT["Review and certify"] BRK -- "Yes" --> NOTE["Version bump, measured size, notice, back series"] NOTE --> CERT CERT --> PUB["Publish, then monitor usage"] classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029 classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029 class RQ,CARD,CI,CERT,PUB,REUSE core class DUP,BRK decision class NOTE risk ``` Text equivalent: A request is checked against the catalogue; an existing metric is reused. Otherwise the owner approves a definition card, and code and card go into one pull request. CI validates, runs a change gate and lists affected dashboards. A breaking change first gets a version bump, a measured size, notice and a back series (red); then review, certification, publication and monitoring. ::: **Happy path (illustrative):** a tier-2 metric goes from request to certified in about a week. **Exception path:** a breaking tier-1 change waits out a notice period (this page suggests ten working days) and ships with its measured size and a back series. Do not skip the duplicate check: Uber reports that "there can be 10X or 100X instances spinned off for a popular metric".[^umetric] ## Practices, frameworks and tools ### Core practices and playbooks | Practice | Trigger | Owner | Output | Verification | |---|---|---|---|---| | Intake and duplicate check | Request for a new number | Analytics engineer | Reuse, rename or new card | No conflict ([Lab 1](#hands-on-labs)) | | Definition cards | New or changed metric | Metric owner | Formula, filters, grain, additivity, time zone, version | Owner sign-off | | Certification | Tier 1 or 2 content | Certifier | Badge and its meaning | Criteria checklist; sample audit | | Change management | Certified definition changes | Metric owner | Version, measured size, notice, back series | Change gate passes | | Freshness signalling | Every refresh | Content owner | "Data as of", warnings | Freshness attainment | | Content review | Quarterly | Platform admin | Archive list, new owners | Active-content ratio | | Conflict response | Two numbers, one metric | Metric owner | Explanation, canonical metric | [Runbook](#worked-runbook-two-numbers-for-one-metric) closed | | Self-service enablement | New creators | Analytics lead | Templates, training[^msown] | Team content on certified models | #### Playbook: certifying a metric 1. **Card.** The owner writes the definition in business language ([template](#templates)); Midas design specs likewise start with "plain-text business definitions".[^midas] 2. **Code and validate.** The analytics engineer encodes the metric with owner, tier, status and version metadata, and CI validates it ([Lab 2](#hands-on-labs)). A tier-1 metric also gets an assertion test or a reconciliation with its system of record. 3. **Review.** A certifier who is not the author checks card, code, tests and consumers; one of Midas's four reviews covers "the source of truth metric definitions implemented in Minerva".[^midas] 4. **Publish.** Badge, owner, version and consumers visible. **Rollback:** revert the pull request; the previous version stays. ### Methods, standards and frameworks There is no AnalyticsOps standard. Separate what binds from what helps: - **Statutory only in its domain.** Producers of UK official statistics "should apply" the *Code of Practice for Statistics* (Edition 3.0) under section 10 of the 2007 Act; accredited official statistics are "required under the Act to continually meet the standards of the Code". Its principles are offered "for analysts and organisations both inside and outside of government", and map onto metric governance: correct errors transparently, keep data "consistent over time", announce planned method changes with "a consistent back series where possible", flag statistics "being developed and tested".[^cop] - **Specifications.** Apache Ossie (formerly Open Semantic Interchange), "an open specification for both semantic layer and ontology", now in the Apache Incubator;[^ossie] MetricFlow's YAML spec for dbt.[^mf] - **Vendor methodologies.** Tableau Blueprint's centralized, delegated and self-governing models;[^tabgov] Microsoft's Fabric adoption roadmap.[^msown][^msmat] - **Reliability practice.** Freshness and correctness objectives from the SRE Workbook.[^srewb] - **A design constraint.** "When a measure becomes a target, it ceases to be a good measure" (Strathern, on what Hoskin called Goodhart's law).[^strathern] ### Tooling landscape Job first, product second. Representative options as of October 2026; plan limits change often. | Job | Tool category | Representative options | Watch-outs | |---|---|---|---| | Define metrics as code | Transformation-layer semantic layer | dbt Semantic Layer with MetricFlow (Apache 2.0 engine)[^mf][^mflicense] | Downstream tools need a dbt Starter, Enterprise or Enterprise+ account;[^dbtmfcmd] no cross-project references yet[^dbtspec] | | | Standalone semantic layer | Cube (open-source Cube Core)[^cube] | Another runtime to operate and secure | | | Warehouse-native | Snowflake semantic views; Databricks metric views[^sfsv][^dbxmv] | Ties definitions to one warehouse | | | BI-native model | LookML; Power BI semantic models as PBIP (Power BI Project) text files; Tableau published data sources[^lookml][^pbip] | Other tools cannot reuse them | | Validate and deploy | CI and release tooling | `dbt sl validate`, `mf validate-configs`;[^dbtvalid] Looker Content Validator;[^lkcv] Fabric deployment pipelines | The Content Validator "does not have an undo function"; each pipeline stage needs a Premium workspace[^deploystart] | | Certify and signal | Badges and warnings | Certification ([table below](#what-each-tool-can-certify)); Tableau data quality warnings[^tabdqw] | Warnings need Tableau Data Management | | Measure usage and retire | Usage telemetry, cleanup | Power BI usage metrics and activity log;[^pbiusage][^pbiactlog] Tableau Admin Insights;[^tabinsights] Looker Unused content;[^lkunused] scripts on admin APIs[^gitlab] | Short retention; deliveries counted differently (below) | | Impact analysis | Lineage | dbt exposures;[^exposures] BI-native lineage | Undeclared consumers (exports, direct SQL) stay invisible | **Usage data (as of October 2026).** The Power BI activity log keeps "a maximum of 28 days", usage metrics 30;[^pbiactlog][^pbiusage] the Microsoft Purview audit log keeps Fabric and Power BI activity for 180 days under Audit (Standard) but needs an audit role and another API.[^fabaudit][^purview] Tableau Admin Insights keeps 90 days, 365 with Advanced Management.[^tabinsights] Power BI logs a subscription snapshot as a report view; Tableau lists subscriptions in a separate data source. Looker's Unused content folder (Looker 26.14 or later; Looker (original) must be Looker-hosted) counts an active schedule as use and does not list public, favourited or API-used content (such as Connected Sheets) as unused.[^pbiusage][^tabinsights][^lkunused] Export usage and delivery data daily. #### What each tool can certify As of October 2026: | Tool | Certified badge on | Dashboards and reports | Who certifies | |---|---|---|---| | Tableau | Published data sources; with Data Management, virtual connections; with Catalog too, databases, tables, files[^tabcert] | No badge: Blueprint certifies workbooks by promoting them to a production project[^tabgovin] | Administrators; project owners or leaders who can publish | | Fabric, Power BI | "Any Fabric or Power BI item except Power BI dashboards"[^endorse] | Reports and semantic models yes; dashboards no | Users a Fabric administrator specifies | | Looker | Dashboards, LookML dashboards, Looks, Explores, self-service Explores[^lkcert] | Yes; no badge on signed embedded content since Looker 26.4 | Admins or holders of `certify_content` | "Certified content" here means a badge where the tool has one, otherwise content in a certified project or built only on certified models or data sources. ### Automation and AI **Automation that clearly helps:** CI validation; the change gate ([Lab 2](#hands-on-labs)); usage-based archiving with owner notice (GitLab's script; Looker's scheduled cleanups);[^gitlab][^lkunused] warnings when a refresh fails.[^tabdqw] Each needs an owner and a soft-delete path. **AI assistants** are a new kind of consumer; writing SQL against raw tables, they inherit every ambiguity. In a 2023 benchmark, GPT-4 answered enterprise questions over an SQL schema with 16% accuracy, and 54% over a knowledge-graph representation of the same database:[^sequeda] a direction, not a planning number (one benchmark, domain and model; authors from data.world, a catalogue vendor built on a knowledge graph;[^dataworld] its context layer was an ontology with mappings, not a semantic layer, so in this page's judgement the gain may not carry over). Cube's agents query its semantic layer, not the warehouse.[^cube] Snowflake semantic views work in Cortex Agents, but the agent "generates the SQL against the physical tables directly",[^sfsv] so policies on those tables, not the view, decide what it can return. **This page's rules:** tier-1 answers use certified metrics and show the definition and version used; queries run with the asking user's permissions; answers are logged. Operating the assistant belongs to [LLMOps](../llmops/). ## AnalyticsOps in practice ### Reference implementation **Assumptions (illustrative):** dbt on a cloud warehouse, one main BI tool, many spreadsheets; finance owns revenue metrics. Each definition, its card and its consumers live in one repository, and CI gates changes. ```text analytics/ models/marts/fct_orders.sql models/marts/fct_orders.yml # semantic model + certified metrics (Lab 2) models/exposures.yml # dashboards that depend on metrics metrics/cards/net_revenue.md # definition card in business language ci/metric_change_gate.py # Lab 2 ci/metric_delta.py # size of a change (Lab 2) ci/profiles.yml # CI-only profile: local DuckDB file, no warehouse ops/content_review.py # Lab 3 CHANGELOG-metrics.md # human-readable change notices ``` The CI workflow parses the project, runs MetricFlow validations without a warehouse (`--skip-dw`, a dbt v1 option[^dbtmfcmd]) and runs the change gate against the base branch, with actions pinned to commit SHAs. Both tools find the CI-only profile through `DBT_PROFILES_DIR`: `mf` has no `--profiles-dir` flag, otherwise looks only in the current directory, and does not read dbt's newer `DBT_ENGINE_PROFILES_DIR`.[^mfprofiles] _Example status: workflow and profile YAML syntax-checked; the gate step's shell logic run in a local Git simulation (new file, changed file, missing base branch), 2026-10-17; neither executed (needs a dbt project and GitHub Actions). Versions checked on PyPI 2026-10-17: dbt-core 1.12.6 (Python 3.10+), dbt-metricflow 0.15.0 (dbt-core 1.11–1.12), dbt-duckdb 1.11.0, PyYAML 6.0.3._ ```yaml name: metric-ci on: pull_request: # no paths filter: a required check skipped by a filter stays "Pending" permissions: contents: read jobs: validate-metrics: runs-on: ubuntu-24.04 env: DBT_PROFILES_DIR: ci # read by dbt and mf (mf has no --profiles-dir flag) steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: fetch-depth: 0 persist-credentials: false # keep the job token out of .git/config - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 with: python-version: "3.12" - name: Install pinned tooling run: >- python -m pip install "dbt-core==1.12.6" "dbt-duckdb==1.11.0" "dbt-metricflow==0.15.0" "pyyaml==6.0.3" - name: Parse project (writes semantic_manifest.json) run: dbt parse - name: Semantic validation without a warehouse run: mf validate-configs --skip-dw - name: Metric change gate against the base branch env: BASE_REF: ${{ github.base_ref }} FILE: models/marts/fct_orders.yml run: | # no base branch: this line fails the step (the default shell is bash -e) git rev-parse --verify --quiet "origin/${BASE_REF}" > /dev/null if git cat-file -e "origin/${BASE_REF}:${FILE}" 2> /dev/null; then git show "origin/${BASE_REF}:${FILE}" > "${RUNNER_TEMP}/base.yml" python ci/metric_change_gate.py "${RUNNER_TEMP}/base.yml" "${FILE}" else echo "No ${FILE} on ${BASE_REF}: new file, gate skipped" fi ``` ```yaml # ci/profiles.yml: the top-level key must match `profile:` in dbt_project.yml analytics: target: ci outputs: ci: type: duckdb path: ci.duckdb # local file: no warehouse, no credentials ``` **Who can reach the token.** It is read-only (`contents: read`), and `persist-credentials: false` keeps it out of `.git/config`, where `actions/checkout` stores it by default,[^checkout] so the packages that later steps install cannot read it there. The base branch reaches the script through an environment variable, against script injection. No `paths` filter: GitHub leaves checks of a workflow skipped by a path filter "Pending", which blocks pull requests that require them.[^ghskip] ### Hands-on labs #### Lab 1 (beginner): find conflicting definitions in your dashboards **Objective:** find names with several definitions, and definitions under several names. **Prerequisites:** Python 3, standard library only (use a supported release; tested with 3.8.10); an inventory of tiles on your 10–20 most-viewed dashboards (name, aggregation, measure, source, filters, time zone). No cost or risk; about two hours. 1. Save the inventory below as `metric-inventory.csv` (illustrative data; filters joined with `&`) and the script as `metric_conflicts.py`. 2. Run `python3 metric_conflicts.py metric-inventory.csv; echo $?`. 3. Replace the sample with your own inventory and run again. ```text dashboard,metric_name,agg,measure,source,filters,time_zone,owner Exec,Revenue,sum,amount,fct_orders,status=completed,UTC,fin-bi Pipeline,Revenue,sum,amount,fct_orders,status in completed+refunded,UTC,sales-ops Marketing,Net Revenue,sum,amount_net,fct_orders,status=completed&is_test=0,UTC,mkt Finance,net_revenue,sum,amount_net,fct_orders,is_test=0&status=completed,UTC,fin-bi Exec,Active customers,count_distinct,customer_id,fct_orders,ordered_in=30d,UTC,fin-bi Product,Active Customers,count_distinct,customer_id,fct_sessions,seen_in=28d,UTC,prod-an Product,MAU,count_distinct,user_id,fct_sessions,seen_in=30d,UTC,prod-an Growth,Monthly active users,count_distinct,user_id,fct_sessions,seen_in=30d,UTC,growth Support,Tickets opened,count,ticket_id,fct_tickets,,Europe/Berlin,support-ops Exec,Tickets opened,count,ticket_id,fct_tickets,,UTC,fin-bi ``` _Example status: verified by execution (Python 3.8.10, 2026-10-17); the output below is the actual output._ ```python """Find conflicting metric definitions in a dashboard inventory. Usage: python3 metric_conflicts.py metric-inventory.csv Standard library only. Reports (1) one name with several definitions and (2) one definition published under several names. Exit code 1 on (1). """ import csv import re import sys from collections import defaultdict def norm_name(name): return re.sub(r"[\s_\-]+", " ", name.strip().lower()) def signature(row): filters = sorted(f.strip() for f in row["filters"].split("&") if f.strip()) parts = [row["agg"], row["measure"], row["source"], "&".join(filters) or "-", row["time_zone"]] return " | ".join(p.strip().lower() for p in parts) def main(path): with open(path, newline="", encoding="utf-8") as fh: rows = list(csv.DictReader(fh)) by_name = defaultdict(lambda: defaultdict(list)) by_sig = defaultdict(set) for r in rows: sig = signature(r) where = f"{r['dashboard']} ({r['owner']})" by_name[norm_name(r["metric_name"])][sig].append(where) by_sig[sig].add(norm_name(r["metric_name"])) conflicts = {n: s for n, s in by_name.items() if len(s) > 1} synonyms = {s: n for s, n in by_sig.items() if len(n) > 1} print(f"{len(rows)} tiles, {len(by_name)} metric names, {len(by_sig)} definitions") print(f"SAME NAME, DIFFERENT DEFINITIONS: {len(conflicts)}") for name, sigs in sorted(conflicts.items()): print(f" '{name}'") for sig, places in sigs.items(): print(f" {sig}\n used in: {'; '.join(places)}") print(f"SAME DEFINITION, DIFFERENT NAMES: {len(synonyms)}") for sig, names in sorted(synonyms.items()): print(f" {' = '.join(sorted(names))}\n {sig}") return 1 if conflicts else 0 if __name__ == "__main__": if len(sys.argv) != 2: sys.exit("usage: python3 metric_conflicts.py ") sys.exit(main(sys.argv[1])) ``` ```text $ python3 metric_conflicts.py metric-inventory.csv; echo $? 10 tiles, 6 metric names, 8 definitions SAME NAME, DIFFERENT DEFINITIONS: 3 'active customers' count_distinct | customer_id | fct_orders | ordered_in=30d | utc used in: Exec (fin-bi) count_distinct | customer_id | fct_sessions | seen_in=28d | utc used in: Product (prod-an) 'revenue' sum | amount | fct_orders | status=completed | utc used in: Exec (fin-bi) sum | amount | fct_orders | status in completed+refunded | utc used in: Pipeline (sales-ops) 'tickets opened' count | ticket_id | fct_tickets | - | europe/berlin used in: Support (support-ops) count | ticket_id | fct_tickets | - | utc used in: Exec (fin-bi) SAME DEFINITION, DIFFERENT NAMES: 1 mau = monthly active users count_distinct | user_id | fct_sessions | seen_in=30d | utc 1 ``` **Expected output:** as above; exit code 1. "Net Revenue" and "net_revenue" are not flagged: names are normalised and filter order ignored. **Validation:** each conflict ends as one canonical metric or two qualified names ("Revenue (bookings, incl. refunded)"). **Troubleshooting:** `KeyError` means a misspelt header; quote fields that contain commas. **Teardown:** delete the files, or keep the inventory as your first catalogue. #### Lab 2 (intermediate): a governed metric and a change gate **Objective:** define certified metrics in dbt's latest YAML spec with a dependent dashboard; block breaking changes, model-level ones included, that lack a version bump and change note; measure a change. **Prerequisites:** Python 3 with PyYAML; optionally a dbt v1.12+ project (Python 3.10+) and a warehouse. No cost or risk; about an hour. 1. Save the YAML below as `base.yml` and the gate as `metric_change_gate.py`. 2. Copy `base.yml` to `head.yml`; in `head.yml` change the `gross_revenue` filter to `{{ Dimension('order__status') }} in ('completed', 'partially_refunded')` and the `refunds` label to `Refunds (EUR)`. 3. Run `python3 metric_change_gate.py base.yml head.yml; echo $?` (expected: blocked). 4. Copy `head.yml` to `head-fixed.yml`; there, set `gross_revenue`'s `definition_version` to 4 and add `change_note: "2026-11-01: include partially refunded orders (FIN-2041)"`; run the gate on it (expected: ok). 5. Save `base.yml` without the top-level `refund_rate` metric as `head-removed.yml`; run the gate (expected: blocked). 6. Save `base.yml` as `head-time.yml` with the model's `agg_time_dimension` set to `shipped_at` and a `shipped_at` time column added; run the gate (expected: blocked). Adding a dimension alone passes. 7. Optional, in a real dbt project: `dbt parse`, then `mf validate-configs --skip-dw` (self-hosted) or `dbt sl validate`.[^dbtvalid] _Example status: YAML parsed with PyYAML; written to the dbt v1.12+ spec as documented;[^dbtspec][^dbtmetrics] not validated with dbt or MetricFlow (not installed in the test environment)._ ```yaml models: - name: fct_orders description: One row per order; refunds netted per order. semantic_model: enabled: true name: orders agg_time_dimension: ordered_at columns: - name: order_id entity: type: primary name: order - name: customer_id entity: type: foreign name: customer - name: ordered_at granularity: day dimension: type: time - name: status dimension: type: categorical - name: is_test dimension: type: categorical metrics: - name: gross_revenue label: Gross revenue description: > Sum of order amounts (EUR, excl. VAT) for completed, non-test orders, dated by order date in UTC. type: simple agg: sum expr: amount_eur_net_of_vat filter: | {{ Dimension('order__status') }} = 'completed' and {{ Dimension('order__is_test') }} = false config: meta: owner: finance-bi tier: 1 status: certified definition_version: 3 - name: refunds label: Refunds description: Refunded amount (EUR, excl. VAT) by order date. type: simple agg: sum expr: refund_amount_eur config: meta: owner: finance-bi tier: 1 status: certified definition_version: 1 - name: net_revenue label: Net revenue description: Gross revenue minus refunds. type: derived expr: gross_revenue - refunds config: meta: owner: finance-bi tier: 1 status: certified definition_version: 2 metrics: - name: refund_rate label: Refund rate description: Refunds as a share of gross revenue. type: ratio numerator: refunds denominator: gross_revenue config: meta: owner: finance-bi tier: 2 status: certified definition_version: 1 exposures: - name: exec_weekly_revenue label: Executive weekly revenue type: dashboard maturity: high url: https://bi.example.com/dashboards/42 depends_on: - metric('net_revenue') - metric('refund_rate') owner: name: Finance BI email: finance-bi@example.com ``` The `config.meta` keys are this page's convention; `meta` "accepts any key-value pairs".[^dbtmeta] Exposures usually live in their own file. _Example status: verified by execution (Python 3.8.10, PyYAML 5.3.1, 2026-10-17); the outputs below are actual. The reviewer's primary-entity change (`order_id` to `order_line_id`) is also blocked._ ```python """Gate metric-definition changes in a pull request (dbt latest YAML spec). Usage: python3 metric_change_gate.py base.yml head.yml Needs PyYAML. A changed definition must raise meta.definition_version and add meta.change_note; a removed metric must have been deprecated first; a new metric needs owner, tier and status. A changed or removed model-level key (agg_time_dimension, semantic model, entity or dimension) redefines every metric in that model. Prints the metrics and dashboards (exposures) affected. Exit code 1 on any violation. It does not see model SQL or upstream tables. """ import re import sys import yaml DEFINITION_KEYS = ("type", "agg", "expr", "filter", "numerator", "denominator", "input_metric", "input_metrics", "window", "grain_to_date", "agg_time_dimension", "non_additive_dimension", "fill_nulls_with", "join_to_timespine", "base_metric", "conversion_metric") REQUIRED_META = ("owner", "tier", "status") def model_keys(model): """Model-level semantics: a change or removal here alters every metric.""" keys = {"agg_time_dimension": model.get("agg_time_dimension"), "semantic_model": model.get("semantic_model")} for col in model.get("columns") or []: for kind in ("entity", "dimension", "granularity"): if col.get(kind) is not None: keys[f"{col['name']}.{kind}"] = col[kind] return keys def load(path): with open(path, encoding="utf-8") as fh: doc = yaml.safe_load(fh) or {} metrics, models = {}, {} for model in doc.get("models") or []: models[model["name"]] = model_keys(model) for m in model.get("metrics") or []: metrics[m["name"]] = dict(m, _model=model["name"]) for m in doc.get("metrics") or []: metrics[m["name"]] = dict(m, _model="(top level)") exposures = {e["name"]: [re.sub(r"^metric\('(.+)'\)$", r"\1", d) for d in e.get("depends_on") or []] for e in doc.get("exposures") or []} return metrics, models, exposures def meta(metric): return (metric.get("config") or {}).get("meta") or {} def definition(metric): return {k: metric.get(k) for k in DEFINITION_KEYS + ("_model",)} def inputs(metric): found = set() for key in ("numerator", "denominator", "input_metric", "base_metric", "conversion_metric"): val = metric.get(key) if val: found.add(val["name"] if isinstance(val, dict) else val) for item in metric.get("input_metrics") or []: found.add(item["name"]) if metric.get("type") == "derived": found |= set(re.findall(r"[a-z_][a-z0-9_]*", metric.get("expr", ""))) return found def downstream(changed, metrics, exposures): affected, frontier = set(changed), set(changed) while frontier: frontier = {n for n, m in metrics.items() if inputs(m) & frontier and n not in affected} affected |= frontier dashboards = sorted(e for e, deps in exposures.items() if set(deps) & affected) return sorted(affected - set(changed)), dashboards def changed_models(base_models, head_models): """Models whose existing semantic keys changed or vanished (additions pass).""" hit = set() for name, old in base_models.items(): new = head_models.get(name, {}) diff = sorted(k for k in old if old[k] != new.get(k)) if diff: print(f"MODEL {name}: {', '.join(diff)} changed or removed") hit.add(name) return hit def main(base_path, head_path): base, base_models, _ = load(base_path) head, head_models, exposures = load(head_path) models_hit = changed_models(base_models, head_models) problems, changed = [], [] for name in sorted(set(base) | set(head)): old, new = base.get(name), head.get(name) if old is None: missing = [k for k in REQUIRED_META if k not in meta(new)] print(f"ADDED {name}") if missing: problems.append(f"{name}: new metric lacks meta {missing}") elif new is None: print(f"REMOVED {name}") changed.append(name) if meta(old).get("status") != "deprecated": problems.append(f"{name}: removed without being deprecated first") elif definition(old) != definition(new) or old["_model"] in models_hit: print(f"REDEFINED {name}") changed.append(name) v_old = int(meta(old).get("definition_version", 0)) v_new = int(meta(new).get("definition_version", 0)) if v_new <= v_old or not meta(new).get("change_note"): problems.append(f"{name}: definition changed: bump definition_version" f" (now {v_new}), add change_note") elif old != new: print(f"COSMETIC {name} (label, description or meta only)") if changed: metrics_hit, dashboards = downstream(changed, head, exposures) print(f"Affected metrics: {', '.join(metrics_hit) or 'none'}") print(f"Affected dashboards: {', '.join(dashboards) or 'none'}") for p in problems: print(f"FAIL {p}") print("RESULT:", "blocked" if problems else "ok") return 1 if problems else 0 if __name__ == "__main__": if len(sys.argv) != 3: sys.exit("usage: python3 metric_change_gate.py ") sys.exit(main(sys.argv[1], sys.argv[2])) ``` ```text $ python3 metric_change_gate.py base.yml head.yml; echo $? REDEFINED gross_revenue COSMETIC refunds (label, description or meta only) Affected metrics: net_revenue, refund_rate Affected dashboards: exec_weekly_revenue FAIL gross_revenue: definition changed: bump definition_version (now 3), add change_note RESULT: blocked 1 $ python3 metric_change_gate.py base.yml head-fixed.yml; echo $? REDEFINED gross_revenue COSMETIC refunds (label, description or meta only) Affected metrics: net_revenue, refund_rate Affected dashboards: exec_weekly_revenue RESULT: ok 0 $ python3 metric_change_gate.py base.yml head-removed.yml; echo $? REMOVED refund_rate Affected metrics: none Affected dashboards: exec_weekly_revenue FAIL refund_rate: removed without being deprecated first RESULT: blocked 1 $ python3 metric_change_gate.py base.yml head-time.yml; echo $? MODEL fct_orders: agg_time_dimension changed or removed REDEFINED gross_revenue REDEFINED net_revenue REDEFINED refunds Affected metrics: refund_rate Affected dashboards: exec_weekly_revenue FAIL gross_revenue: definition changed: bump definition_version (now 3), add change_note FAIL net_revenue: definition changed: bump definition_version (now 2), add change_note FAIL refunds: definition changed: bump definition_version (now 1), add change_note RESULT: blocked 1 ``` **Expected output:** as above. A one-word filter change to `gross_revenue` also changes `net_revenue`, `refund_rate` and the executive dashboard; a new default time dimension changes every metric of the model. Only redefined metrics need a version bump, because `definition_version` versions a metric's own definition (this page's convention); dependants' values still change, so the notice names them and their cards' change logs cite `gross_revenue` v4. **Validation:** the impact list names every dependent exposure; their owners get the [notice](#templates). **Troubleshooting:** `No module named 'yaml'`: install PyYAML; `KeyError: 'name'`: a metric or exposure lacks a name. **Teardown:** delete the files. **Limits:** the gate reads one file, sees only consumers declared as exposures, and not the model's SQL or upstream tables, which [DataOps](../dataops/) tests. **Measure the size of the change.** The notice needs a number: run the affected metrics under both definitions on the same data and window, then compare. _Example status: the `mf query` commands follow the documented syntax (`--csv` exists only in self-hosted `mf`, not in `dbt sl`)[^dbtmfcmd] and were not executed (no warehouse); `metric_delta.py` was verified by execution (Python 3.8.10, 2026-10-17) on the illustrative CSV files shown._ ```bash git switch main && dbt parse mf query --metrics gross_revenue,net_revenue --group-by metric_time__month \ --start-time 2026-07-01 --end-time 2026-09-30 --csv base.csv git switch fin-2041 && dbt parse mf query --metrics gross_revenue,net_revenue --group-by metric_time__month \ --start-time 2026-07-01 --end-time 2026-09-30 --csv head.csv python3 metric_delta.py base.csv head.csv ``` ```text base.csv metric_time__month,gross_revenue,net_revenue 2026-07-01,1204000.00,1150300.00 2026-08-01,1187500.00,1131900.00 2026-09-01,1250200.00,1192800.00 head.csv metric_time__month,gross_revenue,net_revenue 2026-07-01,1228100.00,1174400.00 2026-08-01,1214800.00,1159200.00 2026-09-01,1282700.00,1225300.00 ``` ```python """Size of a metric-definition change: compare base and head query results. Usage: python3 metric_delta.py base.csv head.csv Standard library only. Each CSV has the period in its first column and one column per metric (for example from `mf query ... --csv`), run on the same data and window. Prints each metric's total change and its largest period change. """ import csv import sys def read(path): with open(path, newline="", encoding="utf-8") as fh: rows = list(csv.reader(fh)) header = [h.strip().lower() for h in rows[0]] return {r[0]: dict(zip(header[1:], map(float, r[1:]))) for r in rows[1:]}, header[1:] def pct(old, new): return (new - old) / old * 100 if old else float("nan") def main(base_path, head_path): base, metrics = read(base_path) head, _ = read(head_path) periods = sorted(set(base) & set(head)) print(f"{len(periods)} periods compared: {periods[0]} to {periods[-1]}") for m in metrics: old = sum(base[p][m] for p in periods) new = sum(head[p][m] for p in periods) worst = max(periods, key=lambda p: abs(pct(base[p][m], head[p][m]))) print(f"{m:<14} {old:>14,.2f} -> {new:>14,.2f} {pct(old, new):+.1f}%" f" (largest: {worst} {pct(base[worst][m], head[worst][m]):+.1f}%)") return 0 if __name__ == "__main__": if len(sys.argv) != 3: sys.exit("usage: python3 metric_delta.py ") sys.exit(main(sys.argv[1], sys.argv[2])) ``` ```text $ python3 metric_delta.py base.csv head.csv 3 periods compared: 2026-07-01 to 2026-09-01 gross_revenue 3,641,700.00 -> 3,725,600.00 +2.3% (largest: 2026-09-01 +2.6%) net_revenue 3,475,000.00 -> 3,558,900.00 +2.4% (largest: 2026-09-01 +2.7%) ``` Put the total (+2.3%) on the notice's Size line; the head run's monthly values are the back series. Both runs must read the same snapshot, or a late partition looks like a change. For tier-1 values also keep tests that run on every change: a LookML `test` whose `assert` checks a known historical value,[^lktest] or a reconciliation with the ledger for a closed month. #### Lab 3 (advanced): a usage-based content review **Objective:** a quarterly worklist of stale, orphaned and certified-but-unused content, plus the active-content ratio. **Prerequisites:** Python 3, standard library only; 90 days of usage data plus each item's active deliveries (retention and counting rules: [tooling landscape](#tooling-landscape)). **Risk:** archiving is visible; notify first and soft-delete only. **Status:** script executed; archiving steps not executed. 1. Export one row per item: owner, owner still active, certified, created, last viewed, viewers in 90 days, active deliveries (schedules, subscriptions, embeds or API pulls you can count), keep flag (GitLab uses a "Do NOT Archive" tag).[^gitlab] 2. Run the review with the export date. 3. Notify owners (14 days to object); archive by soft delete; reassign orphans; send certified-but-unused items to the certifier. 4. Record the ratio for the trend. ```text item,owner,owner_active,certified,created,last_viewed,viewers_90d,active_deliveries,keep Exec weekly revenue,fin-bi,Y,Y,2024-02-01,2026-10-16,48,0,N Board pack KPIs,fin-bi,Y,Y,2023-06-12,2026-07-02,0,1,N Revenue by region (copy),a.lee,N,N,2025-03-03,2026-01-20,0,0,N Pipeline coverage,sales-ops,Y,Y,2024-09-09,2026-10-15,31,0,N Pipeline coverage v2,sales-ops,Y,N,2025-11-20,2026-10-10,2,0,N Churn deep dive,prod-an,Y,N,2025-01-15,2026-04-30,0,0,N Support backlog,support-ops,Y,Y,2024-05-05,2026-10-17,22,0,N Marketing ROI,mkt,Y,N,2025-08-01,2026-09-30,6,0,N Q3 campaign test,j.diaz,N,N,2025-07-14,2025-08-02,0,0,N Active customers trend,prod-an,Y,Y,2024-03-18,2026-06-01,0,0,N Year-end close pack,fin-bi,Y,Y,2023-01-10,2026-01-15,0,0,Y Ops weekly (old),ops,Y,N,2022-10-10,,0,0,N New pricing monitor,fin-bi,Y,N,2026-09-01,2026-10-12,9,0,N ``` _Example status: verified by execution (Python 3.8.10, 2026-10-17); the output below is the actual output._ ```python """Quarterly BI content review: find stale, orphaned and unused certified content. Usage: python3 content_review.py content-usage.csv 2026-10-17 [window_days] Standard library only. Input: one row per dashboard or report, exported from the BI tool's usage data. An item counts as used if it was viewed in the window or has active deliveries (schedules, subscriptions, embeds or API pulls). Prints the active-content ratio and a worklist. """ import csv import sys from datetime import date, timedelta def day(text): return date.fromisoformat(text) if text else None def review(row, as_of, window): start = as_of - timedelta(days=window) created, seen = day(row["created"]), day(row["last_viewed"]) viewed = seen is not None and seen >= start used = viewed or int(row["active_deliveries"]) > 0 new = created > start actions = [] if row["owner_active"] != "Y": actions.append("reassign owner") if new: return used, new, actions if not used: if row["keep"] == "Y": actions.append("confirm keep exception") elif row["certified"] == "Y": actions.append("decertify or retire (certified, unused)") else: actions.append("archive after 14-day notice") elif not viewed: actions.append("delivered only: ask recipients if still needed") elif row["certified"] != "Y" and int(row["viewers_90d"]) <= 2: actions.append("merge or make personal (reach <= 2)") return used, new, actions def main(path, as_of_text, window=90): as_of = date.fromisoformat(as_of_text) with open(path, newline="", encoding="utf-8") as fh: rows = list(csv.DictReader(fh)) mature = active = 0 worklist = [] for r in rows: used, new, actions = review(r, as_of, window) if not new: mature += 1 active += used if actions: worklist.append((r["item"], r["owner"], "; ".join(actions))) print(f"{len(rows)} items; {mature} older than {window} days; {active} of them" f" used (viewed or delivered) in the last {window} days") print(f"Active-content ratio: {active / mature:.0%}") print("WORKLIST") for item, owner, action in worklist: print(f" {item:<24} {owner:<11} {action}") return 0 if __name__ == "__main__": if len(sys.argv) not in (3, 4): sys.exit("usage: python3 content_review.py [days]") sys.exit(main(sys.argv[1], sys.argv[2], *(int(a) for a in sys.argv[3:]))) ``` ```text $ python3 content_review.py content-usage.csv 2026-10-17 13 items; 12 older than 90 days; 6 of them used (viewed or delivered) in the last 90 days Active-content ratio: 50% WORKLIST Board pack KPIs fin-bi delivered only: ask recipients if still needed Revenue by region (copy) a.lee reassign owner; archive after 14-day notice Pipeline coverage v2 sales-ops merge or make personal (reach <= 2) Churn deep dive prod-an archive after 14-day notice Q3 campaign test j.diaz reassign owner; archive after 14-day notice Active customers trend prod-an decertify or retire (certified, unused) Year-end close pack fin-bi confirm keep exception Ops weekly (old) ops archive after 14-day notice ``` **Expected output:** as above. "New pricing monitor" is younger than the window; "Board pack KPIs" is e-mailed, not opened, so it counts as used and its owner asks the recipients. **Validation:** with a 180-day window (add `180` as a third argument) the ratio rises to 67% (8 of 12), so always report the window. **Troubleshooting:** `Invalid isoformat string` means a date is not `YYYY-MM-DD`; a ratio that jumps between quarters often means lost export days. **Teardown:** delete the files; restore late objections from the archive. ### Use cases and scenarios All three come from the organisations' own publications, without independent measurement of outcomes. #### Case 1: standardising metric definitions across teams (Uber, sourced) **Problem and context.** Across "30+ teams and 10K+ metrics", "completed trips" existed as Presto/Hive SQL for an operations team and as a Cassandra-based metric for Pricing Engineering.[^umetric] **Method.** uMetric separates the metric definition from dimensions and filters supplied at query time, so "Completed Trips In New York" is not a separate metric; heuristic SQL canonicalisation detects duplicates. **Trade-offs.** A platform team and onboarding effort; Uber described itself as "still at the early stages of the journey". **Success criteria.** New metrics unambiguous and not duplicated. **Pitfall targeted:** per-filter copies whose names "do not capture the complete details of the dimensions and filters".[^umetric] #### Case 2: one metric platform plus certification (Airbnb, sourced) **Problem.** Diverging answers from Data Science and Finance. **Method.** Minerva held "more than 12,000 metrics and 4,000 dimensions" (2021) in a central GitHub repository, serving dashboards, experiments and notebooks from the same definitions, with "certification status, ownership, and popularity" in the catalogue.[^minerva] Midas certification adds design specs, validation and four reviews, one of them of the metric definitions.[^midas] **Trade-offs.** Building to Midas standards "is much slower than building uncertified data".[^midas] **Success criteria.** One answer per question in every tool. **Pitfall.** Certification becomes a bottleneck if applied to everything, which is why Midas reserves it for important company data. #### Case 3: retiring stale dashboards on a schedule (GitLab, sourced) **Problem.** Clutter on GitLab's Tableau Cloud site. **Method.** A quarterly, manually started script moves workbooks not "created, updated, or viewed in the last 90 days" and unused data sources to a restricted Archive folder; a "Do NOT Archive" tag exempts content.[^gitlab] **Trade-offs.** Archived items vanish from search; restores go through the BI team. **Success criteria.** Only used content on the main site. **Pitfall.** A 90-day rule catches annual reports (hence the tag), and a view-based rule misses content consumed by e-mail; Lab 3 handles both. ## Measure, govern, pay ### Metrics and KPIs | KPI | Definition / formula | Data source | Pitfall | Cadence | |---|---|---|---|---| | Governed coverage (tier 1) | Tier-1 dashboard tiles on a certified metric ÷ all tier-1 tiles | Semantic-layer logs, BI metadata | Counting dashboards, not tiles | Monthly | | Definition conflicts | Metric names with more than one active definition ([Lab 1](#hands-on-labs)) | Inventory, catalogue | Only checking certified content | Monthly | | Certified view share | Views of certified content ([as defined](#what-each-tool-can-certify)) ÷ all views, 90 days; Tableau Blueprint compares certified and ad hoc use[^tabgovin] | Usage logs, catalogue | Rewards over-certification | Quarterly | | Active-content ratio | Items older than N days viewed or delivered in the last N days ÷ items older than N days ([Lab 3](#hands-on-labs)) | Usage and delivery logs | Window choice changes the number | Quarterly | | Freshness attainment | Dashboard-days with data by the promised time ÷ dashboard-days promised | Refresh logs | Ignoring failed refreshes | Weekly | | Opening time (tier 1) | Median and, where logs allow, 95th-percentile time to open each tier-1 dashboard, against a target (Power BI reports the median[^pbiusage]) | BI performance logs, query history | Averages hide Monday peaks | Monthly | | Time to reconcile | Median hours from a conflicting-number report to a published explanation | Incident log | Closing without an explanation | Monthly | | Unannounced breaking changes | Breaking changes released without notice | Change gate, change log | Missing BI-tool edits outside Git | Monthly (target 0) | | Cost per active item | (BI licences + refresh and query compute) ÷ active items | Billing, usage | Licences that cannot be cut | Quarterly | Leading: conflicts, coverage, unannounced changes. Lagging: time to reconcile, active-content ratio. #### Worked calculation (illustrative) Of 1,240 dashboards and reports, 1,180 are older than 90 days and 410 of those were viewed or delivered in the last 90 days: **active-content ratio** = 410 ÷ 1,180 = **34.7%**; two-thirds of the mature estate is refreshed and secured but unused. The 24 tier-1 dashboards have 210 tiles, 172 on certified metrics: **governed coverage** = 172 ÷ 210 = **81.9%**; the other 38 tiles are where the next conflict comes from. They promise data by 07:00 UTC; 698 of 720 dashboard-days in a 30-day month met it: **freshness attainment** = **96.9%** against a 98% target. Of 95,000 views, 61,750 were of certified content: **certified view share** = **65%**. Next quarter: fix the 38 tiles, archive the 770 stale items after notice, take the 22 late days to [DataOps](../dataops/). **Vanity traps:** counts of dashboards or metrics; logins; "100% certified", which means the badge has lost its meaning. ### Quality, security, privacy and compliance | Risk | Signal | Mitigation | Owner | |---|---|---|---| | Defined one way, calculated another | Card and code disagree | One pull request for both; definitional tests; tier-1 reconciliation | Analytics engineer | | Certified content bypassed | Warehouse queries re-create a certified metric | Make the certified path easiest; Lab 1 | Analytics lead | | Policy bypass via exports or direct connections | Large exports; unmanaged extracts | Access enforced in warehouse or semantic layer;[^cube] exports limited ([DataSecOps](../datasecops/)) | Platform admin | | Small groups identifiable in aggregates | Counts of a few people | Minimum group size ([PrivacyOps](../privacyops/)) | Metric owner | | Assistant answers outside policy | Uncertified paths; service-account rights | User-scoped queries; logging | Analytics lead | | No evidence of who changed a regulated metric | Edits in the BI tool, outside Git | Pull requests; keep logs beyond platform defaults ([retention](#tooling-landscape)) | Platform admin | **Quality** means code that matches its card and a number that reconciles with its system of record; Midas requires automated checks including "definitional testing".[^midas] **Compliance:** where a metric feeds financial statements or regulatory returns, its change control is likely audited; agree the evidence with your controls team ([ComplianceOps](../complianceops/)). General guidance, not legal advice. ### Cost, capacity and sustainability **Cost drivers:** refresh frequency times model size; dashboard queries on the warehouse; licences; duplicated models. Power BI allows eight scheduled refreshes a day on shared capacity, up to 48 on Premium, PPU or Fabric capacity.[^pbirefresh] **Unused content still costs;** Power BI pausing a semantic model's refresh after two months without report visits is a backstop, not a policy.[^pbirefresh] Minerva built in "cost attribution" and "usage-based retention and garbage collection".[^minerva] **Performance.** Slow tier-1 dashboards are an operating problem (opening-time KPI), worst when everyone opens them on Monday morning. Pre-aggregations and extracts are the main lever; Cube refreshes pre-aggregations in the background,[^cube] Power BI discards cached results after each refresh,[^pbirefresh] and both serve data as old as the last refresh, so show that age. Query tuning belongs to [DBOps](../dbops/), latency monitoring to [ObservabilityOps](../observabilityops/). **Sustainability:** fewer refreshes of unused content means less compute ([GreenOps](../greenops/)); route spend through [FinOps](../finops/). ### Assurance of the operating model - **Quarterly sample audit:** five certified metrics, each with a current card matching the code, an owner, tests and consumers. - **Risk indicators** (this page's thresholds): an ownerless tier-1 metric; any tier-1 conflict; an unannounced breaking change; active ratio falling two quarters running; freshness below target; usage or delivery data unexported for over a week. ## When things go wrong ### Common failures and troubleshooting | Symptom | Likely cause and diagnosis | Remediation | Prevention | |---|---|---|---| | Two dashboards, two numbers | Different definitions (filter, grain, time zone, currency, refunds); compare cards or SQL | Name a canonical metric; rename or retire the other | Duplicate check at intake; [Lab 1](#hands-on-labs) | | A number jumps after a release, no data change | Silent definition change; check Git history | Revert, or publish with size and back series | [Change gate](#hands-on-labs) | | Tiles break after a model change | Renamed or removed fields; Looker's Content Validator finds them, with no undo[^lkcv] | Repair references with the release | Impact list from exposures[^exposures] | | Yesterday's data shown as current | Failed or paused refresh; check refresh history[^pbirefresh] | Re-run; warn on the content[^tabdqw] | Freshness KPI; "data as of" label | | Certified dashboard nobody uses | Need moved on; certification never reviewed[^lkunused] | Decertify or retire after notice | Quarterly content review | **A sourced failure: defined one way, calculated another.** In 2016 Facebook disclosed on its advertiser help page, as quoted by TechCrunch, that it had defined Average Duration of Video Viewed as "total time spent watching a video divided by the total number of people who have played the video" but calculated it over "*only* the number of people who have viewed a video for three or more seconds". Publicis Media told *The Wall Street Journal* the inflation was 60 to 80 percent, TechCrunch reported; Facebook said billing was unaffected.[^fbtc] In 2019 a proposed $40 million settlement acknowledged calculation errors without admitting other allegations, according to Marketing Dive.[^fbmd] A written definition is a control only if something checks the calculation. ### Worked runbook: two numbers for one metric **Illustrative scenario.** At Monday's executive review, "Active customers" shows 48,200 on the executive dashboard and 51,900 on the product dashboard. The CEO asks which is right. ::: figure Triage for conflicting numbers: definition, data or calculation ```mermaid flowchart TB A["Two numbers for one metric"] --> L["Label both: under review"] L --> Q1{"Same definition card?"} Q1 -- "No" --> D1["Definition conflict"] D1 --> F1["Owner picks canonical; rename the other"] Q1 -- "Yes" --> Q2{"Same data snapshot?"} Q2 -- "No" --> D2["Freshness or data incident"] D2 --> F2["Hand to DataOps runbook"] Q2 -- "Yes" --> D3["Calculation or tool fault"] D3 --> F3["Fix filter, time zone, cache or aggregation"] F1 --> P["Publish explanation; add prevention"] F2 --> P F3 --> P classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029 classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029 class A,L,F1,F2,F3,P core class Q1,Q2 decision class D1,D2,D3 risk ``` Text equivalent: Label both numbers as under review. Different definition cards: a definition conflict; the owner picks the canonical metric and the other is renamed. Same cards, different data snapshots: a freshness or data incident for DataOps. Both match: a calculation or tool fault (filter, time zone, cache, aggregation). Every branch ends with a published explanation and a prevention step. ::: 1. **Label (first 30 minutes).** A banner on both dashboards so nobody forwards either number; say when an answer is due. 2. **Compare definitions (two hours).** Both cards, or the queries. Here the executive dashboard counts customers with an order in the last 30 days; the product dashboard, customers with a session in the last 28 days: a definition conflict, not an error. 3. **Decide.** The metric owner (finance, as the board uses it) keeps "Active customers" for the order-based definition; product's becomes "Engaged customers (sessions, 28-day)". Unclear ownership goes to the metrics council that day. 4. **If the definitions had matched:** compare data snapshots (a difference goes to the [DataOps](../dataops/) incident process), then the calculation: filters, time zones, caches, row-level security and aggregation (daily distinct counts summed into a month, or an average of ratios instead of a ratio of totals). A stale filter in one copy, Uber's case, lands here. 5. **Communicate (same day):** both numbers and definitions, the canonical one, what changes where and when. 6. **Prevent (two weeks):** catalogue both metrics; add Lab 1 to the monthly review; certify the canonical metric; gate the product team's models. ### Anti-patterns and myths | Anti-pattern or myth | Why it fails | Warning signal | Do instead | |---|---|---|---| | "The semantic layer will fix trust" | Tools serve definitions; they do not agree them | Layer live, owners unnamed | Name owners and write cards first | | Certify everything | The badge stops meaning anything; reviews become rubber stamps | Most content certified | Certify tier 1 and 2; review samples | | Silent fixes | Consumers see a jump and lose trust | Changes outside Git | Version, notice, back series[^cop] | | Metric as a target without safeguards | People optimise the measure, not the outcome[^strathern] | Sudden gains with no business change | Pair metrics; audit definitions | | Retire by hard delete | Lost history; angry owners | No trash or archive | Notice, soft delete, restore path[^lkunused] | | Retire on views alone | E-mailed and embedded content looks unused | Board packs on the archive list | Count deliveries ([Lab 3](#lab-3-advanced-a-usage-based-content-review)) | ## Deciding and adopting ### When to use and when not to use **Invest in AnalyticsOps when most of these hold:** - Several teams or tools report the same KPIs, and meetings stall on "which number is right". - Numbers reach a board, investors, regulators or customers, or drive prices, bonuses or automated decisions. - Self-service outgrows review, or AI assistants are about to answer metric questions. **Do not invest yet, or keep it minimal, when:** - One team, one tool, a few dozen dashboards: a short catalogue and certified tables are enough. - The data underneath is unreliable: fix [DataOps](../dataops/) first, or governance makes wrong numbers consistently wrong. - No business owner will own definitions: governance without owners is theatre. **Hidden costs:** review time; plan tiers for certification, deployment pipelines or semantic-layer APIs;[^endorse][^deploystart][^dbtmfcmd] notices, measured changes and back series. **Exit criteria:** with tier-1 conflicts at zero and coverage high, keep only the quarterly review. ### Decision framework First decide whether to adopt a semantic layer at all, then where definitions should live; for dbt users the plan decides ([tooling landscape](#tooling-landscape)). ::: figure Decision tree: where should governed metric definitions live? ```mermaid %%{init: {"flowchart": {"wrappingWidth": 150, "nodeSpacing": 24, "rankSpacing": 36}}}%% flowchart TB S["Metrics to govern"] --> Q0{"Shared by teams or tools?"} Q0 -- "No" --> N["Not yet: catalogue plus certified tables"] Q0 -- "Yes" --> Q1{"Consumers beyond one BI tool?"} Q1 -- "No" --> B["BI-native model: LookML or Power BI model"] Q1 -- "Yes" --> Q2{"dbt standard, with a dbt platform plan?"} Q2 -- "Yes" --> T["Transformation layer (dbt Semantic Layer)"] Q2 -- "No" --> Q3{"One warehouse is the hub?"} Q3 -- "Yes" --> W["Warehouse-native semantics"] Q3 -- "No" --> C["Standalone semantic layer"] B --> G["Then: owners, cards, certification, change gate"] T --> G W --> G C --> G classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029 classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029 class S,B,T,W,C,G core class Q0,Q1,Q2,Q3 decision class N neutral ``` Text equivalent: If one team or tool uses the metrics, do not adopt a semantic layer yet; keep a catalogue and certified tables. If all consumers sit in one BI tool, use its model. Beyond one tool, use the dbt Semantic Layer if dbt is the modelling standard and a dbt platform plan is acceptable; otherwise warehouse-native objects if one warehouse is the hub, else a standalone semantic layer. All options then need owners, cards, certification and a change gate. ::: #### Weighted comparison of the options When two options are close, score them. **Illustrative** for a company with dbt, two BI tools, many spreadsheets and a planned AI assistant; scores 1 (poor) to 5 (good); weighted score = Σ(weight × score) ÷ 5, out of 100. | Criterion (weight) | BI-native | Transformation layer | Warehouse-native | Standalone | |---|---|---|---|---| | Reach across consumers (30) | 2 | 4 | 4 | 5 | | Fit with current skills (20) | 5 | 5 | 3 | 2 | | Operating burden (20) | 5 | 4 | 4 | 2 | | Lock-in and portability (15) | 2 | 3 | 2 | 3 | | Cost incl. plan tiers (15) | 4 | 3 | 3 | 3 | | **Weighted score** | **70** | **78** | **67** | **64** | The transformation layer wins because dbt skills exist and the plan is affordable. Weight reach more (embedded analytics) and the standalone layer, with access rules and caching in one API,[^cube] catches up; without the assistant and second BI tool, the BI-native model suffices. ### Maturity model These levels are **this Atlas's model** for metric governance. For platform adoption, Microsoft's five levels, based on the Capability Maturity Model (CMM), from 100 Initial to 500 Efficient, advise aiming for "at least maturity level 300 or 400".[^msmat] | Level | Observable behaviour | Evidence | Exit criteria | |---|---|---|---| | 0. Numbers by negotiation | Conflicts found in meetings; definitions in queries | "Which number?" escalations | Top dashboards inventoried (Lab 1) | | 1. Catalogued | Top metrics listed with owners | Catalogue; conflict list | Every tier-1 metric has owner and card | | 2. Governed core | Tier-1 metrics as code; badges; change gate | Pull requests, CI logs | No tier-1 conflict or unannounced change for a quarter | | 3. Managed self-service | Team content on certified models; usage review; freshness visible | Coverage, active ratio | Tier-1 coverage ≥ 90%; active ratio rising | | 4. Metrics as products | Every consumer, assistants included, uses the same versioned definitions | Incident and query logs | Sustain; periodic audit | Level 2 suits most tier-1 metrics, level 3 heavy self-service; assess per domain. ### Adoption roadmap: 30/60/90 days Assumes an analytics lead, a part-time analytics engineer, one BI tool plus spreadsheets, and a finance sponsor. | Window | Goals | Activities | Metrics at the end | |---|---|---|---| | Days 1–30: see the problem | Know the conflicts | Inventory the top 20 dashboards (Lab 1); owners for the top 15 metrics; daily usage and delivery export | Conflict count; owners named | | Days 31–60: govern the core | One definition per tier-1 metric | Cards; tier-1 metrics in one semantic layer or BI model; certification criteria; change gate (Lab 2); "data as of" labels | Tier-1 coverage; no unannounced changes | | Days 61–90: open it up | Self-service on certified models | First content review (Lab 3); creator training; governance review; runbook rehearsal | Active ratio; time to reconcile; freshness | **Quick wins:** settle the loudest conflicts; archive clearly dead content after notice. **Dependencies:** reliable tables ([DataOps](../dataops/)), admin access to usage data, a sponsor who arbitrates definitions. ## Toolkit ### Checklists #### Preflight for a new metric - [ ] Catalogue searched; no existing metric or qualified variant fits. - [ ] Owner named, tier assigned, card complete. - [ ] Encoded with owner, tier, status and version metadata; validation passes. #### Certification - [ ] Card and code match, in one pull request; tier 1 reconciled with its system of record. - [ ] Consumers declared; freshness promise written. - [ ] Certifier is not the author; notes say what the badge means. #### Quarterly content review - [ ] Usage and delivery data cover the whole window, with no export gaps. - [ ] Worklist produced (Lab 3); owners notified; keep exceptions re-confirmed; archive restorable. - [ ] Active ratio and coverage recorded with the window used. #### Closing a conflicting-number incident - [ ] Class recorded (definition, data, calculation); explanation sent to all who saw both numbers. - [ ] Canonical metric certified; the other renamed or retired; prevention step merged. ### Templates #### Metric definition card ```text METRIC: Version: Status: draft | certified | deprecated Owner (accountable): Steward (encodes): Tier: 1 | 2 | 3 Business definition: Formula: of from Grain: Additivity: Includes: <...> Excludes: <...> Time zone and calendar: Currency rule: Late data: Freshness promise: Reconciles with: Known caveats: <...> Consumers: Change log: ``` #### Breaking-change notice ```text METRIC CHANGE NOTICE — v -> v Effective: What changes: -> Why: Size: Affected: Action for you: Contact: Objections by: ``` #### Content retirement notice ```text CONTENT REVIEW — To: These items have not been viewed for days (usage data from , exported ): | last viewed | proposed action Reply by to keep an item (state why and for how long). Archived items can be restored from until . ``` #### Certification criteria (scorecard) | Criterion | Pass condition | |---|---| | Ownership | Named business owner and steward | | Definition | Card complete and matches code | | Correctness | Tests pass; tier 1 reconciled | | Freshness | Promise written; attainment measured | | Consumers | Declared; impact list possible | | Usage | Viewed in the last 90 days, an active schedule or subscription, or a documented exception | ### FAQ **Is AnalyticsOps just DataOps for dashboards?** Partly: it reuses DataOps habits, but its object is the definition people read; perfect data can still produce disagreeing dashboards. **Who should own a metric definition: the BI team or the business?** The business owns meaning and approves changes; analytics owns encoding, tests and process. A BI team that owns meaning ends up arbitrating between departments it does not run. **Is a dbt model tagged `certified` in its `meta` a certified metric?** No. dbt treats `meta` "as metadata only, since it doesn't affect how a resource is materialized";[^dbtmeta] a tag is a claim; certification is a recorded decision against published criteria by someone other than the author. A certified *table* (as in Midas) is trusted data; a certified *metric* also fixes the definition. **A source table loaded late and a KPI was wrong for a morning. Whose incident is it?** The late load is a [DataOps](../dataops/) incident; AnalyticsOps owns the consumer side: the warning on the content, the "data as of" label, the note to people who saw the number, and freshness attainment. **We changed a definition and the chart shows a step. What now?** Publish a back series under the new definition, or annotate the break with its size; the UK statistics code asks for "a consistent back series where possible".[^cop] ## Going further ### Advanced topics and emerging direction - **Interchange standards (emerging).** Apache Ossie is incubating;[^ossie] lossless exchange between vendors is unproven. - **Warehouse-native semantics (established, evolving).** Documented products;[^sfsv][^dbxmv] check your BI tools' support. - **Assistants as metric consumers (real but limited).** Governed context helps;[^sequeda] letting a model choose which metric answers a business question is speculative. - **Metrics shared with experimentation and ML (established at scale).** Airbnb serves A/B tests and reporting from the same definitions;[^minerva] the boundary with [MLOps](../mlops/) blurs. ### Learning path - **Beginner (one day):** read to the [glossary](#glossary-and-mental-model); Lab 1 on five dashboards; the Minerva and uMetric posts. - **Practitioner (one to four weeks):** Lab 2 in a real dbt or LookML project; cards for ten metrics; the [runbook](#worked-runbook-two-numbers-for-one-metric) as a tabletop; then [DataOps](../dataops/). - **Leader or architect (a quarter):** the [judgement call](#a-judgement-call-correct-the-board-metric-now-or-at-quarter-end); the decision framework; Lab 3 across the estate; then [BizOps](../bizops/) and [DecisionOps](../decisionops/). - **Portfolio project:** take one contested metric to a certified, versioned definition and retire its duplicate. ## Wrap-up ### Executive recap and cheat sheet | Topic | What to remember | |---|---| | Definition | An ambiguous label; here, operating the consumption layer so each important metric has one trusted definition. | | Core artefacts | Definition cards, a semantic layer, certification criteria, a change gate, a usage-based content review. | | Decisions | Who owns meaning (the business); where definitions live (decision tree); what to certify (tier 1 and 2). | | Metrics | Governed coverage, definition conflicts, active-content ratio, freshness attainment, time to reconcile. | | Watch-outs | Tool before definitions; certify everything; silent changes; retiring on views alone; short log retention. | | Next actions | Inventory the top 20 dashboards; name owners; export usage and delivery logs; gate tier-1 changes. | ### A judgement call: correct the board metric now or at quarter end? **Situation (illustrative).** Five weeks before quarter end, an analytics engineer finds that "Active customers", a tier-1 board metric, counts reactivated customers twice in their return month; Lab 2's comparison puts the overstatement at 3.1%, and last quarter's board pack used the figure. **Option A: correct now** (version 3 to 4), recompute the back series, publish a correction with its size, brief the board chair. **Option B: freeze until quarter end** for comparability, footnote it, switch next quarter. **Assumptions:** a defect, not a debatable choice; not in statutory accounts; one bonus plan references it; two days of work. **The attractive choice and why it can be wrong.** B looks calmer but is the wrong default for an *error*: every week people decide on a number known to be wrong, the bonus plan pays against it, and a footnote found later costs more trust than a correction; Facebook's gap, once public, became a legal problem.[^fbtc][^fbmd] The UK statistics code draws the line: errors are corrected "transparently and as soon as possible"; *planned method changes* get advance notice and a back series.[^cop] **Recommendation.** Choose A, with the back series and measured size; the remuneration committee decides how the bonus plan treats the restatement. Choose B only for a *definition choice* reasonable people would make differently (28 versus 30 days), scheduled with notice. **Uncertainty:** for external or regulated reporting, legal and audit may impose their own timing. ### Learning objectives and mastery check You should now be able to **explain** the label's four uses; **diagnose** whether conflicting numbers come from definitions, data or calculation; **design** a card, a certification process and a change gate; **choose** where definitions live; and **measure** governance. 1. A sales dashboard and a finance dashboard both show "Revenue", with different values, and every pipeline test passes. What is the most likely class of problem, and whose page owns it? ::: answer Show answer for question 1 A definition problem, owned here rather than by DataOps: rival definitions, or copies of one definition that have drifted (filters, grain, time zone, refunds). Compare the definition cards first, then the queries. ::: 2. A pull request changes one filter in `gross_revenue` and nothing else. Why is that a breaking change, and what must accompany it? ::: answer Show answer for question 2 It changes `gross_revenue` and every metric and dashboard built on it (in Lab 2, `net_revenue`, `refund_rate` and the executive dashboard). `gross_revenue` needs a version bump and a change note; its dependants keep their versions but are named in the notice to affected owners, with the measured size and a back series. ::: 3. The quarterly review shows a 34.7% active-content ratio with a 90-day window. A manager proposes a 365-day window "to be fair". What is the trade-off? ::: answer Show answer for question 3 It protects annual content but hides a year of unused refreshes; the ratio rises with nothing improved. Keep 90 days, explicit keep exceptions and counted deliveries, and report the window with the number. ::: 4. Leadership sets a target of 95% certified view share. What could go wrong? ::: answer Show answer for question 4 Certification becomes easier rather than content better: Goodhart's law in Strathern's form ([methods](#methods-standards-and-frameworks)). Pair the target with a sample audit of certified items and with conflict counts. ::: 5. An AI assistant answers "What was net revenue last month?" with a figure 2% above the dashboard. What do you check, in order? ::: answer Show answer for question 5 The definition and version used (certified or raw tables), the permissions it queried with (row-level security changes totals), the data snapshot. Then require certified metrics, definition shown, for tier-1 answers. ::: ### Conclusion AnalyticsOps is a label without an owner, but the problem behind this page's meaning is well documented: Airbnb and Uber found their teams giving different answers to simple questions, and Facebook's advertisers learned what a gap between definition and calculation costs. The fix is unglamorous: write down what each important number means, name who decides, encode it once, change it in the open and say by how much, show whether it is fresh and certified, and retire what nobody uses. This page's bet: teams that treat metrics as products spend their meetings deciding; teams that treat them as by-products of dashboards keep negotiating the same numbers. ## Sources and further reading Each footnote's "Checked" date is the day its links were last opened: 2026-10-17, or 2026-10-18 for links added or reopened in the final polish. Other dates are the source's own. Claim-level support: `research/analyticsops-ledger.md`. [^modelop]: [What is AnalyticOps?](https://www.modelop.com/opendatagroup/analyticops) — ModelOp, legacy Open Data Group content, undated. Checked 2026-10-17. [^odgrebrand]: [Open Data Group Rebrands as ModelOp to Lead in Enterprise ModelOps](https://www.modelop.com/blog/open-data-group-officially-becomes-modelop) — ModelOp, 2019-10-16. Checked 2026-10-17. [^aoa]: [PyPI metadata for package "aoa" (Python client for Teradata AnalyticOps Accelerator)](https://pypi.org/pypi/aoa/json) — Python Package Index; latest release 7.0.1, 2023-03-15. Checked 2026-10-17. [^menninger]: [Analytic Ops: The Last Mile of Data Ops](https://research.isg-one.com/analyst-perspectives/analytic-ops-the-last-mile-of-data-ops) — David Menninger, Ventana Research (now ISG Software Research), 2021-11-24. Checked 2026-10-17. [^wiiisdom]: [Introducing AnalyticsOps](https://wiiisdom.com/solutions/introducing-analyticsops/) — Wiiisdom, 2022-03-09 (modified 2024-07-03). Checked 2026-10-17. [^umetric]: [The Journey Towards Metric Standardization](https://www.uber.com/us/en/blog/umetric/) — Xiaodong Wang et al., Uber Blog, 2021-01-12. Checked 2026-10-17. [^minerva]: [How Airbnb Achieved Metric Consistency at Scale, Part I (archived copy)](https://web.archive.org/web/20241231232628/https://medium.com/airbnb-engineering/how-airbnb-achieved-metric-consistency-at-scale-f23cc53dea70) — Robert Chang et al., The Airbnb Tech Blog, 2021-04-30; Internet Archive snapshot of 2024-12-31 (the live page blocks automated access). Checked 2026-10-17. [^midas]: [Data Quality at Airbnb, Part 2 — A New Gold Standard (archived copy)](https://web.archive.org/web/20241220191630/https://medium.com/airbnb-engineering/data-quality-at-airbnb-870d03080469) — Vaughn Quoss, Jonathan Parks, Paul Ellwood, The Airbnb Tech Blog, 2020-11-24; Internet Archive snapshot of 2024-12-20. Checked 2026-10-17. [^fbtc]: [Facebook miscalculation significantly inflated average video view times for years](https://techcrunch.com/2016/09/22/facebook-miscalculation-significantly-inflated-average-video-view-times-for-years/) — Devin Coldewey, TechCrunch, 2016-09-22. Checked 2026-10-17. [^fbmd]: [Facebook nears $40M settlement of advertiser lawsuit](https://www.marketingdive.com/news/facebook-nears-40m-settlement-of-advertiser-lawsuit/564534/) — Marketing Dive, 2019-10-08. Checked 2026-10-17. [^gitlab]: [Tableau Admin Guide](https://handbook.gitlab.com/handbook/enterprise-data/platform/tableau/tableau-admin-guide/) — GitLab Handbook, last modified 2025-05-27. Checked 2026-10-17. [^msown]: [Microsoft Fabric adoption roadmap: Content ownership and management](https://learn.microsoft.com/en-us/power-bi/guidance/fabric-adoption-roadmap-content-ownership-and-management) — Microsoft Learn, 2024-12-30. Checked 2026-10-18. [^msmat]: [Microsoft Fabric adoption roadmap maturity levels](https://learn.microsoft.com/en-us/power-bi/guidance/fabric-adoption-roadmap-maturity-levels) — Microsoft Learn, 2024-12-30. Checked 2026-10-17. [^endorse]: [Endorsement overview](https://learn.microsoft.com/en-us/fabric/governance/endorsement-overview) — Microsoft Learn, 2024-07-11. Checked 2026-10-17. [^pbirefresh]: [Data refresh in Power BI](https://learn.microsoft.com/en-us/power-bi/connect-data/refresh-data) — Microsoft Learn, 2026-09-18. Checked 2026-10-17. [^pbiusage]: [Monitor usage metrics in workspaces](https://learn.microsoft.com/en-us/power-bi/collaborate-share/service-modern-usage-metrics) — Microsoft Learn, 2025-12-01. Checked 2026-10-17. [^pbiactlog]: [Access the Power BI activity log](https://learn.microsoft.com/en-us/power-bi/guidance/admin-activity-log) — Microsoft Learn, 2024-12-30. Checked 2026-10-17. [^fabaudit]: [Track user activities in Microsoft Fabric](https://learn.microsoft.com/en-us/fabric/admin/track-user-activities) — Microsoft Learn. Checked 2026-10-17. [^purview]: [Auditing solutions in Microsoft Purview](https://learn.microsoft.com/en-us/purview/audit-solutions-overview) — Microsoft Learn. Checked 2026-10-17. [^pbip]: [Power BI Desktop projects (PBIP)](https://learn.microsoft.com/en-us/power-bi/developer/projects/projects-overview) — Microsoft Learn, 2026-09-23. Checked 2026-10-17. [^deploystart]: [Get started with deployment pipelines](https://learn.microsoft.com/en-us/fabric/cicd/deployment-pipelines/get-started-with-deployment-pipelines) — Microsoft Learn, 2025-12-15. Checked 2026-10-17. [^tabgov]: [Tableau Governance Models](https://help.tableau.com/current/blueprint/en-us/bp_governance_models.htm) — Tableau Blueprint, Tableau Help. Checked 2026-10-17. [^tabgovin]: [Governance in Tableau](https://help.tableau.com/current/blueprint/en-us/bp_governance_in_tableau.htm) — Tableau Blueprint, Tableau Help. Checked 2026-10-17. [^tabcert]: [Use Certification to Help Users Find Trusted Data](https://help.tableau.com/current/online/en-us/datasource_certified.htm) — Tableau Cloud Help. Checked 2026-10-17. [^tabinsights]: [Use Admin Insights to Create Custom Views](https://help.tableau.com/current/online/en-us/adminview_insights.htm) — Tableau Cloud Help. Checked 2026-10-17. [^tabdqw]: [Set a Data Quality Warning](https://help.tableau.com/current/online/en-us/dm_dqw.htm) — Tableau Help. Checked 2026-10-17. [^lookml]: [What is LookML?](https://docs.cloud.google.com/looker/docs/what-is-lookml) — Looker documentation, Google Cloud, updated 2026-10-13. Checked 2026-10-17. [^lkcert]: [Content certification](https://docs.cloud.google.com/looker/docs/content-certification) — Looker documentation, Google Cloud. Checked 2026-10-17. [^lktest]: [test (LookML data tests)](https://docs.cloud.google.com/looker/docs/reference/param-model-test) — Looker documentation, Google Cloud, updated 2026-10-13. Checked 2026-10-17. [^lkcv]: [Using the Content Validator](https://docs.cloud.google.com/looker/docs/content-validation) — Looker documentation, Google Cloud, updated 2026-10-13. Checked 2026-10-17. [^lkunused]: [Managing unused content](https://docs.cloud.google.com/looker/docs/manage-unused-content) — Looker documentation, Google Cloud, updated 2026-10-13; see also [Deleted and unused content for admins](https://docs.cloud.google.com/looker/docs/admin-spaces). Checked 2026-10-18. [^mf]: [About MetricFlow](https://docs.getdbt.com/docs/build/about-metricflow) — dbt Developer Hub. Checked 2026-10-17. [^dbtspec]: [Migrate to the latest YAML spec](https://docs.getdbt.com/docs/build/latest-metrics-spec) — dbt Developer Hub. Checked 2026-10-17. [^dbtmetrics]: [Creating metrics](https://docs.getdbt.com/docs/build/metrics-overview) — dbt Developer Hub; see also [Derived metrics](https://docs.getdbt.com/docs/build/derived), [Ratio metrics](https://docs.getdbt.com/docs/build/ratio) and [Measures, "Non-additive dimensions"](https://docs.getdbt.com/docs/build/measures#non-additive-dimensions). Checked 2026-10-18. [^dbtvalid]: [Validations](https://docs.getdbt.com/docs/build/validation) — dbt Developer Hub. Checked 2026-10-17. [^dbtmfcmd]: [MetricFlow commands](https://docs.getdbt.com/docs/build/metricflow-commands) — dbt Developer Hub. Checked 2026-10-17. [^dbtmeta]: [meta](https://docs.getdbt.com/reference/resource-configs/meta) — dbt Developer Hub. Checked 2026-10-18. [^exposures]: [Add Exposures to your DAG](https://docs.getdbt.com/docs/build/exposures) — dbt Developer Hub. Checked 2026-10-17. [^mflicense]: [MetricFlow repository README, "License History"](https://github.com/dbt-labs/metricflow) — dbt Labs, GitHub; licence change commit dated 2025-10-14. Checked 2026-10-17. [^transform]: [dbt Labs Signs Definitive Agreement to Acquire Transform, Accelerating Development of the dbt Semantic Layer](https://www.getdbt.com/blog/dbt-labs-signs-definitive-agreement-to-acquire-transform-accelerating-development-of-the-dbt-semantic-layer) — dbt Labs (PR Newswire), 2023-02-08. Checked 2026-10-17. [^ossie]: [Apache Ossie (Incubating): The New Name for Open Semantic Interchange](https://ossie.apache.org/updates/ossie-enters-apache-incubator/) — Josh Klahr, Apache Ossie (incubating), 2026-07-10. Checked 2026-10-17. [^sfsv]: [Overview of semantic views](https://docs.snowflake.com/en/user-guide/views-semantic/overview) — Snowflake Documentation. Checked 2026-10-17. [^dbxmv]: [Unity Catalog metric views](https://docs.databricks.com/aws/en/uc-semantics/metric-views) — Databricks on AWS, updated 2026-09-11. Checked 2026-10-17. [^cube]: [Introduction](https://docs.cube.dev/docs/introduction) — Cube documentation. Checked 2026-10-17. [^carroll]: [What is analytics engineering?](https://www.getdbt.com/blog/what-is-analytics-engineering) — Claire Carroll, dbt Labs, 2019-03-26. Checked 2026-10-17. [^srewb]: [Data Processing Pipelines](https://sre.google/workbook/data-processing/) — The Site Reliability Workbook (Google, O'Reilly, 2018). Checked 2026-10-17. [^cop]: [Code of Practice for Statistics, Edition 3.0 (PDF)](https://osr.statisticsauthority.gov.uk/wp-content/uploads/sites/3/2026/08/Code-of-Practice-for-Statistics-3.0.pdf) — Office for Statistics Regulation, UK Statistics Authority, Edition 3.0. Checked 2026-10-17. [^strathern]: ['Improving ratings': audit in the British University system](https://www.cambridge.org/core/journals/european-review/article/abs/improving-ratings-audit-in-the-british-university-system/FC2EE640C0C44E3DB87C29FB666E9AAB) — Marilyn Strathern, European Review 5(3):305–321, 1997; the publisher page shows the abstract, and the quoted sentence was checked in a full-text copy. Checked 2026-10-17. [^sequeda]: [A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databases](https://arxiv.org/abs/2311.07509) — Juan Sequeda, Dean Allemang, Bryon Jacob (data.world), arXiv:2311.07509, 2023-11-13. Checked 2026-10-18. [^dataworld]: [The Data Catalog Platform](https://data.world/) — data.world (part of ServiceNow), product home page. Checked 2026-10-17. [^checkout]: [actions/checkout v7.0.1, action.yml (`persist-credentials`, default `true`)](https://github.com/actions/checkout/blob/3d3c42e5aac5ba805825da76410c181273ba90b1/action.yml) — GitHub. Checked 2026-10-17. [^mfprofiles]: [dbt-metricflow 0.15.0, `cli_configuration.py`](https://github.com/dbt-labs/metricflow/blob/bd05dd9a145f290a1bf3090cbdbb3b11d7322701/dbt-metricflow/dbt_metricflow/cli/cli_configuration.py) — dbt Labs, GitHub; see also [About profiles.yml](https://docs.getdbt.com/docs/local/profiles.yml) — dbt Developer Hub. Checked 2026-10-18. [^ghskip]: [Skipping workflow runs](https://docs.github.com/en/actions/how-tos/manage-workflow-runs/skip-workflow-runs) — GitHub Docs. Checked 2026-10-17. **Changelog.** 2026-10-17, r0: first draft (research, ledger, labs executed, six diagrams). 2026-10-17, r1: revised after the independent fact check: per-tool certification table; model-level change gate and measured change size; deliveries count as use; usage-log retention, attributions, CI hardening, glossary and FAQ fixes; cuts; labs re-run. 2026-10-18, post-score polish: CI profile via `DBT_PROFILES_DIR`, `ci/profiles.yml` shown; additivity; dependants' versions; Looker 26.14; `meta` and benchmark wording; clearer sentences; repetition cut; after the polish check, Sources dating and the base-branch check explained.