--- title: "NoOps" slug: noops category: delivery term_status: aspirational evidence_status: verified summary: "NoOps is the aspiration that automation and fully managed platforms can remove the need for a dedicated operations team. In practice it is a trade-off: operational work shrinks and moves to providers, platforms and developers, but it does not disappear." audience: [CTOs and engineering leaders, Architects, Startup teams, Platform and cloud engineers, Beginners] last_verified: 2026-10-16 key_takeaways: - "NoOps is an aspiration, not a discipline. A Forrester analyst proposed it in February 2011 as an alternative to DevOps, which he called \"a step backward\"; Forrester's April 2011 report softened it to automation that augments DevOps." - "The label has two meanings: no dedicated operations team because operations is bought or automated (this page's focus), and fully automated operations with no human involvement, which TechTarget links to initiatives such as AIOps." - "Managed services displace operations work to the provider, a platform team, developers on call, automation code and vendor management. Little of it is simply deleted." - "In every cloud service model, including SaaS, your data, identities, configuration and access policies remain your responsibility." - "Estimate what remains as two costs: hours of work, and coverage (someone reachable when it breaks). For a small team that promises 24/7 response, coverage usually dominates." - "A service cannot be more available than its critical dependencies allow, so a managed-first design still needs a plan for provider failure." - "Less operations fits standard, spiky workloads. Stable high-volume workloads and unusual hardware, runtime or latency needs often fit better with more control." related: [serverlessops, platformops, cloudops, devops, aiops, automationops, finops, sre] --- ## What NoOps is ### In 30 seconds **NoOps** ("no operations") is the idea that a software organisation can automate and buy infrastructure work so thoroughly that application teams need no dedicated operations team. Forrester analyst Mike Gualtieri proposed it in February 2011 in a blog post that called DevOps "a step backward" and promised that "application developers will never have to speak with an operations professional again"; even he expected that "Ops can run this public, private, or hybrid infrastructure".[^gualtieri2011] That April, Forrester's report *Augment DevOps With NoOps* softened the idea: "Continue to pursue DevOps, but prepare to augment it with NoOps".[^forrester2011] **In one line: NoOps is a design goal for buying and automating as much operational work as is sensible, not a state in which operational work stops existing.** Managed services take over patching, hardware, capacity and much availability engineering. You still own data, identities, configuration, cost, announced runtime upgrades, the response when the provider fails, and the decision to leave. This page shows how to estimate and own that remainder. ::: note Two meanings in circulation 1. **No *dedicated* operations team**: operations is bought or automated, and developers or a platform team absorb the rest. Netflix used the word this way in 2012;[^cockcroft2012] this page evaluates this meaning. 2. **No human involvement at all.** TechTarget describes "True NoOps" as "a completely automated IT environment that requires absolutely no human involvement", relying on initiatives such as AIOps; "Some IT vendors claim to have largely achieved NoOps".[^techtarget2023] See [emerging direction](#advanced-topics-and-emerging-direction) and [AIOps](../aiops/). ::: **For** leaders deciding whether to hire operations staff, architects choosing building blocks, and engineers who inherit a "NoOps" stack. ### At a glance | Item | Summary | |---|---| | Term status | **Aspirational.** No standard definition, governing body or certification. Reviews conclude that full "no operations" has not arrived (Tozzi, 2022: "NoOps hasn't materialized"; TechTarget, 2023: "few organizations have reached" it), while some vendors claim otherwise.[^tozzi2022][^techtarget2023] | | In scope | Managed, PaaS, serverless and SaaS as an operating choice; the operations that remain; lock-in, control and cost. | | Out of scope | Serverless mechanics → [ServerlessOps](../serverlessops/); building a platform → [PlatformOps](../platformops/); cloud governance, quotas and provider events → [CloudOps](../cloudops/); automation engineering → [AutomationOps](../automationops/); spend → [FinOps](../finops/). | | Prerequisites | Delivery basics ([DevOps](../devops/)); cloud service models ([CloudOps](../cloudops/)). | | Key outputs | A responsibility register per service; a work-and-coverage estimate; routed provider signals; a dependency-failure plan; an exit-cost estimate. | ### The problem NoOps tries to solve Operations work that does not differentiate the product is expensive. Google's SRE book calls the repetitive part **toil**: "manual, repetitive, automatable, tactical, devoid of enduring value".[^sretoil] Developers also waited on operations queues; at Netflix in 2012, hours a week of meetings with operations became seconds of self-service.[^cockcroft2012] | Operating reality | Self-managed (before) | Managed-first (after) | What did not go away | |---|---|---|---| | Servers and OS | You patch hosts | Provider does | Runtime and dependency upgrades | | Capacity | You forecast and buy | Elastic within quotas | Quota headroom, cost of scale | | Availability | You build redundancy | Provider, inside its service | Provider-failure design; restores | | Security | Every layer | Provider's layers | Identities, access, configuration, data | | Incidents | Code and infrastructure | Mostly code | Provider incidents become yours | | Cost | Mostly fixed | Variable | Unit economics | **Getting it wrong** cuts both ways: self-managing commodity infrastructure wastes time on toil; assuming operations has gone leaves duties unowned until incidents expose them. Hosting alone is not enough: DORA's 2024 research concluded that "Organizations may be better off staying in the data center if they are not willing to radically transform their application or service."[^doraflex] ### What NoOps is not - **Not the absence of operations.** Serverless "does not mean that we no longer use servers to host and run code; nor does it mean that operations engineers are no longer required" (CNCF).[^cncf2018] - **Not a job title, a certification or a synonym for serverless.** PaaS, managed Kubernetes, managed databases, SaaS and internal platforms are other routes ([ServerlessOps](../serverlessops/), [PlatformOps](../platformops/)). - **Not simply the opposite of DevOps, although it began that way.** Replying to Netflix in 2012, John Allspaw quoted Mark Imbriaco calling its model the "'developers own their code throughout the entire lifecycle' approach" that "the rest of us" had used "for years".[^allspaw2012] - **Not a headcount plan.** What Netflix described, Allspaw wrote, "is Operations", whatever the titles: "the responsibility to make it safe to make whatever change to production is necessary".[^allspaw2012] This page's inference: removing operations skills leaves the work with people who may lack the training or time. ## Where NoOps comes from and where it sits ### Origins and evolution | Year | Milestone | What it shows | |---|---|---| | 2011 | Gualtieri's post "I Don't Want DevOps. I Want NoOps." (February); Forrester's report (April, Gualtieri and O'Donnell)[^gualtieri2011][^forrester2011] | A provocation, softened within months | | 2012 | Adrian Cockcroft on "NoOps" at Netflix; Allspaw's reply. Cockcroft later added: "lets just call it PaaS for now, and work on a better definition of DevOps"[^cockcroft2012][^allspaw2012] | Its best-known proponent steps back | | 2016–2018 | Charity Majors, "WTF is operations? #serverless"; CNCF Serverless Whitepaper v1.0[^majors2016][^cncf2018] | Serverless revives the claim; Majors and the CNCF reject it | | 2019 | UC Berkeley, *Cloud Programming Simplified*[^berkeley2019] | Serverless "handles virtually all the system administration operations", with less predictable cost and more lock-in | | 2026 | AWS and Microsoft document AI operations agents ([emerging direction](#advanced-topics-and-emerging-direction)) | The "no humans" meaning returns | ### Glossary and mental model | Term | Meaning on this page | |---|---| | **Low-ops** | This Atlas's term for realistic NoOps: a small, explicitly owned operational remainder. | | **Serverless** | Functions plus managed back-end services, billed by demand.[^cncf2018] | | **Shared responsibility** | The provider's published split of who operates each control; Google's "shared fate" adds secure defaults.[^azuresrm][^gcpsrm] | | **Residual operations** | Duties that remain with you, such as runtime upgrades and restore tests. | | **Work and coverage** | Hours spent on duties, versus someone being reachable. | | **SLI, SLO, SLA** | SLI: "a carefully defined quantitative measure of some aspect of the level of service"; SLO: a target for an SLI; SLA: a contract "that includes consequences of meeting (or missing) the SLOs".[^sreslo] | | **Quota** | A provider limit: "hard limits, which you cannot change, and soft limits, which you can request increases for".[^lambdaquota] | | **ADR, FTE, MFA** | Architecture decision record; full-time equivalent; multi-factor authentication. | **Mental model: operations is displaced, not deleted.** The provider operates below a **responsibility line**, you above it. Each abstraction since 2011 has moved the line up, never to the top: Microsoft's matrix keeps data, identities and configuration with the customer in every model.[^azuresrm] For each task that "went away", ask **where did it go?** ::: figure Where operations work goes when you adopt managed services ```mermaid flowchart LR subgraph Before["Self-managed: one ops queue"] P1["Patch OS and hosts"] P2["Plan and buy capacity"] P3["Build redundancy"] P5["Answer pages"] P4["Deploy releases"] P7["Configure services"] P6["Manage access and data"] end subgraph After["Managed-first: work is displaced"] D1["Provider: hardware, OS patching, in-service redundancy"] D5["Management: quotas, cost, vendors, exit"] D3["Product team: on-call, upgrades, deploys, failure design"] D2["Platform team: paved road, guardrails"] D4["Automation code: needs an owner"] D6["Always yours: data, identity, config, access"] end P1 --> D1 P2 --> D1 P2 --> D5 P3 --> D1 P3 --> D3 P5 --> D3 P4 --> D3 P4 --> D2 P4 --> D4 P7 --> D6 P6 --> D6 classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029 classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029 class P1,P2,P3,P4,P5,P6,P7 neutral class D1,D2,D3,D4,D5 core class D6 risk ``` Text equivalent: One self-managed operations queue (left) splits after adopting managed services (right). Patching goes to the provider; capacity to the provider and to management; redundancy to the provider and the product team; pages and deploys to the product team, the platform team and automation that needs an owner. Configuration, access and data stay with you (red). ::: ### How NoOps relates to other Ops Neighbours teach the mechanisms of "less ops"; this page asks what remains and who owns it (an editorial split, not a standard). | Neighbouring page | Shared scope | Its unique focus | NoOps's unique focus | |---|---|---|---| | [DevOps](../devops/) | Developers owning what they ship | CI/CD, DORA metrics | Whether developers can absorb what remains | | [PlatformOps](../platformops/) | Hiding infrastructure | A platform run as a product | Build a platform, buy one, or neither | | [DevExOps](../devexops/) | Removing waits for developers | Measuring friction | Whether "no ops" removed friction or moved pages | | [CloudOps](../cloudops/) | Shared responsibility, quotas, provider events | Governing the provider's control plane | Sizing what remains | | [InfraOps](../infraops/) | Infrastructure as code | The IaC lifecycle | IaC as a duty that survives managed services | | [KubernetesOps](../kubernetesops/) | Managed clusters | Upgrades and troubleshooting | What a managed control plane still leaves you | | [ITOps](../itops/) | Incidents, change, service levels | ITIL practices, CMDB, SLAs | Which practices remain (provider-driven change, supplier oversight) | | [ServerlessOps](../serverlessops/) | Shedding operations | Cold starts, concurrency, retries | Whether serverless reduces operations *for you* | | [SRE](../sre/) | Toil, on-call, availability | SLOs and error budgets | How much of that work survives | | [AIOps](../aiops/) | Self-running operations | ML on operational data | Testing "autonomous" claims | | [AutomationOps](../automationops/) | Removing manual work | Safe automation | Automation as a destination of displaced work | | [FinOps](../finops/) | Pay-per-use economics | Financial operating model | Cost as a residual duty | ## Principles and operating model ### A proposed charter and principles There is no official NoOps manifesto; this charter is **this Atlas's proposal** (one-page version with goals in the [templates](#templates)). **Mission:** buy or automate undifferentiated operations; keep every remaining duty owned, measured and rehearsed. 1. **Displacement, not deletion.** For every task a change "removes", name the new owner. 2. **The always-retained band.** Data, identities, configuration and access policies are yours in every model.[^azuresrm][^gcpsrm] 3. **Own the critical path.** In Charity Majors's 2016 talk, as reported by InfoQ, "services are not magical pixie dust"; one of her top tips was to "own the critical path (and keep it short)".[^infoq2016] 4. **Design for provider failure.** "A service cannot be more available than the intersection of all its critical dependencies."[^calculus2017] 5. **Operations is a property, not a team.** "Operational excellence is not a role, it's an emergent property."[^majors2016] 6. **Keep decisions reversible.** Know the exit cost before you depend on a service. **Non-goals:** cutting headcount for its own sake; outsourcing accountability to an SLA. **Accepted trade-off:** lock-in, which "may be stronger with serverless computing than for serverful computing".[^berkeley2019] ### Building blocks of a low-ops system Each block moves the line by a different amount; the linked pages teach the mechanics. | Building block | Provider absorbs | You still own | Main risk | Exit cost | |---|---|---|---|---| | Self-managed on IaaS ([SysOps](../sysops/)) | Hardware, hypervisor | Guest OS patching and everything above[^awssrm] | Toil and hiring | Low | | Managed Kubernetes ([KubernetesOps](../kubernetesops/)) | Control plane; node OS patches, applied automatically only with auto-upgrade or a release channel (GKE example)[^gkesrm] | Workloads, images, RBAC, upgrade policy; manual upgrades when auto-upgrades are postponed[^gkesrm] | Kubernetes complexity | Moderate | | PaaS and serverless containers ([ServerlessOps](../serverlessops/)) | OS, runtime platform, scaling | Code, dependencies, configuration; restarts (Heroku's Cedar dynos restart "once every 24 hours (plus up to 216 random minutes)", losing local files)[^herokucycle][^cloudrun] | Platform limits | Moderate | | Functions (FaaS) | Servers, managed-runtime patching | Code, dependencies, quotas; major runtime upgrades, "a customer-driven operation" on Lambda[^lambdart][^lambdaquota] | Cost at steady volume; lock-in[^berkeley2019] | Moderate to high | | Managed data ([DBOps](../dbops/), [BackupOps](../backupops/)) | Infrastructure, engine operations | Data, backup strategy, access, schema[^warel] | Data gravity | Export-dependent | | SaaS ([IdentityOps](../identityops/)) | Almost the whole application | Accounts, access, configuration, data[^azuresrm] | Identity hygiene | Export-dependent | | Internal platform ([PlatformOps](../platformops/)) | Whatever the platform team runs | Product teams keep code and on-call | Becomes a ticket queue | Internal | **Loops you keep**, whatever you buy: change control as code ([InfraOps](../infraops/)), guardrails ([CloudSecOps](../cloudsecops/)), user-facing SLIs ([ObservabilityOps](../observabilityops/)), routed signals, restore tests. ### Roles, ownership and RACI NoOps redistributes roles rather than removing them ([Case 1](#use-cases-and-scenarios)). In Team Topologies terms, stream-aligned teams own a flow of work and platform teams provide "a compelling internal product".[^teamtop] A named, often rotating **residual-ops owner** keeps the registers current. **RACI for recurring residual duties** (R responsible, A accountable, C consulted, I informed): | Duty | Product team | Platform team | Security | FinOps / finance | Leadership | |---|---|---|---|---|---| | Runtime and dependency upgrades | R/A | C | I | — | I | | Provider notices triage | R | A | C | I | — | | Abuse and security notices | C | I | R/A | — | I | | Quota headroom | R | A | — | — | I | | Access reviews and secret rotation | R | C | A | — | I | | Backup and restore tests | R/A | C | I | — | I | | Provider-incident response | R/A | C | C | — | I | | Cost review and anomalies | R | C | — | A | I | | Exit-cost estimate and vendor review | C | R | C | C | A | ### Lifecycle: Day 0, Day 1, Day 2 | Phase | Activities | Exit criteria | |---|---|---| | **Day 0: decide and design** | Workload profile; register; work-and-coverage estimate; dependency availability; exit estimate; ADR | ADR approved; every duty owned; coverage agreed | | **Day 1: adopt and launch** | IaC and guardrails; signals; budgets; quota alarms; SLIs; restore test; failure runbook | Launch checklist passed; first restore done; rota live | | **Day 2: operate and improve** | Quarterly review; currency, quota and cost trends; game days | Decision reaffirmed or changed | | **Exit (when needed)** | Export data; parallel run; decommission | Workload moved; lessons recorded | ### Operating model and governance - **Intake gate** ([template](#templates)): what moves, what remains, who owns it, the failure plan, the exit cost. - **A responsibility register** per service, with the code ([format](#reference-implementation)), reviewed quarterly. - **Signal routing** to the owning team's queue, not a shared mailbox. - **Escalation:** product team → platform team → provider support, plus a second path, since AWS support case creation was impaired in December 2021 and October 2025.[^aws2021][^aws2025] - **Change control:** provider-driven changes (deprecations, plan changes) are planned work with deadlines. ::: decision Governance proportionality A five-person startup needs the register, the signals and a monthly 30-minute review; an enterprise also needs the intake gate and an owner for the registers. Scale ceremony to the cost of an unowned duty. ::: ## Architecture and workflow ### Reference architecture ::: figure Reference architecture: a managed-first workload and the control loops you keep ```mermaid flowchart TB U["Users"] --> E["Managed edge and API"] L1(["Yours: IaC and guardrails"]) --> E L1 --> C E --> C["Managed functions or containers"] E -.-> L2(["Yours: SLIs and on-call"]) C --> D[("Managed data and queues")] C -.-> L3(["Yours: provider, quota, cost signals"]) L4(["Yours: backups, restore tests"]) --> D classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029 class U,E,C,D neutral class L1,L2,L3,L4 core ``` Text equivalent: Users reach a managed edge and API layer, then managed functions or containers, then managed data and queues (grey, provider-operated). Four customer-owned loops (green, labelled "Yours") act on or receive signals from what they govern: IaC and guardrails act on the API layer and compute; SLIs and on-call watch the edge; provider, quota and cost signals come from compute; backups and restore tests act on the data. ::: Your side also includes throttling at the edge (an API layer can accept more requests than the functions behind it can run[^lambdaquota]), backup and versioning strategy,[^warel] and idempotency in queues. **Alternative:** at stable high load, reserved containers or managed Kubernetes, "a middle ground in this continuum".[^berkeley2019] ### End-to-end workflow The core workflow **estimates and assigns residual operations**; in its exception path, an unownable duty changes the design. ::: figure Residual-operations estimation workflow with exception path ```mermaid flowchart TB T["Trigger: new service, migration or a NoOps proposal"] --> P["Profile workload: load, state, latency, regulation"] P --> S["Choose candidate building blocks"] S --> R["Map responsibility line using provider models"] R --> I["List residual duties with frequency and effort"] I --> O{"Every duty has an owner?"} O -- "Yes" --> E["Estimate work hours and coverage"] O -- "No" --> X{"Can we buy, automate or drop it?"} X -- "Yes" --> I X -- "No" --> RD["Redesign or choose another building block"] RD --> S E --> F{"Fits team capacity and risk appetite?"} F -- "Yes" --> A["Record ADR, wire signals, launch"] F -- "No" --> RD A --> M["Measure residual hours, pages, currency, cost"] M --> Q["Quarterly review"] Q --> I classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029 classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029 class T,P,S,R,I,E,A,M,Q core class O,X,F decision class RD risk ``` Text equivalent: A trigger leads to profiling the workload, choosing building blocks, mapping the responsibility line and listing residual duties. An unowned duty is bought, automated or dropped, or the design changes. With every duty owned, the team estimates work and coverage; if that fits capacity and risk appetite it records an ADR, wires signals and launches, otherwise it redesigns. Measurement and a quarterly review feed back into the duty list. ::: ## Practices, frameworks and tools ### Core practices and playbooks The recurring practices (responsibility mapping, intake, signal routing, runtime currency, quota headroom, data protection, failure design, cost guardrails, exit readiness, automation ownership) have owners in the [RACI](#roles-ownership-and-raci), measures in the [KPIs](#metrics-and-kpis) and a cadence in the [checklists](#checklists). Intake has the most leverage, so it gets a playbook. #### Playbook: adopting a managed service 1. **Map the line** from that specific service's shared responsibility documentation; it differs by service.[^lambdasec] 2. **List and estimate residual duties** in the [register](#reference-implementation): work and coverage. 3. **Wire signals and rehearse** one restore and one provider-failure drill before launch. 4. **Accept** when every duty has an owner, signals are tested and the exit cost is in the ADR. 5. **Contingency:** keep the previous deployment path for one review cycle and a restored data export. ### Methods, standards and frameworks There is no NoOps standard. The decision draws on NIST SP 800-145 for service models;[^nist145] the providers' shared responsibility and resiliency models for where the line sits (guidance, not contract: Microsoft calls its page "illustrative guidance");[^awssrm][^azuresrm][^gcpsrm][^warel] SLAs, which are contracts for service credits;[^slalambda][^sreslo] the Google SRE book and *The Calculus of Service Availability* for toil, on-call and dependencies;[^sretoil][^sreoncall][^calculus2017] Team Topologies for team shapes;[^teamtop] and, in the EU, the Data Act (Regulation (EU) 2023/2854) on switching.[^dataact] ### Tooling landscape Start with the jobs ([building blocks](#building-blocks-of-a-low-ops-system)), then judge products on limits, runtime policy, restart semantics, recovery, export paths and the vendor's roadmap. Representative options as of October 2026: functions (AWS Lambda, Azure Functions); serverless containers and PaaS (Google Cloud Run, "a fully managed application platform";[^cloudrun] Azure App Service; Heroku, which announced in February 2026 that it is transitioning to a sustaining engineering model and has since shipped some updates[^heroku2026][^herokuoct2026]); managed Kubernetes (Google Kubernetes Engine[^gkesrm]); managed data (Amazon DynamoDB, Azure SQL Database). **Lock-in:** function semantics and back-end services differ between providers;[^berkeley2019] portable layers cut exit cost but add operations. ### Provider health signals compared Each feed is a duty to route and read (as of October 2026); organisation-wide wiring belongs to [CloudOps](../cloudops/). | Provider | Signal | How to route it | Caveats | |---|---|---|---| | AWS | AWS Health events in four categories (`issue`, `accountNotification`, `scheduledChange`, `investigation`), all of which AWS advises monitoring; abuse notifications arrive as a separate event type[^healthschema][^healthconcepts][^ebhealthref] | EventBridge rules per Region: one for health events, one for abuse events ([Lab 2](#hands-on-labs))[^healtheb] | Backup copies go to US West (Oregon); global events only to US East (N. Virginia)[^healthregion] | | Microsoft Azure | Service Health: service issues, planned maintenance, health advisories, security advisories[^azsh] | Service Health alerts with action groups (e-mail, SMS, push, webhook, ITSM)[^azsh] | Action group region must be "Global"; Resource Health alerts are separate[^azsh] | | Google Cloud | Personalized Service Health: incidents relevant to your projects or organisation[^gcpsh] | Alerts on the events it logs in Cloud Logging; an API[^gcpsh] | Check that your products and locations are supported[^gcpsh] | ### Automation and AI Automation is a destination of displaced work, so operate it: an unowned rule or script becomes a hidden failure mode ([AutomationOps](../automationops/)). It clearly helps with signal routing, currency scans, guardrails as code and provider-assisted upgrades (AWS points Lambda customers moving to Java 21 or 25 to AWS Transform custom[^lambdart]), whose output still needs review. Much triage needs no model: AWS Health events can carry an `actionability` value.[^healthschema] Any agent with change permissions needs an owner, least privilege, audit logs and a kill switch; changes to data, identity and spend keep human approval. AI agents sold as autonomous operations are assessed [below](#advanced-topics-and-emerging-direction). ## NoOps in practice ### Reference implementation The core artefact is a **responsibility register with an estimator**, one per service, next to its code: each duty has an owner, frequency and effort; `coverage` says when someone must be reachable. **Assumptions (illustrative):** `orders-api` runs on a managed API layer, functions and a managed NoSQL table, with business-hours paging and 120 productive hours per person per month. _Example status: verified by execution (Python 3.8.10, 2026-10-16) together with the script below._ ```json { "service": "orders-api", "assumptions": {"productive_hours_per_person_month": 120}, "duties": [ {"id": "runtime-upgrades", "area": "runtime and dependencies", "owner": "orders-team", "per_month": 2, "hours_each": 3}, {"id": "provider-notices", "area": "provider change notices", "owner": "orders-team", "per_month": 4, "hours_each": 0.5}, {"id": "quota-review", "area": "quotas and limits", "owner": "platform-team", "per_month": 1, "hours_each": 1}, {"id": "cost-review", "area": "cost", "owner": "orders-team", "per_month": 1, "hours_each": 3}, {"id": "cost-anomalies", "area": "cost", "owner": "orders-team", "per_month": 0.5, "hours_each": 2}, {"id": "access-review", "area": "identity and access", "owner": "security-team", "per_month": 1, "hours_each": 2}, {"id": "secret-rotation", "area": "identity and access", "owner": "", "per_month": 0.33, "hours_each": 2}, {"id": "restore-test", "area": "data protection", "owner": "orders-team", "per_month": 0.33, "hours_each": 4}, {"id": "alert-tuning", "area": "observability", "owner": "orders-team", "per_month": 2, "hours_each": 1}, {"id": "provider-incidents", "area": "incident response", "owner": "orders-team", "per_month": 0.25, "hours_each": 4}, {"id": "compliance-evidence", "area": "compliance", "owner": "security-team", "per_month": 1, "hours_each": 2}, {"id": "vendor-admin", "area": "vendor management", "owner": "", "per_month": 1, "hours_each": 1.5}, {"id": "pipeline-iac-upkeep", "area": "delivery pipeline", "owner": "platform-team", "per_month": 2, "hours_each": 2} ], "on_call": {"owner": "orders-team", "pages_per_week": 1.5, "hours_per_page": 1.5}, "coverage": {"reachable_hours_per_week": 45, "people_per_shift": 1, "max_share_of_weeks_on_call": 0.25}, "projects": [ {"id": "exit-plan-and-game-day", "owner": "orders-team", "hours": 24, "amortise_months": 12} ] } ``` The estimator reports monthly **work** per duty and owner, and **coverage** (person-hours of being reachable, minimum rota); it exits with status 1 when a duty is unowned. _Example status: verified by execution (Python 3.8.10, 2026-10-16); the output below is the actual output._ ```python """Estimate the operations that remain after adopting managed services: work and coverage. Usage: python3 residual_ops.py residual-ops.json Standard library only. Prints monthly work hours per duty and per owner, an FTE equivalent, and the coverage cost: person-hours of being reachable and the minimum rota size. Exit code 1 if any duty has no owner. """ import json import math import sys from collections import defaultdict WEEKS_PER_MONTH = 52 / 12 def owner_of(item): return (item.get("owner") or "").strip() or "UNOWNED" def work_rows(inv): rows = [] for d in inv.get("duties", []): rows.append((d["id"], owner_of(d), float(d["per_month"]) * float(d["hours_each"]))) oc = inv.get("on_call") if oc: hours = float(oc["pages_per_week"]) * WEEKS_PER_MONTH * float(oc["hours_per_page"]) rows.append(("on-call-pages", owner_of(oc), hours)) for p in inv.get("projects", []): rows.append((p["id"] + " (amortised)", owner_of(p), float(p["hours"]) / max(1, int(p["amortise_months"])))) return sorted(rows, key=lambda r: r[2], reverse=True) def main(path): with open(path, encoding="utf-8") as fh: inv = json.load(fh) rows = work_rows(inv) total = sum(r[2] for r in rows) per_owner = defaultdict(float) for _, owner, hours in rows: per_owner[owner] += hours capacity = float(inv["assumptions"]["productive_hours_per_person_month"]) print(f"Residual operations estimate for {inv['service']}") print("WORK") for duty, owner, hours in rows: print(f" {duty:<34} {owner:<14} {hours:6.1f} h/month") print(f" {'TOTAL':<34} {'':<14} {total:6.1f} h/month") print(f" FTE equivalent (at {capacity:.0f} productive h/person-month): {total / capacity:.2f}") for owner, hours in sorted(per_owner.items(), key=lambda kv: kv[1], reverse=True): share = 100 * hours / total if total else 0.0 print(f" owner {owner:<20} {hours:6.1f} h/month ({share:4.1f}%)") cov = inv.get("coverage") if cov: reach = float(cov["reachable_hours_per_week"]) * int(cov["people_per_shift"]) * WEEKS_PER_MONTH rota = math.ceil(int(cov["people_per_shift"]) / float(cov["max_share_of_weeks_on_call"])) print("COVERAGE") print(f" reachable {cov['reachable_hours_per_week']} h/week x {cov['people_per_shift']} per shift" f" = {reach:.0f} person-hours/month on call") print(f" minimum rota at <= {float(cov['max_share_of_weeks_on_call']):.0%} of weeks on call: {rota} people") if per_owner.get("UNOWNED"): print(f"WARNING: {per_owner['UNOWNED']:.1f} h/month of duties have no owner") return 1 return 0 if __name__ == "__main__": if len(sys.argv) != 2: sys.exit("usage: python3 residual_ops.py ") sys.exit(main(sys.argv[1])) ``` ```text $ python3 residual_ops.py residual-ops.json Residual operations estimate for orders-api WORK on-call-pages orders-team 9.8 h/month runtime-upgrades orders-team 6.0 h/month pipeline-iac-upkeep platform-team 4.0 h/month cost-review orders-team 3.0 h/month provider-notices orders-team 2.0 h/month access-review security-team 2.0 h/month alert-tuning orders-team 2.0 h/month compliance-evidence security-team 2.0 h/month exit-plan-and-game-day (amortised) orders-team 2.0 h/month vendor-admin UNOWNED 1.5 h/month restore-test orders-team 1.3 h/month quota-review platform-team 1.0 h/month cost-anomalies orders-team 1.0 h/month provider-incidents orders-team 1.0 h/month secret-rotation UNOWNED 0.7 h/month TOTAL 39.2 h/month FTE equivalent (at 120 productive h/person-month): 0.33 owner orders-team 28.1 h/month (71.6%) owner platform-team 5.0 h/month (12.7%) owner security-team 4.0 h/month (10.2%) owner UNOWNED 2.2 h/month ( 5.5%) COVERAGE reachable 45 h/week x 1 per shift = 195 person-hours/month on call minimum rota at <= 25% of weeks on call: 4 people WARNING: 2.2 h/month of duties have no owner ``` **How to read it.** About 39 hours a month, a third of a person, is the *work*; unowned duties are how "NoOps" stacks fail quietly. **Coverage** is larger: 195 person-hours a month of being reachable, and at least four people if nobody is on call for more than a quarter of their weeks. Google caps SRE on-call at 25% of *time* and, assuming week-long shifts, derives a minimum of eight engineers for a single-site 24/7 rota with a primary and a secondary;[^sreoncall] on business-hours rotas, where people do project work between pages, a cap on *weeks* is a conservative proxy. **Check the inputs:** Google reports about 6 hours per on-call incident;[^sreoncall] at that rate the total becomes about 68.5 hours (0.57 FTE). ### Hands-on labs #### Lab 1 (beginner): build and estimate a responsibility register **Objective:** a work-and-coverage estimate for one service, with unowned duties found. **Prerequisites:** Python 3.8 or later; no cloud account, cost or risk; under an hour. 1. Save the [reference](#reference-implementation) register as `residual-ops.json` and the estimator as `residual_ops.py`. 2. Run `python3 residual_ops.py residual-ops.json`. 3. Give both unowned duties an owner (for example `orders-team`) and run again. 4. Change `coverage` to `{"reachable_hours_per_week": 168, "people_per_shift": 2, "max_share_of_weeks_on_call": 0.25}` (24/7, primary and secondary) and run again. 5. Replace the example with your own service's duties. **Expected output** (checked by execution). Step 2: the reference output; `echo $?` prints `1`. Step 3: no warning, exit 0, total 39.2 h/month, `orders-team` 30.2 h/month (77.1%). Step 4: work unchanged; coverage 1,456 person-hours a month, minimum rota 8. **Validation:** every duty has an owner, and the people doing the work find the total plausible. **Troubleshooting:** `JSONDecodeError` is often a trailing comma. **Teardown:** delete the directory, or commit the register. #### Lab 2 (intermediate): audit and test provider-signal routing **Objective:** prove that every class of provider signal reaches an owner; test two AWS rules without deploying anything. **Prerequisites:** read access to your providers' health settings; for step 3, AWS CLI v2 with credentials. No cost or risk. **Status:** step 3 not executed. 1. **Inventory.** Per account or project, list the signal classes in the [comparison table](#provider-health-signals-compared) plus quota and budget alerts, and where each goes: a team queue, a person, a shared mailbox or nowhere. 2. **Trace.** Who acknowledged the latest notice of each class, and how long after it arrived? 3. **Test the AWS rules.** Save the two patterns below as `pattern-ops.json` and `pattern-abuse.json` and the sample event as `sample-health-event.json`, then run the script; it evaluates patterns without sending anything.[^clitest] 4. **Record.** Every class that reaches nobody, or only a shared mailbox, becomes an owned duty in your register. **Why two rules.** AWS Health sends two event types to EventBridge, `AWS Health Event` and `AWS Health Abuse Event`,[^ebhealthref] and for security use cases AWS suggests focusing on "all AWS Health Abuse events".[^healthrule] The operations rule filters no category, since AWS calls it "a best practice that you monitor all event type categories";[^healthconcepts] route pages on the optional `detail.actionability` field.[^healthschema] It keeps only events whose `backupEvent` is "false", which drops the copies AWS also delivers to the backup Region, as AWS suggests.[^healthregion] The abuse rule does not: the schema marks `backupEvent` as required for both types,[^healthschema] but no abuse-event sample confirms it, so the worst case is a duplicate. Region coverage and deployment as code belong to [CloudOps](../cloudops/). _Example status: JSON syntax-checked; both patterns matched against the sample and its abuse variant with a local exact-match simulation (2026-10-16); not deployed; no AWS API called._ ```json { "source": ["aws.health"], "detail-type": ["AWS Health Event"], "detail": {"backupEvent": ["false"]} } ``` ```json { "source": ["aws.health"], "detail-type": ["AWS Health Abuse Event"] } ``` ```json { "version": "0", "id": "7bf73129-1428-4cd3-a780-95db273d1602", "detail-type": "AWS Health Event", "source": "aws.health", "account": "123456789012", "time": "2026-10-16T09:00:00Z", "region": "eu-west-1", "resources": ["arn:aws:lambda:eu-west-1:123456789012:function:orders-api"], "detail": { "eventArn": "arn:aws:health:eu-west-1::event/LAMBDA/AWS_LAMBDA_PLANNED_LIFECYCLE_EVENT/AWS_LAMBDA_PLANNED_LIFECYCLE_EVENT_EXAMPLE", "service": "LAMBDA", "eventTypeCode": "AWS_LAMBDA_PLANNED_LIFECYCLE_EVENT", "eventTypeCategory": "scheduledChange", "eventScopeCode": "ACCOUNT_SPECIFIC", "communicationId": "example-communication-1", "startTime": "Fri, 30 Apr 2027 00:00:00 GMT", "lastUpdatedTime": "Fri, 16 Oct 2026 09:00:00 GMT", "statusCode": "upcoming", "eventRegion": "eu-west-1", "eventDescription": [{"language": "en_US", "latestDescription": "Example: a runtime used by this function reaches end of support."}], "affectedEntities": [{"entityValue": "arn:aws:lambda:eu-west-1:123456789012:function:orders-api", "lastUpdatedTime": "Fri, 16 Oct 2026 09:00:00 GMT", "status": "PENDING"}], "page": "1", "totalPages": "1", "backupEvent": "false", "affectedAccount": "123456789012", "actionability": "ACTION_REQUIRED", "personas": ["OPERATIONS"] } } ``` _Example status: syntax-checked with `bash -n`; not executed (requires AWS credentials)._ ```bash #!/usr/bin/env bash set -euo pipefail export AWS_REGION="eu-west-1" # any Region; the test creates nothing # Same event with the abuse detail-type, to prove each rule catches only its own type. sed 's/"AWS Health Event"/"AWS Health Abuse Event"/' sample-health-event.json > sample-abuse-event.json for pattern in pattern-ops.json pattern-abuse.json; do for event in sample-health-event.json sample-abuse-event.json; do printf '%s + %s: ' "$pattern" "$event" aws events test-event-pattern --event-pattern "file://$pattern" --event "file://$event" \ --query Result --output json done done ``` **Expected output** (local simulation; the AWS call was not made): `true` for each rule with its own event type, `false` for the crossed pairs. **Validation:** with `eventTypeCategory` set to `accountNotification` the operations rule still matches; with `backupEvent` "true" it does not; steps 1–2 pass when every signal class has a named owner. **Troubleshooting:** `InvalidEventPatternException` means an invalid pattern;[^apitest] an event error, a missing mandatory field (`id`, `account`, `source`, `time`, `region`, `resources`, `detail-type`).[^clitest] **Teardown:** delete the working files; nothing was created. #### Lab 3 (advanced): provider-failure and exit tabletop **Objective:** test whether a managed-first service survives a hard dependency's failure and whether its exit cost is known. **Prerequisites:** two hours with the team, residual-ops owner, security and finance; the dependency list; no production changes. 1. **Availability.** Compare your hard dependencies' SLA product and measured product with your SLO ([worked calculation](#metrics-and-kpis)). 2. **Scenario A, regional disruption.** "The primary Region's managed database endpoint stops resolving; the status page has not updated." Walk through the [worked runbook](#common-failures-and-troubleshooting). 3. **Scenario B, forced upgrade.** "40% of functions use a runtime deprecated in 60 days; a security fix ships next week." (See the deprecated-runtime risk under [security](#quality-security-privacy-and-compliance).) 4. **Scenario C, exit.** "A service you use will be retired in a year", as Google Cloud IoT Core was in 2023.[^iotcore2022] Estimate the exit in person-weeks. **Expected output:** an availability comparison, runbook gaps, a dated exit estimate. **Validation:** every gap has an owner and a date (an unknown dependency list is the first finding). **Teardown:** none; follow up in 30 days. ### Use cases and scenarios #### Case 1: Netflix's "NoOps" was reorganised operations (sourced) In 2012 Cockcroft wrote that "There is no ops organization involved in running our cloud": developers deployed through a self-service portal and carried the pager, a reliability team would "never make changes to production applications", and transferred ITops staff built base images. Yet "Netflix stil [sic] does Ops to run its datacenter DVD business."[^cockcroft2012] **Lesson:** the label rested on a platform, operations skills and developer on-call. #### Case 2: Prime Video's serverless design did not fit its load (sourced, secondary) InfoQ reports that Prime Video's stream monitoring on AWS Step Functions and Lambda handled only "around 5% of the expected load", hit an account limit on state transitions and was costly. The team "packed all of the components into a single process" on Amazon ECS on EC2 and reported "a 90% reduction in operational costs", meaning running cost, not operations work (the original post has moved).[^primevideo2023] **Lesson:** the building block must fit the workload; this page adds that the team accepted more infrastructure to run for a lower unit cost. #### Case 3: a fully managed SaaS still left identity with the customer (sourced) In 2024 Mandiant investigated a campaign (UNC5537) that stole data from Snowflake customer instances. It "has not found any evidence" of a breach of Snowflake's enterprise environment; accounts "were not configured with multi-factor authentication", unrotated credentials stayed valid for years, and network allow lists were missing.[^mandiant2024] **Lesson:** SaaS moves the line almost to the top, but identity hygiene stays yours. ## Measuring, securing and paying for what remains ### Metrics and KPIs | KPI | Definition / formula | Pitfall | |---|---|---| | Residual work | Σ(occurrences × hours each) + pages per week × 52/12 × hours per page + Σ(project hours ÷ amortisation months), per month | Drifts low | | Coverage requirement | Reachable hours per week × people per shift; rota = people per shift ÷ maximum share of weeks on call | Confused with work | | Unowned duty share | Unowned hours ÷ residual work (target 0) | A team name with no person | | Page load per shift | Incidents ÷ shifts; Google's upper bound is two per 12-hour shift[^sreoncall] | Counting alerts, not incidents | | Runtime currency | Function versions ≥ 180 days from deprecation ÷ all versions (Trusted Advisor or API) | Notices list only `$LATEST` and skip container images[^lambdart] | | Quota headroom | 1 − (peak usage ÷ quota), per limiting quota | Checking one quota only[^lambdaquota] | | Dependency availability | SLA-implied: Π(SLA); measured: Π(measured availability) | Treating the SLA product as a ceiling | | Restore readiness | Days since the last timed restore, per store | Backups never restored | | Total unit cost | (Bill + residual work × loaded hourly rate) ÷ business units (residual work includes amortised exit planning) | Leaving people time out | Review work, coverage and unit cost quarterly, the rest monthly; ownership, currency, headroom and restores are leading indicators. ::: figure Measurement loop for the operations that remain ```mermaid flowchart TB M["Measure: residual hours, pages, currency, headroom, unit cost"] --> C{"Classify each duty"} C -- "Buyable" --> B["Buy"] C -- "Repetitive" --> A["Automate, with an owner"] C -- "Unneeded" --> X["Eliminate"] C -- "Essential" --> K["Keep and rehearse"] B --> V["Verify the duty moved and nothing new appeared"] A --> V X --> V K --> V V --> M classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029 class M,B,A,X,K,V core class C decision ``` Text equivalent: Measure residual hours, pages, currency, headroom and unit cost; buy, automate (with an owner), eliminate, or keep and rehearse each duty; verify that it moved and that nothing new appeared, such as automation to maintain; measure again. ::: #### Worked calculations **1. SLA-implied versus measured availability.** A request path uses a managed API layer (SLA 99.95% monthly), functions (99.95%) and a managed NoSQL table (99.99% for standard tables).[^slaapigw][^slalambda][^sladdb] If each ran exactly at its SLA with independent failures: 0.9995 × 0.9995 × 0.9999 ≈ **99.89%**, about **47.5 minutes** of unavailability in a 30-day month, against 21.6 minutes for one 99.95% dependency. This is *not* a ceiling: AWS calls such a product "only likely to produce a rough order of magnitude calculation" and notes that a workload on two or three 99.99% dependencies "can still achieve 99.99% availability itself, or higher".[^awsavail] The *Calculus* ceiling (principle 4) applies to *actual* availability:[^calculus2017] the SLA product shows what the contracts support, the measured product what users get. If the latter misses your target, add mitigations (caches, graceful degradation) or dependencies with an extra nine.[^calculus2017] **2. Total unit cost (illustrative).** Bill 1,800 currency units a month; 39.2 residual hours at a loaded 90 an hour (3,528); 30 million requests: (1,800 + 3,528) ÷ 30 = **177.6 per million requests**, 66% of it people time. The 39.2 hours already include 2 hours a month of amortised exit planning, so no separate exit reserve is added. Here people time outweighs the bill, which is why the "NoOps" pitch is really about people; at high, steady volume the balance can flip ([Case 2](#use-cases-and-scenarios)). **Vanity traps:** "servers we manage: 0", "percentage serverless", "operations headcount", "provider SLA met". ### Quality, security, privacy and compliance | Risk | Likelihood / impact | Mitigation | Owner | |---|---|---|---| | Credential-only access to a SaaS or data platform | Plausible; high ([Case 3](#use-cases-and-scenarios)) | MFA or SSO; rotation; allow lists | Security | | Over-privileged function, automation or agent roles | Common; medium–high | Least privilege per function or agent | Product team | | Deprecated runtimes | Common; medium–high | Upgrade; after deprecation AWS "may no longer apply security patches"[^lambdart] | Product team | | Data loss from a bad write | Plausible; high | Backup and versioning strategy are yours[^warel] | Product team | | Abuse notice unread | Plausible; high | Route abuse events to security ([Lab 2](#hands-on-labs)) | Security | | Data in an unexpected jurisdiction | Plan-dependent | Choose Regions deliberately | Security / legal | **Auditability.** Provider attestations cover the provider's scope;[^lambdasec] evidence for *your* configuration is a recurring duty ([ComplianceOps](../complianceops/)); EU switching rules are under [exit cost](#cost-capacity-and-sustainability). ### Cost, capacity and sustainability **Cost drivers change shape.** The Berkeley paper names "Paying in proportion to resources used instead of for resources allocated" as a defining serverless trait and "Serverless computing can have unpredictable costs" as a pitfall;[^berkeley2019] at high, steady volume, per-use charges can exceed reserved capacity.[^primevideo2023] **Capacity becomes quota management.** AWS documents 1,000 concurrent Lambda executions per Region by default and API Gateway's default of 10,000 requests per second; it recommends load testing.[^lambdaquota] **Exit cost is a liability** that grows with proprietary code and data, and providers change on their own schedule: Google Cloud IoT Core was discontinued in August 2023, and Heroku's February 2026 announcement changed its roadmap.[^iotcore2022][^heroku2026] In the EU (not legal advice), the European Commission states that the Data Act removes switching charges, including data egress charges for switching, from 12 January 2027 (cost-based charges are allowed until then);[^dataact] that cuts transfer costs, not engineering work. **Sustainability.** AWS lists "Use managed services" as a sustainability design principle, because shared services maximise utilisation;[^wasus] treat it as directional ([GreenOps](../greenops/)). ### Assurance of the operating model Service monitoring belongs to [ObservabilityOps](../observabilityops/); this asks whether the residual duties get done. - **Register audit.** Quarterly, sample three duties per service and check their evidence. - **Risk indicators** that open an action (this Atlas's thresholds): any unowned duty; headroom below 20%; restore older than 90 days; provider notice unacknowledged after 14 days; exit estimate older than 12 months. - **Signal health.** Test the signal path twice a year; health events arrive at least once, so de-duplicate.[^healtheb] ## When things go wrong ### Common failures and troubleshooting | Symptom | Likely cause and diagnosis | Remediation | Prevention | |---|---|---|---| | A fix cannot be deployed to some functions | Runtime past deprecation[^lambdart] | Upgrade, test, deploy | Currency metric | | Throttling at peak | Regional concurrency quota; check `ClaimedAccountConcurrency` (Max)[^lambdametrics] | Quota increase; queueing | Headroom alarm; load tests | | Bill spikes overnight | Runaway retries or recursion | Stop the loop; cap concurrency | Budgets and anomaly alerts | | Data overwritten, no way back | Backup assumed to be the provider's job | Restore what exists | Timed restore tests[^warel] | | Users see errors, status page green | Status lags, or your configuration | Treat it as yours | Your own SLIs[^aws2021] | #### Worked runbook: a managed dependency fails An **illustrative scenario** modelled on documented events. In October 2025 an empty DNS record for DynamoDB in us-east-1 caused API errors from 11:48 PM PDT on 19 October to 2:40 AM on 20 October; Lambda event processing and EC2, ECS, EKS and Fargate launches stayed impaired for hours, and the whole event ended at 2:20 PM PDT.[^aws2025] In December 2021 launches and DNS changes were impaired while running EC2 instances were unaffected.[^aws2021] ::: figure Handling a provider incident that becomes your incident ```mermaid sequenceDiagram autonumber participant U as Users participant S as Service participant O as On-call participant P as Provider U->>S: Requests S-->>U: Errors, latency S->>O: SLO burn alert O->>P: Check health feed O->>O: Declare, assign roles alt Tested multi-Region design O->>S: Fail over to another Region else No tested failover O->>S: Degrade: queue writes, serve cache end O-->>U: Status page update O->>P: Open case if possible P-->>O: Recovered opt Service failed over O->>S: Fail back end O->>S: Drain queues, watch throttling O-->>U: Resolved, review scheduled ``` Text equivalent: Users see errors; an SLO burn alert reaches on-call, who checks the provider's health feed and declares an incident. With a tested multi-Region design the service fails over; otherwise it degrades (queued writes, cached reads). On-call updates the status page and opens a provider case if possible. After recovery, a service that failed over fails back; either way queues are drained while throttling is watched, and users are told it is resolved. ::: 1. **Detect** from your SLO alert or synthetic check; the absence of a provider event is not evidence of health. 2. **Declare:** incident commander and communications lead ([IncidentOps](../incidentops/)); **scope** dependency, Region and user journeys from the register. 3. **Mitigate:** degrade by feature flag, queue writes, serve cached reads, pause batch jobs, break retry storms. Fail over only to a tested design, and avoid actions that need the affected Region's control plane, which was impaired in both events.[^aws2021][^aws2025] 4. **Communicate** every 30 minutes on your own status page. 5. **Recover:** fail back if you failed over; drain queues idempotently and watch throttling. In 2025, downstream recovery took hours longer than the DNS fault.[^aws2025] 6. **Learn:** separate the provider's root cause from your contributing factors; AWS places testing, including "game day practices", on the customer's side.[^warel] ### Anti-patterns and myths | Anti-pattern or myth | Why it fails | Warning signal | Do instead | |---|---|---|---| | "NoOps by announcement": disbanding the ops team on migration day | Duties land on people without the skills or time | Pages bounce between teams | Redeploy operations skills, as Netflix did[^cockcroft2012] | | "The provider's SLA is our SLO" | An SLA is a contract, not a measurement[^sreslo] | SLO equals one provider SLA | Set the SLO from user needs | | "Managed means backed up" | Backup strategy remains yours[^warel] | No restore on record | Time a restore | | "We can always migrate later" | Proprietary wiring and data gravity accumulate[^iotcore2022] | No exit estimate in any ADR | Exit estimate; tested export | | "The AI agent is our ops team" | Automation fails at scale too; someone owns its permissions and mistakes[^sreauto] | Agent with write access, no owner | Review mode, least privilege, kill switch | ## Deciding and adopting ### When to use and when not to use **Pursue "less ops" (managed-first) when most of these hold:** - A common workload shape (web, API, events, batch) with spiky or low load, where pay-per-use and scale-to-zero help.[^berkeley2019][^cloudrun] - A small team whose scarce skill is product engineering; limits that fit with headroom; an exit cost you accept. **Avoid or limit it when:** - Load is high and steady, so reserved capacity is probably cheaper.[^primevideo2023] - The workload needs exotic hardware, kernel settings, or runtime and maintenance control the provider lacks. - Latency must be consistently low. The Berkeley paper calls it a fallacy that functions *cannot* serve low-latency applications, since they can be pre-warmed;[^berkeley2019] warm capacity is then a cost you pay. - A provider retirement would be existential and the exit cost is unknown, or "NoOps" would remove operations skills rather than redeploy them. **A counterpoint.** In 2022 David Heinemeier Hansson, co-owner and CTO of 37signals, called renting computers "(mostly) a bad deal" for medium-sized companies like his with stable growth: "The savings promised in reduced complexity never materialized".[^dhh2022] One company's view, but a useful test: if your operations team would not shrink and load is stable, the managed premium needs another justification. ### Decision framework ::: figure Decision tree: how much operations to buy for a workload ```mermaid %%{init: {"flowchart": {"wrappingWidth": 120, "nodeSpacing": 24, "rankSpacing": 36}}}%% flowchart TB S["Workload to place"] --> Q1{"Sold as SaaS?"} Q1 -- "Yes, data rules fit" --> SaaS["SaaS"] Q1 -- "No" --> Q2{"Fits managed limits?"} Q2 -- "No" --> SM["Self-managed or managed Kubernetes"] Q2 -- "Yes" --> Q3{"Load high and steady?"} Q3 -- "No" --> MF["Managed-first: PaaS or functions"] Q3 -- "Yes" --> Q4{"Cheaper on reserved capacity?"} Q4 -- "No" --> MF Q4 -- "Yes" --> CT["Reserved containers or Kubernetes"] SaaS --> Q5{"Duties owned, coverage affordable?"} SM --> Q5 MF --> Q5 CT --> Q5 Q5 -- "Yes" --> GO["Adopt; wire signals; record ADR and exit cost"] Q5 -- "No" --> RD["Not yet: redesign, cut coverage needs or staff up"] classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029 classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029 class S,SaaS,SM,CT,MF,GO core class Q1,Q2,Q3,Q4,Q5 decision class RD risk ``` Text equivalent: A standard business function sold as SaaS goes to SaaS if data rules fit (you keep identity, configuration, data and export). A workload outside managed limits (with headroom) goes self-managed or to managed Kubernetes. Spiky or low load goes managed-first (PaaS, serverless containers or functions); high, steady load goes to reserved containers or Kubernetes only if that is cheaper by total unit cost. Every option then passes one check: adopt only if the remaining duties are owned and coverage is affordable; otherwise redesign, reduce coverage needs or staff up. ::: When options are close, use a **weighted matrix (illustrative):** a spiky customer-facing API, five-person team, moderate compliance; scores 1 (poor) to 5 (good); weighted score = Σ(weight × score) ÷ 5, out of 100. | Criterion (weight) | Self-managed (VMs or Kubernetes) | Managed containers or PaaS | Functions + managed data | SaaS product | |---|---|---|---|---| | Team capacity for operations (25) | 1 | 4 | 5 | 5 | | Fit to load shape (20) | 3 | 4 | 5 | 4 | | Control and compliance needs (20) | 5 | 4 | 3 | 2 | | Total unit cost at expected volume (20) | 3 | 4 | 4 | 3 | | Exit cost tolerance (15) | 5 | 4 | 2 | 2 | | **Weighted score** | **64** | **80** | **79** | **67** | PaaS and functions tie and the extremes drop out; weight exit cost higher and PaaS wins, weight team capacity higher and functions do. Record the weights in the ADR. ### Maturity model These levels are **this Atlas's model**; no official one exists. The diagram adds proposed regression triggers, because this maturity is easily lost. ::: figure Maturity levels and the events that push an organisation back ```mermaid flowchart TB L0["Level 0: NoOps by assertion"] --> L1["Level 1: Owners named"] L1 --> L2["Level 2: Signals and recovery proven"] L2 --> L3["Level 3: Remainder engineered"] L3 --> L4["Level 4: Low-ops by design"] L2 -. "Unowned duty found in an incident" .-> L1 L3 -. "Restore or signal test fails" .-> L2 L4 -. "Service adopted without intake" .-> L3 L1 -. "Reorganisation leaves registers stale" .-> L0 classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029 classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029 classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 class L0 risk class L1,L2 neutral class L3,L4 core ``` Text equivalent: Solid arrows lead from Level 0 (NoOps by assertion) to Level 4 (low-ops by design). Dotted arrows show regressions: stale registers after a reorganisation (Level 1 to 0); an unowned duty found in an incident (2 to 1); a failed restore or signal test (3 to 2); a service adopted without intake (4 to 3). ::: | Level | Observable behaviour | Exit criteria | |---|---|---| | 0. NoOps by assertion | "The provider handles it"; duties found in incidents | Registers for the top five services | | 1. Owners named | Registers for top services; basic budgets and alerts | Every duty of every service has a named owner | | 2. Signals and recovery proven | Zero unowned duties; signals routed and tested; restore log | Runtime currency ≥ 95%; two quarterly reviews held | | 3. Remainder engineered | Residual hours and pages tracked; toil automated with owners; game days | Residual hours flat as services grow | | 4. Low-ops by design | Intake requires residual estimate, failure plan and exit cost | Sustain; periodic external review | Assess per product area; Level 2 suits most organisations. ### Adoption roadmap: 30/60/90 days For an organisation already running some managed services, led by a part-time residual-ops owner: | Window | Goals | Activities | Metrics at the end | |---|---|---|---| | Days 1–30: see the remainder | Duties visible | Registers for the top five services (Lab 1); dependency list; runtime inventory; SaaS accounts without MFA | Unowned hours; runtime currency | | Days 31–60: own and wire | Every duty owned; signals routed | Owners; signal audit (Lab 2); quota alarms; a timed restore per data store; intake form | Zero unowned duties; restore readiness | | Days 61–90: rehearse and decide | Resilience and reversibility | Tabletop (Lab 3); degraded-mode runbooks; exit estimates for the top three dependencies; first quarterly review | Residual baseline; exit estimates | ## Toolkit ### Checklists #### Preflight before adopting a managed service - [ ] Shared responsibility page summarised; quotas checked against peak, with headroom. - [ ] Residual duties owned and estimated; coverage stated; failure plan written. - [ ] Data location, backup and export path confirmed; exit cost in the ADR. - [ ] MFA or SSO, credential rotation and least-privilege roles designed in. #### Launch readiness - [ ] Signals deployed as code and tested in every used Region: health and abuse events, budgets, quota alarms, SLIs. - [ ] Synthetic checks independent of the provider's status page; rota and escalation path, including a second path. - [ ] One timed restore done; degrade-versus-fail-over rules and dependency-failure runbook written. #### Quarterly review - [ ] Unowned duties at zero; residual hours, coverage and pages against the trend. - [ ] Runtime currency across all versions; quota headroom; unit cost including people time. - [ ] Provider incidents and roadmap announcements reviewed. - [ ] Every automation and agent has an owner and a kill switch. ### Templates #### One-page low-ops charter ```text LOW-OPS CHARTER — Owner: Review: quarterly Mission: Buy or automate undifferentiated operations; own the remainder explicitly. Scope: Services . Out of scope: . Principles: Displacement not deletion · Always-retained band (data, identity, config, access) · Design for provider failure · Measure the remainder · Keep decisions reversible. Goals: 0 unowned duties · runtime currency >= % · restore test <= 90 days old · exit estimate <= 12 months old · residual work and coverage tracked per service. Coverage: Paging hours: . Rota: . Max share of weeks on call: %. Non-goals: Cutting operations skills; adopting abstractions for their own sake; treating provider SLAs as our SLOs. Decision rights: Adopting a new managed service requires an intake form and an ADR. ``` #### Managed-service intake form ```text MANAGED-SERVICE INTAKE — for Date: 1. Job to be done and alternatives considered: 2. Provider responsibility (link to provider's shared responsibility page): 3. Residual duties (area · owner · frequency · effort) and coverage needed: 4. Limits and quotas that could bind (value · our peak · headroom): 5. SLA · measured availability · our SLO · mitigation if it fails: 6. Data: classification · location · backup · export path tested? (Y/N) 7. Security: identity model · MFA/SSO · least-privilege roles · audit logs: 8. Cost: pricing basis · expected monthly cost · budget alert: 9. Exit cost: person-weeks · money · biggest unknown: 10. Decision · approver · review trigger: ``` #### Architecture decision record: buy versus run ```text ADR-: for Status: proposed | accepted | superseded by ADR- Date: Context: Workload profile; team capacity; compliance; volume forecast and horizon. Options: 1) ... 2) ... 3) ... (scores from the weighted matrix, with weights) Decision: