--- title: "PatchOps" slug: patchops category: security term_status: overlapping-synonym evidence_status: verified summary: "PatchOps is an informal name for patch management run as an operation: getting vendor fixes onto every asset through rings, maintenance windows, reboots and rollback, then proving they took effect. The established practice is enterprise patch management." audience: [Beginners, System and endpoint engineers, IT operations managers, Security engineers, Architects, Leaders] last_verified: 2026-10-17 key_takeaways: - "'PatchOps' is a rare label, not a discipline; the practice is enterprise patch management, which NIST describes as preventive maintenance and has documented since 2002." - "VulnerabilityOps decides what must be fixed and by when; PatchOps decides how a fix reaches every asset safely, and proves it is in effect." - "Plan per maintenance group and per scenario (routine, emergency, mitigation, unpatchable) in advance, so a new advisory needs a lookup, not a meeting." - "Rings are the test: a small canary group, gates on install failures, incidents and reporting rate, then wider rings, and forced installation once the deadline and grace period have passed." - "A patch is not done when it is installed: count it only after the restart that activates it, on an asset that is still reporting." - "Rollback is narrower than people assume; plan for roll-forward fixes, mitigations and restores as well as uninstalls." related: [vulnerabilityops, endpointops, sysops, itops, releaseops, containerops, secops, kubernetesops] --- ## What PatchOps is ### In 30 seconds **PatchOps** is an informal name for running patch management as an operation: knowing which assets need which vendor fix, rolling fixes out in rings and maintenance windows, handling restarts, backing out when a fix breaks something, and proving each fix is in effect. The established name is **enterprise patch management**, which NIST defines as "the process of identifying, prioritizing, acquiring, installing, and verifying the installation of patches, updates, and upgrades throughout an organization"[^sp800-40]. **In one line: PatchOps gets the fix onto every asset that needs it, on schedule, without causing the outage it was meant to prevent.** It is for engineers who run patch cycles, managers who own deadlines, and leaders deciding how much to patch in place at all. ### At a glance | Question | Answer | |---|---| | Goal | Every in-scope asset runs the fixes its maintenance plan requires, by the deadline, with disruption kept inside agreed windows | | Users | Server and endpoint engineers, IT operations and change managers, platform teams, security teams that set deadlines | | Scope on this page | Cadences, applicability, maintenance groups and plans, rings, windows, restarts and live patching, rollback, emergency patching, firmware and third-party software, evidence | | Out of scope | What to fix first → [VulnerabilityOps](../vulnerabilityops/); device enrolment → [EndpointOps](../endpointops/); image rebuilds → [ContainerOps](../containerops/); change approval → [ITOps](../itops/); host mechanics → [SysOps](../sysops/) | | Prerequisites | Basic server or endpoint administration ([SysOps](../sysops/) helps) | | Key outputs | Maintenance plans, rings, a cycle calendar, compliance reports counting only effective fixes, an exception register | | Term status | Overlapping synonym for patch management (see [What PatchOps is not](#what-patchops-is-not)) | ### The problem it solves Almost every fix needs someone to schedule disruption: a service restart, a reboot, a firmware flash. NIST states the tension: "Deploying patches more quickly reduces the window of opportunity for attackers but increases the risk of operational disruption", while testing first does the reverse and "still might miss problems". Because assets are so dynamic and dispersed, NIST adds, many organisations cannot keep up, and patching often becomes "primarily reactive"[^sp800-40]. Documented failures are rarely exotic. At Equifax in 2017, a company-wide fix missed the one portal attackers used ([use case 2](#use-cases-and-scenarios))[^gao-equifax]. When WannaCry hit the UK's National Health Service in May 2017, infected organisations "had unpatched, or unsupported Windows operating systems", although NHS Digital had issued critical alerts in March and April[^nao-wannacry]. ### What PatchOps is not - **Not a recognised discipline.** No standards body, certification or manifesto defines "PatchOps". The only use of the name found in October 2026 searches is a 2025 IEEE conference paper naming a Windows patching tool PatchOps[^amrita]. The practice is documented as patch management in NIST SP 800-40[^sp800-40], CIS Safeguards 7.3 and 7.4[^cis7] and, for industrial systems, IEC TR 62443-2-3[^iec-62443-2-3]. - **Not vulnerability prioritisation.** Deciding which flaws matter belongs to [VulnerabilityOps](../vulnerabilityops/); this page receives a prioritised item with a due date and delivers it. - **Not releasing your own software.** That is [ReleaseOps](../releaseops/); both use staged rollouts. - **Not device management in general.** Enrolment and configuration belong to [EndpointOps](../endpointops/), even when the same console deploys patches. - **Not only security updates.** Microsoft's monthly security updates also carry non-security fixes[^ms-cycle], and bug-fix, firmware and feature updates travel through the same rings. ::: note Term status and editorial distinction This Atlas classes PatchOps as an **overlapping synonym** for enterprise patch management. Its split of work with [VulnerabilityOps](../vulnerabilityops/) (what and by when), [SecOps](../secops/) (maintenance groups and timeframes) and [ITOps](../itops/) (change authority) is an **editorial distinction**; no standard defines it. ::: ## Where PatchOps comes from and where it sits ### Origins and evolution NIST's guide has existed since 2002 (revised in 2005, 2013 and April 2022); Revision 4 argues that organisations "would benefit more from rethinking their patch management planning than their patch management technology"[^sp800-40]. Four shifts shaped current practice: 1. **Vendor cadences.** Microsoft publishes cumulative security updates on the second Tuesday of each month, an optional preview in the fourth week, and out-of-band (OOB) releases "as needed"[^ms-cycle]. Oracle's quarterly cumulative Critical Patch Updates arrive on the third Tuesday of January, April, July and October; since May 2026, smaller Critical Security Patch Updates fill the third Tuesday of the other eight months, so an Oracle estate now has a monthly security lane beside the quarterly one (as of October 2026)[^oracle-cpu]. Linux repositories publish continuously. 2. **Cumulative updates.** Windows client updates "include all previously released fixes"[^ms-cycle], so one install catches a machine up, but one regression can now hold back a whole month of fixes. 3. **Cloud management.** Microsoft deprecated Windows Server Update Services (WSUS) in September 2024 in favour of cloud tools[^wsus]; AWS added organisation-wide patch policies in December 2022[^aws-policies]. 4. **Fewer reboots, more replacement.** Hotpatching and kernel live patching reduce restarts[^hotpatch][^livepatch], and image-based assets are patched by rebuilding them. ### Glossary and mental model | Term | Meaning | Why it matters | |---|---|---| | Patch, update, upgrade | A change to installed firmware, OS or application that fixes problems or adds capability[^sp800-40] | Major upgrades need separate planning | | Applicability | Whether a patch is relevant to an asset | Bad applicability data creates silent gaps | | Maintenance group | Assets with similar characteristics that share software maintenance needs in each risk response scenario[^sp800-40] | The unit you plan for | | Maintenance plan | Actions and timeframes for one group in one scenario[^sp800-40] | Turns an advisory into a lookup | | Ring | A wave of assets that receives an update before the next | The first ring is your test | | Deferral, deadline, grace period | Days before an update is offered; days after that before it is due; extra days after the deadline before the restart is forced | The three knobs of user-facing rollout | | Pending restart | Installed, not active until a restart | The most common false "compliant" | | KB number | Microsoft's knowledge base article number for an update (e.g. KB5124008) | The identifier in release notes, tickets and reports | | Exception | An approved, dated deviation from a plan | Must expire | **Mental model: three timelines.** The *vendor timeline* says when a fix exists. The *risk deadline*, set by [VulnerabilityOps](../vulnerabilityops/), says when it must be in effect. The *disruption calendar* (windows, freezes, restart tolerance) says when you may touch each asset. PatchOps fits the first into the third before the second runs out, and keeps the evidence. ### How PatchOps relates to other Ops ::: figure Where patch rollout sits among its neighbours ```mermaid flowchart LR V["VulnerabilityOps: what, by when"]:::neutral --> P S["SecOps: groups, exceptions"]:::neutral --> P I["ITOps: change authority"]:::neutral --> P P["PatchOps: rollout and proof"]:::core P --> SY["SysOps: servers"]:::neutral P --> EP["EndpointOps: devices"]:::neutral P --> CO["ContainerOps: rebuilds"]:::neutral P --> KO["KubernetesOps: nodes"]:::neutral P -.->|"verified status"| V classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029 ``` Text equivalent: VulnerabilityOps (what and by when), SecOps (maintenance groups, exceptions) and ITOps (change authority) feed PatchOps, which runs rollout, restarts and proof and returns verified status to VulnerabilityOps (dashed arrow). It delivers through SysOps (servers), EndpointOps (user devices), ContainerOps (image rebuilds) and KubernetesOps (node pools). ::: | Neighbour | Shared scope | This page | The neighbour | |---|---|---|---| | [VulnerabilityOps](../vulnerabilityops/) | Remediation | Delivery mechanics, rollout evidence | Prioritisation, due dates, exception policy | | [SecOps](../secops/) | Maintenance groups | Turning groups into plans and running them | Agreeing groups, timeframes and the register | | [ITOps](../itops/) | Change | Pre-approved standard changes per cycle | Change authority, configuration management database (CMDB) | | [SysOps](../sysops/), [EndpointOps](../endpointops/) | Same consoles and agents | Programme across the estate | Host and device operation | | [ContainerOps](../containerops/), [InfraOps](../infraops/) | Immutable assets | Deadlines and verification | Rebuilding images and hosts | ## Principles and operating model ### Charter and principles NIST's four principles are the closest thing the field has to a charter: "Problems are inevitable; be prepared for them", "Simplify decision making", "Rely on automation" and "Start improvements now". It also reframes the politics: "Disruptions from patching are largely controllable, while disruptions from incidents are largely uncontrollable"[^sp800-40]. **A one-page charter (this Atlas's proposal):** - **Mission:** every in-scope asset runs the fixes its maintenance plan requires, by the deadline, inside agreed disruption limits. - **Principles:** decide plans before advisories arrive; let early rings be the test; count only effective fixes; give every asset, even an unpatchable one, a maintenance group; rebuild image-based assets rather than repair them; make exceptions expire. - **Goals:** on-time effective coverage per group; emergency lead time; patch-caused incident rate. - **Non-goals:** choosing what to fix ([VulnerabilityOps](../vulnerabilityops/)), accepting risk ([RiskOps](../riskops/)), releasing your own software ([ReleaseOps](../releaseops/)). ### Fundamentals: the building blocks 1. **Inventory and applicability.** Continuous, automated discovery with authenticated scanning; of the old monthly or quarterly scan-based inventory, NIST says "That model should no longer be used"[^sp800-40]. 2. **Sources and policy.** Each vendor's rhythm and emergency lane; maintenance groups, approval rules and ring membership. 3. **Delivery and activation.** Agents that download, verify and install; restarts, live patches and firmware flashes. 4. **Verification and monitoring.** Confirming "the patch is installed and has taken effect", then that it stays so: not uninstalled, restored from an old backup or reset[^sp800-40]. 5. **Exceptions and mitigations.** Dated deviations, compensating controls, and a removal date for every temporary mitigation. ### Roles, ownership and RACI This RACI is this Atlas's proposal (R responsible, A accountable, C consulted, I informed). | Activity | PatchOps lead | Asset owner | VulnerabilityOps | Change authority ([ITOps](../itops/)) | Risk owner | |---|---|---|---|---|---| | Define maintenance groups and plans | R | C | C | C | A | | Assign assets to groups and rings | R | A | I | I | — | | Release triage and ring calendar | A/R | I | C | I | — | | Ring promotion go/no-go | A/R | C | I | I | — | | Emergency patch delivery | A/R | C | C (sets due date) | C (approves change) | I | | Approve an exception | C | R | C | — | A | | Compliance report and evidence | A/R | I | C | I | I | Often missing: a **ring captain** per cycle, who can halt promotion without a meeting, and **canary volunteers** (administrators, engineers and early adopters, as NIST suggests[^sp800-40]) who report problems quickly. ### Lifecycle of a patch NIST's software vulnerability management life cycle (know, plan, execute: prepare, implement, verify, continuously monitor) covers every risk response, not only patching[^sp800-40]. Adapted here, it shows two operational exits: no usable patch, and a patch that causes problems. ::: figure Life cycle of one fix, from advisory to monitored state ```mermaid flowchart TB K["Know: inventory matches an advisory"]:::core --> PL["Plan: pick scenario and maintenance plan"]:::decision PL -->|"patch available"| PR["Prepare: acquire, validate, schedule"]:::core PL -->|"no patch or unpatchable"| MI["Mitigate: disable, isolate, track"]:::risk PR --> DE["Deploy: rings and windows"]:::core DE -->|"gate fails"| HA["Pause, roll back or mitigate"]:::risk DE --> VE["Verify: installed and in effect"]:::core VE --> MO["Monitor: still installed, behaving"]:::core HA --> PL MI --> MO MO --> K classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029 classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029 ``` Text equivalent: Inventory matches an advisory (know); planning picks a scenario and plan. An available patch is acquired, validated, scheduled and deployed in rings and windows; a failed gate leads to pause, rollback or mitigation and back to planning. A successful deployment is verified as in effect, then monitored. With no patch, or an unpatchable asset, the asset is mitigated and monitored. Monitoring feeds the inventory again. ::: ### Operating model and governance | Cadence | Activity | Output | |---|---|---| | Each vendor release (e.g. second Tuesday) | Triage: applicability, known issues, ring calendar | Cycle plan, canary start | | Daily during a cycle | Gate review by the ring captain | Promote, hold or halt | | Weekly | Stragglers: unreporting assets, failed installs, pending restarts | Tickets to owners | | Monthly | Compliance report by maintenance group; exception expiries | Evidence pack, escalations | | Quarterly | Plan review: groups, rings, deadlines; exceptions "reviewed regularly", as NIST asks[^sp800-40] | Revised plans | | Annually | End-of-support calendar and supplier review | Upgrade and replacement plans | **Change control.** Pre-approve routine cycles as standard changes, with the ring plan as the procedure, plus an emergency route ([ITOps](../itops/) owns the change practice). **Escalation.** After the grace period, overdue assets go to their owner, then to the risk owner for an exception or enforcement ([Rings](#rings-trading-speed-for-evidence) says what enforcement means per ring). ## Architecture and workflow ### Reference architecture Components, whichever products provide them ([tooling](#tooling-landscape)): ::: figure Reference architecture of a patch practice ```mermaid flowchart TB VS["Vendor feeds and repositories"]:::neutral --> MR["Internal mirror or cache"]:::neutral INV["Inventory and applicability"]:::core --> POL["Policy: groups, approvals, rings"]:::core POL --> ORC["Orchestrator: windows, rate, error limit"]:::core HS["Health signals: alerts, probes, tickets"]:::risk -->|"gate input"| ORC MR -->|"content"| AG["Agents on servers and devices"]:::neutral ORC -->|"commands"| AG AG --> RB["Restart coordinator"]:::neutral AG -->|"results"| CS["Compliance store"]:::core CS -.->|"updates"| INV classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029 classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029 ``` Text equivalent: Vendor content flows through an internal mirror or cache to agents on servers and devices. Inventory feeds policy (groups, approvals, rings), which drives an orchestrator that enforces windows, rollout rate and an error limit and commands the agents; health signals are its gate input. Agents hand restarts to a restart coordinator and report to a compliance store, which updates the inventory. ::: | Component | Design notes | |---|---| | Mirror or cache | Freezes the tested set and serves offline sites; validate authenticity and integrity "preferably by automated means"[^sp800-40] | | Inventory and applicability | Authenticated data; unreporting assets are a finding, not a gap to hide | | Orchestrator | Windows, concurrency, pausing; an error limit acts as a circuit breaker | | Restart coordinator | One failure domain at a time; drain or fail over first | | Health signals | Alerts, incident tickets and synthetic tests per role; the gate is only as good as these | | Compliance store | Installed, active, pending restart, failed, exception, unreporting; keep history for audits and incident responders[^sp800-40] | **Trust boundary.** The patch system runs code as administrator on every asset it manages. Treat its consoles, service accounts and mirrors as tier-0: least privilege, strong authentication, logs, and separate rights to approve content and to target assets. ### End-to-end workflow: one monthly cycle **Happy path.** Triage confirms applicability and known issues; canaries install within hours; gates pass; every ring has its own deadline and later rings meet theirs with restarts in windows; the compliance report closes the cycle. ::: figure Ring promotion with gates and the halt path ```mermaid flowchart TB R["Vendor release"]:::neutral --> T["Triage: applicability, known issues"]:::core T --> R0["Ring 0: canaries"]:::core R0 --> G1{{"Gate 1 healthy?"}}:::decision G1 -->|"yes"| R1["Ring 1: pilot"]:::core R1 --> G2{{"Gate 2 healthy?"}}:::decision G2 -->|"yes"| R2["Ring 2: broad"]:::core R2 --> G3{{"Gate 3 healthy?"}}:::decision G3 -->|"yes"| R3["Ring 3: sensitive, node by node"]:::core R3 --> EN["Final ring deadline: forced install"]:::core EN --> VF["Verify effective coverage"]:::core G1 -->|"no"| H["Halt: pause release, run runbook"]:::risk G2 -->|"no"| H G3 -->|"no"| H H -->|"fix or OOB release"| T classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029 classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029 classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029 ``` Text equivalent: A vendor release is triaged and installed on ring 0 canaries. Each healthy gate promotes it: gate 1 to ring 1 (pilot), gate 2 to ring 2 (broad), gate 3 to ring 3 (sensitive systems), node by node. After the final ring's deadline installation is forced and effective coverage verified. A failed gate halts the release and starts the runbook; a fix or out-of-band release returns the cycle to triage. ::: **Exception paths.** - *A gate fails:* pause the release and follow the [worked runbook](#worked-runbook-a-monthly-update-breaks-a-server-role-in-ring-1). - *An asset misses its window:* it rolls to the next one; after the grace period, enforcement. - *An emergency arrives mid-cycle:* the emergency plan pre-empts routine rings, which resume afterwards. - *No usable patch:* emergency mitigation with a removal date ([decision framework](#decision-framework)). ## Practices, cadences and tools ### Core practices and playbooks NIST's four scenarios are the backbone of the playbooks[^sp800-40]: 1. **Routine patching.** "Most patching falls under this scenario", and it is often "postponed and neglected", which makes later emergencies harder because earlier patches must go in first. 2. **Emergency patching.** The same approach "with a highly accelerated schedule"; a canary period of "a few minutes to a few hours" may still help. 3. **Emergency mitigation.** Disable functionality or isolate the asset while no good patch exists; schedule both the patch and the mitigation's removal. 4. **Unpatchable assets.** Approved long-term mitigations per maintenance group, re-evaluated periodically, including whether the asset is still worth its cost. | Practice | Trigger | Owner | Verification | |---|---|---|---| | Routine ring rollout | Vendor release | PatchOps lead | Effective coverage by deadline | | Emergency patching | Priority item from VulnerabilityOps | PatchOps lead, change authority | Asset-level evidence returned | | Emergency mitigation | No patch, or a faulty one | Service owner | Mitigation verified; removal dated | | Unpatchable-asset plan | End of support, certified systems, operational technology (OT) | Risk owner | Periodic reassessment | | Restart orchestration | Pending restarts | Platform or device team | Pending-restart age | | [Third-party applications](#third-party-applications) | Vendor releases, catalogue updates | Endpoint and app owners | Installed version per asset | | Firmware and UEFI (PC boot firmware) | Original equipment manufacturer (OEM) releases | Hardware owners | Firmware version inventory | | End-of-support planning | Vendor lifecycle calendars | Asset owners | Share on supported versions | Two asset classes are taught elsewhere: smartphones, one of NIST's example maintenance groups[^sp800-40], in [EndpointOps](../endpointops/); firewall, VPN and other network appliances, upgraded one peer at a time where paired, in [NetOps](../netops/). Macs stay in scope: on supervised Apple devices the device-management service can defer an update by 1 to 90 days per group and enforce installation by a deadline[^apple-updates]; give Macs a maintenance group and rings of their own. ### Rings: trading speed for evidence In NIST's phased deployment "a small subset of the assets to be patched receive the patch first": "In effect, this is how the patching gets tested"[^sp800-40]. In a 2019 study of 102 US administrators, staggered deployment was the most common testing strategy (43 of 102) and only 14 described positive testing experiences; one would "install on non-important machines and let them bake for 1+ months"[^keepers], longer than many risk deadlines allow. NIST's own example sits in between: a first round of technically knowledgeable users that "lasts a few days", then early adopters who "try app updates for a few weeks"[^sp800-40]. Vendor defaults are much faster. As of October 2026, the ring policies that Windows Autopatch groups create defer quality updates by 0, 1 and 2 days for their Test, Ring 1 and Last rings, with deadlines of 0, 0 and 1 days and grace periods of 0, 1 and 2 days[^autopatch-groups]; Autopatch recommends automatic approval for security updates[^autopatch-wqu]. | Ring | Who belongs | Share | Soak | Gate signals | |---|---|---|---|---| | 0: canaries | IT staff devices; one server per role and build | 1–2 % | Hours to 2 days | Install, boot, synthetic tests per role | | 1: pilot | Volunteers from every department; non-critical servers of every role | 5–10 % | 2–4 days | Plus tickets tagged to the update | | 2: broad | Everyone else | Most | To deadline | Failure and reporting rates | | 3: sensitive | Strict-window systems; clusters node by node | Small | Scheduled window | Service health per node | Shares and soaks are illustrative; three rules matter more: - **Representativeness beats size.** A canary ring without each server role (domain controller, Remote Desktop Services (RDS) host, database) will pass and let a role-specific regression through. NIST wants early adopters "representative of the entire user community"[^sp800-40]. - **Gates need functional probes.** A clean install can still break sign-in or printing. - **Freeze what was tested.** Rings that pull "everything approved today" may install packages the canaries never saw. Freeze the set per cycle with a repository snapshot, an explicit list or a cut-off date (AWS `ApproveUntilDate`)[^aws-rule]. On Windows Server a cut-off date has a trap: by default Windows Server 2019 and 2022 remove superseded updates, so if a newer or out-of-band release arrives after the cut-off, it is not approved, the frozen update may no longer install, and Systems Manager still reports the node compliant[^aws-predef]. Approve the new release, start it at ring 0, and check Windows rings by build number or an authenticated scan. **What "forced" means for ring 3.** For strict-window assets a deadline never triggers an out-of-window restart: the next scheduled window becomes mandatory, an asset that misses it needs a dated exception from the risk owner, and enforcement is indirect, as in NIST's example of "preventing network access for unpatched assets until they are patched"[^sp800-40], not an unscheduled push. ### Restarts, live patching and Kubernetes nodes Many fixes take effect only after a restart of the software or the platform[^sp800-40]. The single-host checks are in [SysOps](../sysops/#packages-reboots-and-the-next-boot); across a fleet, three facts catch teams out: - **Library updates need service restarts.** A running service keeps the old library in memory; from Ubuntu 24.04, `needrestart` restarts affected services automatically unless they are excluded[^ubuntu-auto]. - **Tools reboot more than you expect.** With AWS `RebootIfNeeded`, Patch Manager "doesn't evaluate whether a reboot is required by the patch" and reboots anyway[^aws-rpb]. - **Deadlines override users.** Once a Windows quality-update deadline and any grace period have passed, the device ignores active hours and restarts after a 15-minute warning[^autopatch-ux]. **Live patching.** Canonical Livepatch "patches high and critical Linux kernel vulnerabilities without requiring a system reboot"[^livepatch]. Windows Server Hotpatch avoids restarts in two months out of three, but quarterly and unplanned baselines, non-security updates, .NET, drivers and firmware still need them; as of October 2026, Azure Arc-enabled Hotpatch for Windows Server 2025 is "now available at no extra cost"[^hotpatch]. **Kubernetes nodes** are hosts too. Kured, a CNCF Sandbox project, watches for `/var/run/reboot-required`, takes a cluster lock so that by default one node reboots at a time, can wait while Prometheus alerts fire, and cordons and drains each node[^kured]. Replacing node pools from updated images belongs to [KubernetesOps](../kubernetesops/). ### Third-party applications CIS Safeguard 7.4 asks for automated application patching "on a monthly, or more frequent, basis", checked per asset against an authoritative source of version details[^cis7]. Browsers, office suites and runtimes need the same plans as the OS, with three differences: - **Verify by installed version**, not deployment status: many applications update outside the OS package manager. - **Vendor channels are rings you configure.** Google rolls Chrome Stable releases out "over a few weeks"; administrators can choose the Extended Stable channel or pin a version, which Google says "should be done only temporarily", and components such as Widevine keep updating even when browser updates are off[^chrome-updates]. For Microsoft 365 Apps on Monthly Enterprise Channel, Microsoft suggests that a select group of users takes each version first, then a gradual deployment[^m365-channels]. Map these groups onto your rings. - **Catalogues package updates, sometimes without rings.** As of October 2026, Intune Enterprise App Management (an add-on licence) offers prepared Microsoft and non-Microsoft Win32 apps; its auto-update "doesn't support rollout rings or deployment plans for staged update deployment" and has no rollback, and self-updating catalogue apps follow the vendor's process[^intune-eam]. Where rings matter, use its guided supersedence instead. ### Rollback and its limits "Roll back if it breaks" is a plan only if the mechanism exists for that asset type: | Asset type | Back-out options | Limits | |---|---|---| | Windows quality update | Pause; uninstall; Known Issue Rollback (KIR) Group Policy | KIR "only affects non-security updates and fixes"[^kir]; pausing does not roll back installed devices[^autopatch-wqu] | | Linux packages | `dnf history undo `; an older version if a repository still has it | dnf can refuse the undo[^dnf-cmd] and `apt-get` has none[^apt-get] ([SysOps](../sysops/#packages-reboots-and-the-next-boot)); plan VM or filesystem snapshots | | Firmware | Vendor downgrade, if permitted | Confirm before relying on it; stage firmware slowest | | Image-based hosts, containers | Redeploy the previous image | Reintroduces the flaw; time-box it | The common path is **roll forward**: install the vendor's fixed or out-of-band release. NIST also lists "restoring the software or asset from backups"[^sp800-40], so restores ([BackupOps](../backupops/)) belong in the plan for stateful systems. ### Methods, standards and frameworks | Source | Type | What it gives you | |---|---|---| | NIST SP 800-40 Rev. 4 (2022)[^sp800-40] | US federal guidance, voluntary elsewhere | Scenarios, maintenance groups, canaries, metrics, procurement questions | | CIS Safeguards 7.3, 7.4 (v8.1)[^cis7] | Framework recommendation | Automated OS and application patching, at least monthly | | IEC TR 62443-2-3:2015[^iec-62443-2-3] | IEC technical report | Patch management for industrial control systems | | CISA BOD 26-04 (2026)[^bod2604] | Binding on US federal civilian agencies | Remediation clocks; revoked BOD 22-01, so older two-week references are historical | | Vendor lifecycle policies[^ubuntu-cycle][^ws-channels] | Vendor policy | How long each version gets security fixes | NIST maps patch planning to SP 800-53 controls including CM-8 (inventory), CM-3 (change control) and SI-2 (flaw remediation)[^sp800-40]. Prioritisation methods (CVSS severity scores, EPSS exploit probabilities, CISA's KEV catalogue of exploited flaws, Stakeholder-Specific Vulnerability Categorization (SSVC) decision trees) live in [VulnerabilityOps](../vulnerabilityops/). ### Tooling landscape Product facts are as of October 2026. | Category | Verified examples | Watch-outs | |---|---|---| | Client update services | Windows Autopatch and Intune quality update and expedite policies[^autopatch-wqu][^intune-expedite]; WSUS (deprecated, supported)[^wsus] | Precedence when legacy rings and cloud policies overlap | | Server and hybrid patching | AWS Systems Manager Patch Manager[^aws-policies]; Azure Update Manager[^azure-um]; Google Cloud VM Manager Patch[^gcp-patch] | Reboot defaults; per-Region compliance views | | Third-party applications | Intune Enterprise App Management[^intune-eam]; vendor update policies[^chrome-updates][^m365-channels] | Catalogue auto-update without rings; self-updaters on the vendor's schedule | | Host-native automation | unattended-upgrades[^ubuntu-auto]; dnf-automatic[^dnf-auto] | No rings or fleet reporting on their own[^ubuntu-auto] | | Live patching | Canonical Livepatch[^livepatch]; Windows Server Hotpatch[^hotpatch] | Coverage limits; licensing | | Restart coordination | Kured[^kured] | Drains need pod disruption budgets | | Firmware | fwupd with the Linux Vendor Firmware Service[^lvfs]; OEM tools | Vendor coverage varies | Selection criteria: platforms; ring and deadline controls; freezing the tested set; reboot behaviour; compliance that separates installed from active; API and history export; privilege model; cost; how it reads vendor metadata ([use case 3](#use-cases-and-scenarios)). ### Automation and AI NIST is blunt: "There is no way that an organization can keep up with patching without automation"[^sp800-40]. Automate approval rules, calendars, gate evaluation, restarts and evidence; keep human decisions for halting a moving release, emergency changes and exceptions. Language models can summarise release notes and draft change records; use them as readers, not deciders, and never let one promote a ring. ## PatchOps in practice ### Reference implementation **Assumptions.** About 4,000 Windows devices on a cloud update service; a few hundred Windows, Ubuntu and RHEL-family servers on premises and in AWS; one Kubernetes cluster; a few unpatchable industrial PCs; one small platform team. A starting design, not a product recommendation. **1. Maintenance plans as data.** An illustrative schema in version control records who changed a deadline and feeds calendars and reports from one source. _Example status: syntax-checked (YAML), not executed; the schema is illustrative._ ```yaml # patch-policy.yaml: maintenance groups and their plans (Atlas illustrative schema) version: 3 release_anchor: second-tuesday # routine cycle starts on the vendor release day groups: - name: workstations rings: [ring0-it-staff, ring1-volunteers, ring2-everyone] routine: {deadline_days: 7, grace_days: 2, restart: deadline-forced} emergency: {canary_hours: 4, deadline_days: 2} unreporting_after_days: 7 - name: linux-servers rings: [ring0-one-per-role, ring1-nonprod, ring2-prod-a, ring3-prod-b] routine: {deadline_days: 14, window: "Tue-Thu 01:00-05:00 UTC", restart: in-window} emergency: {canary_hours: 2, deadline_days: 3, change: emergency-route} freeze_tested_set: true # later rings install what ring 0 tested - name: k8s-nodes method: reboot-coordinator # Kured, one node at a time routine: {deadline_days: 14} - name: legacy-ot method: unpatchable mitigations: [network-isolation, allow-list-firewall, no-internet] review: quarterly replacement_due: 2027-06-30 exceptions: require: [owner, reason, compensating_control, expiry] max_days: 90 ``` **2. One Windows Server ring on AWS.** One baseline serves all rings, which differ only in their windows[^aws-rule][^aws-mw][^aws-mwtask]. It reuses the filter of AWS's predefined Windows baseline (which waits 7 days[^aws-predef]) but approves after 2; Microsoft publishes around 10:00 Pacific on the second Tuesday[^ms-cycle], so a 01:00 UTC window three days later finds that month's update approved. Side effects: `DefaultBaseline: true` sets the account and Region default for Windows; `AvailableSecurityUpdatesComplianceStatus: NON_COMPLIANT` reports security updates that are available but not yet approved as non-compliant[^aws-rule]. Later rings would approve anything released since; freeze them only with the Windows Server supersedence trap in [Rings](#rings-trading-speed-for-evidence) in mind. _Example status: syntax-checked (YAML), not deployed. Requires the SSM Agent on targets, instances tagged `PatchRing=ring1`, and AWS Identity and Access Management (IAM) permission to create Systems Manager resources._ ```yaml AWSTemplateFormatVersion: "2010-09-09" Description: Ring 1 Windows Server patching (illustrative; review before use) Resources: WindowsSecurityBaseline: Type: AWS::SSM::PatchBaseline Properties: Name: windows-server-security-2d OperatingSystem: WINDOWS DefaultBaseline: true AvailableSecurityUpdatesComplianceStatus: NON_COMPLIANT # Windows Server only ApprovalRules: PatchRules: - PatchFilterGroup: PatchFilters: - Key: CLASSIFICATION Values: [CriticalUpdates, SecurityUpdates] - Key: MSRC_SEVERITY Values: [Critical, Important] ApproveAfterDays: 2 ComplianceLevel: CRITICAL Ring1Window: Type: AWS::SSM::MaintenanceWindow Properties: Name: ring1-windows-servers Schedule: "cron(0 1 ? * TUE#2 *)" # 01:00 UTC on the second Tuesday... ScheduleOffset: 3 # ...run three days later (Friday) ScheduleTimezone: Etc/UTC Duration: 4 Cutoff: 1 AllowUnassociatedTargets: false Ring1Targets: Type: AWS::SSM::MaintenanceWindowTarget Properties: WindowId: {Ref: Ring1Window} ResourceType: INSTANCE Targets: - Key: tag:PatchRing Values: [ring1] Ring1Install: Type: AWS::SSM::MaintenanceWindowTask Properties: WindowId: {Ref: Ring1Window} TaskType: RUN_COMMAND TaskArn: AWS-RunPatchBaseline Priority: 1 MaxConcurrency: "10%" # one tenth of the ring at a time MaxErrors: "2" # circuit breaker Targets: - Key: WindowTargetIds Values: [{Ref: Ring1Targets}] TaskInvocationParameters: MaintenanceWindowRunCommandParameters: Parameters: Operation: [Install] RebootOption: [RebootIfNeeded] ``` To validate, run the document with `Operation: Scan` on a ring-0 instance: compliance shows missing patches and nothing installs. `ApproveAfterDays` is not supported for Debian or Ubuntu Server[^aws-rule], so Ubuntu rings need another way to delay and freeze, such as a dated repository snapshot. **3. Ubuntu host defaults.** unattended-upgrades already applies security updates daily, after a random timer delay, and reboots only if `Automatic-Reboot` is turned on (default off)[^ubuntu-auto]. For ring servers, keep it for the security pocket, block packages that need manual steps, and leave reboots to the ring's window, which takes one failure domain at a time and checks `/var/run/reboot-required` itself: Patch Manager might not reboot hosts whose packages another tool installed[^aws-rpb]. For needrestart, add one key at a time: assigning a whole new `override_rc` hash replaces the shipped list that stops it restarting dbus, networking, docker, libvirt and more[^needrestart-conf]. _Example status: not executed on a host; apt directives checked against Ubuntu's documentation[^ubuntu-auto]; the needrestart line evaluated in Perl 5.30 after upstream `needrestart.conf` (2026-10-17): 50 shipped exclusions plus `haproxy` remained, where the whole-hash form left only `haproxy`._ ```text // /etc/apt/apt.conf.d/52patchops-local (overrides; keep the distribution file intact) Unattended-Upgrade::Package-Blacklist { "postgresql-"; // database restarts handled by the DB team in-window }; Unattended-Upgrade::Automatic-Reboot "false"; // default; the ring window reboots hosts Unattended-Upgrade::Mail "patch-reports@example.com"; Unattended-Upgrade::MailReport "on-change"; ``` ```text # /etc/needrestart/conf.d/50-patchops.conf (Perl syntax) # Add a key; never assign a new hash, which would drop the shipped exclusions. $nrconf{override_rc}{qr(^haproxy\.service$)} = 0; # restarted by the LB runbook ``` **4. RHEL-family host defaults.** `dnf-automatic` (DNF 4) can install only security advisories[^dnf-auto]; for ring servers, leave reboots to the window, as on Ubuntu. _Example status: not executed; options checked against the DNF 4 documentation. DNF 5 distributions ship a different implementation._ ```ini # /etc/dnf/automatic.conf (excerpt) [commands] upgrade_type = security apply_updates = yes reboot = never random_sleep = 1800 [emitters] emit_via = stdio,motd ``` Enable it with `systemctl enable --now dnf-automatic.timer`. With `reboot = when-needed` instead, timing gets loose: the timer adds up to an hour of random delay on top of `random_sleep`, and the reboot waits five minutes[^dnf-auto]. **5. Kubernetes node reboots.** Arguments for the `kured` container, from the configuration reference[^kured]. _Example status: syntax-checked (YAML fragment), not executed._ ```yaml args: - --reboot-days=tue,wed,thu - --start-time=1am - --end-time=5am - --time-zone=UTC - --period=10m - --concurrency=1 - --drain-timeout=30m - --prometheus-url=http://prometheus.monitoring.svc.cluster.local - --alert-filter-regexp=^RebootRequired$ # alert names to IGNORE; any other active alert defers reboots - --blocking-pod-selector=app=nightly-batch ``` ### Hands-on labs #### Lab 1 (beginner): read one Linux host's patch state **Objective:** see the difference between "available", "installed" and "in effect" on one host. **Prerequisites:** an Ubuntu/Debian or RHEL-family host and a normal user account; 15 minutes. **Cost and risk:** none; the script only reads (`apt-get -s` simulates[^apt-get]; `dnf needs-restarting -r` reports[^needs-restarting]). 1. Save the script as `patch-state.sh` and run `bash patch-state.sh`. 2. Compare `kernel_running` with `kernel_newest_installed`. 3. On Ubuntu, read `/var/run/reboot-required.pkgs` if it exists: those packages asked for the restart. 4. Note which number a compliance report would show, and which reflects real protection. _Example status: verified by execution on Ubuntu 20.04.6 (apt branch, unprivileged user, 2026-10-16; revised version re-run 2026-10-17). The dnf branch was not run on a RHEL-family host; its logic was tested with stub `dnf` commands (plugin present or missing). Real output is not reproduced (production host)._ ```bash #!/usr/bin/env bash # patch-state.sh: read-only summary of one Linux host's patch state. # Changes nothing; run as an ordinary user. Debian/Ubuntu (apt) or RHEL family (dnf). set -uo pipefail echo "host=$(hostname -s) kernel_running=$(uname -r)" if command -v apt-get >/dev/null 2>&1; then sim=$(apt-get -s -o Debug::NoLocking=1 upgrade 2>/dev/null | grep '^Inst ' || true) total=$(printf '%s\n' "$sim" | grep -c '^Inst ' || true) sec=$(printf '%s\n' "$sim" | grep -c -- '-security' || true) echo "pending_upgrades=${total} from_security_pocket=${sec}" lists_mtime=$(stat -c %Y /var/lib/apt/lists 2>/dev/null || echo 0) echo "package_lists_age_hours=$(( ($(date +%s) - lists_mtime) / 3600 ))" if [ -f /var/run/reboot-required ]; then echo "reboot_required=yes since=$(date -r /var/run/reboot-required +%F)" sed 's/^/ requested_by: /' /var/run/reboot-required.pkgs 2>/dev/null | sort -u else echo "reboot_required=no" fi elif command -v dnf >/dev/null 2>&1; then lines=$(dnf -q updateinfo list --security --available 2>/dev/null | wc -l) echo "security_update_lines=${lines}" # one line per package, not per advisory if dnf needs-restarting --help >/dev/null 2>&1; then dnf needs-restarting -r >/dev/null 2>&1; rc=$? case $rc in 0) echo "reboot_required=no" ;; 1) echo "reboot_required=yes" ;; *) echo "reboot_required=unknown (exit ${rc})" ;; esac else echo "reboot_required=unknown (needs-restarting missing: dnf-plugins-core)" fi else echo "unsupported package manager" >&2; exit 2 fi newest=$(ls -1 /boot/vmlinuz-* 2>/dev/null | sed 's|/boot/vmlinuz-||' \ | sort -V | tail -n 1) echo "kernel_newest_installed=${newest:-unknown}" if [ -n "${newest:-}" ] && [ "$newest" != "$(uname -r)" ]; then echo "CHECK: running kernel is not the newest installed (reboot pending?)" fi exit 0 ``` **Expected output (shape, values illustrative):** ```text host=web-03 kernel_running=5.15.0-1084-aws pending_upgrades=12 from_security_pocket=4 package_lists_age_hours=6 reboot_required=yes since=2026-10-14 requested_by: linux-image-5.15.0-1085-aws kernel_newest_installed=5.15.0-1085-aws CHECK: running kernel is not the newest installed (reboot pending?) ``` **Validation:** `CHECK` appears exactly when the newest kernel in `/boot` differs from `uname -r`. **Troubleshooting:** a large `package_lists_age_hours` means stale counts; `from_security_pocket` approximates by matching a `-security` origin. **Teardown:** delete the script. #### Lab 2 (intermediate): plan a ring calendar and evaluate gates **Objective:** derive a ring calendar from the release day and apply promotion gates to health data. **Prerequisites:** Python 3.8 or later; no network. **Cost and risk:** none; synthetic data. 1. Save `rings.py` and run `python3 rings.py` (defaults to October 2026), then `python3 rings.py 2026 11`. 2. Change ring 1's observations so that `reported` is 380 and `failed` is 5, and run again: the gate should promote. 3. Add `"incidents": 1` to ring 0 and confirm the decision becomes `HALT`. _Example status: verified by execution (Python 3.8.10; published version re-run 2026-10-17). Self-tests confirmed the second-Tuesday function against Microsoft's documented release dates of 2026-01-13, 2026-07-14 and 2026-09-08[^rh-w11]._ ```python """rings.py: plan one monthly patch cycle and decide whether each ring may be promoted. Standard library only (Python 3.8+). All data below is synthetic. """ import datetime as dt import sys def second_tuesday(year, month): first = dt.date(year, month, 1) offset = (1 - first.weekday()) % 7 # Monday=0, Tuesday=1 return first + dt.timedelta(days=offset + 7) # start: days after release; deadline: days after start; devices: ring size RINGS = [ {"name": "ring0-canary", "start": 0, "deadline": 1, "devices": 40}, {"name": "ring1-pilot", "start": 2, "deadline": 2, "devices": 400}, {"name": "ring2-broad", "start": 5, "deadline": 3, "devices": 3200}, {"name": "ring3-critical", "start": 9, "deadline": 4, "devices": 120}, ] GATE = {"max_install_failure_rate": 0.02, "max_new_incidents": 0, "min_reporting_rate": 0.90} def plan(release): for ring in RINGS: start = release + dt.timedelta(days=ring["start"]) due = start + dt.timedelta(days=ring["deadline"]) yield ring["name"], start, due def gate(obs): """Return (decision, reasons) for one ring's observed health.""" reasons = [] reported = obs["reported"] / obs["targeted"] failure = obs["failed"] / max(obs["reported"], 1) if reported < GATE["min_reporting_rate"]: reasons.append(f"only {reported:.0%} of devices reported") if failure > GATE["max_install_failure_rate"]: reasons.append(f"install failure rate {failure:.1%}") if obs["incidents"] > GATE["max_new_incidents"]: reasons.append(f"{obs['incidents']} incident(s) linked to the update") if obs["incidents"]: return "HALT", reasons # pause the release, open the runbook return ("HOLD", reasons) if reasons else ("PROMOTE", ["all gate checks passed"]) def main(): year, month = (int(x) for x in (sys.argv[1:3] or ["2026", "10"])) release = second_tuesday(year, month) print(f"release (second Tuesday): {release}") for name, start, due in plan(release): print(f" {name:15s} start {start} install deadline {due}") observed = { # synthetic health signals, one row per finished ring "ring0-canary": {"targeted": 40, "reported": 39, "failed": 0, "incidents": 0}, "ring1-pilot": {"targeted": 400, "reported": 352, "failed": 9, "incidents": 0}, } for name, obs in observed.items(): decision, why = gate(obs) print(f"gate after {name}: {decision} ({'; '.join(why)})") if __name__ == "__main__": main() ``` **Expected output (actual run for October 2026):** ```text release (second Tuesday): 2026-10-13 ring0-canary start 2026-10-13 install deadline 2026-10-14 ring1-pilot start 2026-10-15 install deadline 2026-10-17 ring2-broad start 2026-10-18 install deadline 2026-10-21 ring3-critical start 2026-10-22 install deadline 2026-10-26 gate after ring0-canary: PROMOTE (all gate checks passed) gate after ring1-pilot: HOLD (only 88% of devices reported; install failure rate 2.6%) ``` **Validation:** 352 ÷ 400 = 88 % reporting (below 90 %) and 9 ÷ 352 = 2.6 % failures (above 2 %), so ring 1 holds. **Troubleshooting:** `ValueError: month must be in 1..12` means a bad month argument. HOLD means "chase missing reports, re-evaluate"; HALT means "pause the release"; decide in advance who may override. **Teardown:** delete the file. #### Lab 3 (advanced): compliance numbers that do not lie **Objective:** compare an honest on-time coverage figure with a typical dashboard figure from the same data. **Prerequisites:** Python 3.8 or later. **Cost and risk:** none; synthetic inventory of 16 assets. 1. Save `compliance.py` and run it. 2. For each group, explain the gap between `honest` and `dashboard`. 3. Change `crit-03`'s exception date to `2026-11-20` and re-run; note which column changes. _Example status: verified by execution (Python 3.8.10; published version re-run 2026-10-17); every figure was also checked by hand._ ```python """compliance.py: honest patch-compliance metrics for one release. Standard library only (Python 3.8+). The inventory below is synthetic. """ import csv import datetime as dt import io import statistics AS_OF = dt.date(2026, 10, 30) RELEASE = dt.date(2026, 10, 13) DEADLINE_DAYS = {"workstations": 7, "servers": 14, "critical": 21, "legacy-ot": None} STALE_AFTER_DAYS = 7 INVENTORY = """id,group,last_seen,installed,reboot_pending_since,exception_until ws-01,workstations,2026-10-30,2026-10-14,, ws-02,workstations,2026-10-29,2026-10-15,, ws-03,workstations,2026-10-30,2026-10-19,, ws-04,workstations,2026-10-30,2026-10-22,, ws-05,workstations,2026-10-12,,, ws-06,workstations,2026-10-30,2026-10-16,2026-10-16, srv-01,servers,2026-10-30,2026-10-18,, srv-02,servers,2026-10-30,2026-10-18,, srv-03,servers,2026-10-30,2026-10-25,, srv-04,servers,2026-10-30,,,2026-11-15 srv-05,servers,2026-10-01,,, srv-06,servers,2026-10-30,2026-10-20,2026-10-20, crit-01,critical,2026-10-30,2026-10-24,, crit-02,critical,2026-10-30,2026-10-24,, crit-03,critical,2026-10-30,,,2026-10-20 ot-01,legacy-ot,2026-10-30,,,2027-03-31 """ def d(s): return dt.date.fromisoformat(s) if s else None def main(): rows = list(csv.DictReader(io.StringIO(INVENTORY))) print(f"as of {AS_OF}, release {RELEASE}, {len(rows)} assets in inventory") print(f"{'group':13s} {'in':>3s} {'stale':>5s} {'on_time':>7s} {'honest':>7s} " f"{'dashboard':>9s} {'inst_lag':>8s} {'reboot>3d':>9s} {'exc_expired':>11s}") for group, days in DEADLINE_DAYS.items(): g = [r for r in rows if r["group"] == group] stale = [r for r in g if (AS_OF - d(r["last_seen"])).days > STALE_AFTER_DAYS] fresh = [r for r in g if r not in stale] effective = [r for r in g if r["installed"] and not r["reboot_pending_since"]] on_time = [r for r in effective if days is not None and (d(r["installed"]) - RELEASE).days <= days] exc = [r for r in g if r["exception_until"]] exc_expired = [r for r in exc if d(r["exception_until"]) < AS_OF] # "dashboard" view: counts exceptions as compliant and ignores stale agents dash_num = len([r for r in fresh if r["installed"] or r["exception_until"]]) dashboard = dash_num / len(fresh) if fresh else 0.0 honest = len(on_time) / len(g) if g else 0.0 # median install lag, including assets still waiting for a restart (not the KPI) lags = [(d(r["installed"]) - RELEASE).days for r in g if r["installed"]] median = statistics.median(lags) if lags else float("nan") reboot_old = [r for r in g if r["reboot_pending_since"] and (AS_OF - d(r["reboot_pending_since"])).days > 3] print(f"{group:13s} {len(g):3d} {len(stale):5d} {len(on_time):7d} " f"{honest:7.0%} {dashboard:9.0%} {median:8.1f} " f"{len(reboot_old):9d} {len(exc_expired):11d}") if __name__ == "__main__": main() ``` **Expected output (actual run):** ```text as of 2026-10-30, release 2026-10-13, 16 assets in inventory group in stale on_time honest dashboard inst_lag reboot>3d exc_expired workstations 6 1 3 50% 100% 3.0 1 0 servers 6 1 3 50% 100% 6.0 1 0 critical 3 0 2 67% 100% 11.0 0 1 legacy-ot 1 0 0 0% 100% nan 0 0 ``` **Validation:** workstations ws-01–03 were active within 7 days, ws-04 was late (day 9), ws-05 is stale and ws-06 awaits a restart: 3 of 6 = 50 %. The dashboard drops ws-05 and counts ws-04 and ws-06: 5 of 5 = 100 %. It also counts `crit-03`, whose exception expired on 2026-10-20. `inst_lag` is the median install lag and includes assets awaiting a restart, so it is not the KPI "time to effective patch". **Troubleshooting:** dates must be `YYYY-MM-DD`. **Interpretation:** the legacy-OT row is not a patch figure; that group's metric is mitigations verified each quarter. **Teardown:** delete the file. ### Use cases and scenarios **1. A monthly update with two regressions (documented, September 2026).** After the 8 September 2026 security update (KB5124008), Microsoft recorded that "some organizations might experience issues with Remote Desktop Services"; an out-of-band update on 14 September resolved it. Another issue that month could break the domain trust of some Credential Guard-protected Windows 11 devices; Microsoft offered a Known Issue Rollback and fixed it on 22 September[^rh-w11]. *Method:* a ring 0 with one RDS host and one Credential Guard device, gated by synthetic remote-desktop and domain sign-ins. *Trade-off:* halting delays the month's security fixes, so scope the halt to the affected role. *Success:* the affected role contained within a day. *Pitfall:* a laptop-only canary ring, which passes both. **2. Patched everywhere except where it mattered (documented, Equifax 2017).** Equifax officials told the US Government Accountability Office (GAO) that the Apache Struts flaw "was not properly identified as being present on the online dispute portal when patches for the vulnerability were being installed throughout the company"[^gao-equifax] ([SecOps](../secops/#use-cases-and-scenarios) tells the full story). *Method:* route advisories from inventory to owners, not mailing lists. *Trade-off:* it needs authenticated, component-level inventory whose agents or scan credentials are privileged. *Success:* for any component, a query lists every asset running it, with its owner, in minutes. *Pitfall:* trusting one scan; NIST warns that passive scans "will generally underreport vulnerabilities"[^sp800-40]. **3. A tool approved the wrong thing (documented, November 2024).** Some Windows Server 2019 and 2022 machines upgraded themselves to Windows Server 2025 where third-party products managed updates; Microsoft said feature-update metadata "must be interpreted as Optional and not Recommended"[^rh-ws2022]. *Method:* approval rules by classification (never auto-approve upgrades) and a canary ring that also tests the patch tool after it changes. *Success:* an unexpected classification stops at ring 0. *Trade-off:* wanted upgrades need their own project. *Pitfall:* assuming the tool excludes upgrades; Microsoft asked organisations to check that third-party tools are "configured not to deploy feature updates"[^rh-ws2022]. ## Measure, govern, pay ### Metrics and KPIs NIST warns that "Overly simplistic metrics" such as organisation-wide counts of vulnerabilities and the percentage patched "are not actionable"; it proposes the share patched by the deadline and mean and median time to patch, per combination of asset importance and vulnerability importance[^sp800-40]. | KPI | Definition and formula | Pitfall | Cadence | |---|---|---|---| | On-time effective coverage | Assets with the cycle's fixes installed **and active** by the deadline ÷ **all** in-scope assets in the group | Dividing by reporting assets only | Per cycle | | Time to effective patch | Median and 90th percentile days from release (or the risk deadline's start) to active, per group | Means hide long tails | Monthly | | Unreporting rate | Assets with no check-in for N days ÷ inventory | Dropping them from denominators | Weekly | | Pending-restart age | Assets with a restart pending > N days; oldest age | Counting them as compliant | Weekly | | Ring failure rate | Failed installs ÷ reported, per ring | Ignoring silent devices | Per ring | | Patch-caused incidents | Incidents attributed to updates per cycle | Shown alone, it rewards slowness | Monthly | | Emergency lead time | Hours from priority item to 95 % of targeted assets active | Stopping the clock at "deployed" | Per event | | Exception health | Open exceptions; share past expiry | Counting exceptions as fixes | Monthly | | Automation coverage | Per asset, OSs and applications under automated patching ÷ all installed (CIS M7/M1)[^cis7]; list assets with none | Coverage without enforcement | Quarterly | **Worked calculation.** Lab 3's 15 patchable assets give 8 ÷ 15 = 53 % on-time effective coverage, and 13 ÷ 13 = 100 % on the dashboard view. CIS's Update Effectiveness measure, (M2 + M4) ÷ M1, also counts out-of-date systems *with a documented exception* (M4)[^cis7]: a legitimate design choice, but report exceptions beside the figure so nobody mistakes one for a fix. **Targets:** start from today's median per group, pair every speed target with a disruption measure, review quarterly. ### Quality, security and compliance - **The patch system is a privileged attack path** (see the [trust boundary](#reference-architecture)); protect internal mirrors like the vendor's servers. - **Update channels outside your rings.** In July 2024 CrowdStrike customers chose sensor versions (N, N-1, N-2) through Sensor Update Policies, but the crash came from Rapid Response Content, "configuration data; it is not code or a kernel driver", delivered separately; CrowdStrike then added customer control over its deployment[^cs-hub]. Ask each vendor which channels you can stage ([ReleaseOps](../releaseops/) covers the release-safety lessons). - **Regulated and certified systems.** Some manufacturers require delayed updates for safety-related assets, and self-applied updates may void the warranty[^sp800-40]; industrial systems have IEC TR 62443-2-3[^iec-62443-2-3]. Give them their own maintenance group. - **Evidence and privacy.** Keep per-asset history (what, when, who, restart, verification); retention follows your audit regime. Telemetry identifies devices and users: collect only what reports need. ### Cost, capacity and sustainability **Cost drivers:** staff time per cycle, canary and test capacity, downtime windows, tooling, paid extended support for old versions (Ubuntu's Expanded Security Maintenance extends security updates to 10 years[^ubuntu-cycle]) and unpatchable software. NIST suggests asking suppliers about maintenance before buying, and fewer components, managed services and smaller platforms to reduce the patch load[^sp800-40]. These levers probably cut energy use too (our inference; no measurements found). **Capacity arithmetic (illustrative).** 600 servers, a 4-hour window with a 1-hour cutoff, about 45 minutes per server including restart and checks, and at most 10 % (60) at a time: 3 usable hours give 4 waves, so 240 servers per window and 3 windows per cycle. If the deadline allows only two windows, raise concurrency, shorten restarts, or move workloads to replaceable images. ### Assurance of the operating model - **Two sources must agree.** Reconcile the patch tool's view with an authenticated scan monthly; disagreement is a finding. - **Sample by hand.** Each month, check a few assets per group directly (version, restart time, service state). - **Leading indicators.** Rising unreporting rate or restart age, a shrinking canary ring, gates that never fail. ## When things go wrong ### Common failures and troubleshooting | Symptom | Likely cause | Diagnosis | Remediation | Prevention | |---|---|---|---|---| | Coverage stalls at 85–90 % | Offline or broken agents; devices outside management | Inventory vs. check-ins | Repair agents; restrict network access for long-unreporting devices | Unreporting rate as a KPI | | Scanner still finds the flaw after "patching" | Restart pending; library still loaded | `/var/run/reboot-required`, `needs-restarting` | Restart in window | Count only effective fixes | | Later rings install untested packages | Approvals not frozen | Compare package lists between rings | Freeze the tested set | Snapshot or list per cycle | | Windows Server "compliant" on an old build | Cut-off date before a superseding release[^aws-predef] | Build number vs. the release | Approve the new release; restart at ring 0 | Independent build check | | Servers reboot in business hours | Tool default (`RebootIfNeeded`) or deadline with zero grace | Policy review | Move reboots into windows | Test policy changes on ring 0 | | Unexpected major upgrade | Approval rule matched an upgrade classification | Tool approval history | Restore or rebuild affected hosts | Never auto-approve upgrades[^rh-ws2022] | | Emergency fix takes days | No pre-agreed emergency plan or change route | Timeline review | Use the emergency route | Rehearse twice a year | | Exceptions never close | No expiry or owner | Register review | Expire and escalate | Expiry required on creation | ### Worked runbook: a monthly update breaks a server role in ring 1 _Illustrative runbook built on documented mechanisms._ **Trigger.** Eighteen hours after ring 1 installed the monthly update, the synthetic remote-desktop sign-in fails on 3 of 8 RDS hosts. Ring 0 passed because it had no RDS host. 1. **Halt (15 minutes).** The ring captain pauses promotion; an Autopatch pause can take up to eight hours to reach devices[^autopatch-wqu]. For server tools, disable the next ring's window or task. Tell the service desk. 2. **Scope (1 hour).** Which roles, builds and configurations fail, and what do they share? Record the update identifiers. 3. **Check the vendor (1 hour).** Read the known-issues page; in September 2026 it held exactly this problem ([use case 1](#use-cases-and-scenarios)). 4. **Contain per role.** Apply a vendor-documented mitigation; a Known Issue Rollback if the faulty change is non-security[^kir]; or fail over to healthy hosts. Uninstalling reopens the month's vulnerabilities, so it needs a time-boxed exception from the risk owner. 5. **Keep the rest moving.** If the tool can scope the pause, continue for unaffected groups; their fixes are still due. 6. **Roll forward.** Run the fixed or out-of-band release through the rings from ring 0 (it is cumulative[^ms-cycle]). In September 2026 the out-of-band fix was itself one of the updates behind that month's Credential Guard sign-in issue[^rh-w11]. 7. **Close.** Remove mitigations on a date, close exceptions, add an RDS host and its sign-in probe to ring 0, and record time to halt. ### Anti-patterns and myths - **"Test every patch in a lab for two weeks first."** Labs rarely match production and delay extends exposure ([FAQ](#faq)). - **"Exceptions are permanent."** NIST's answer is a maintenance group of their own, with planned mitigations[^sp800-40]. - **"Change freeze means patch freeze."** Keep an emergency lane open, with named approvers. ## Deciding and adopting ### When to use and when not to use **Run a formal practice** with more than a few dozen assets, several operating systems or owners, audit duties, strict uptime, or a history of patch-caused outages. **Keep it light, or move the work,** when most of the estate is SaaS (patching transferred to the provider[^sp800-40]); when hosts and containers are rebuilt from images, so patching lives in the pipeline and this practice only sets deadlines and verifies; or when the estate is small enough that unattended updates and a weekly check meet the deadlines ([deep comparisons](#deep-comparisons)). **Do not** create a "PatchOps team" just to hold the label if endpoint and platform teams can own their rings with a shared calendar and one report, and do not force a monthly routine on assets that cannot take it (OT, certified medical devices). ### Decision framework For each asset class, decide the delivery mechanism before the next advisory arrives. ::: figure Choosing how a fix reaches an asset ```mermaid flowchart TB Q1{"Fix exists?"}:::decision -->|"no"| A2["Emergency mitigation, re-check"]:::risk Q1 -->|"yes"| Q2{"Built from an image?"}:::decision Q2 -->|"yes"| A1["Rebuild and redeploy"]:::core Q2 -->|"no"| Q3{"Restart in window?"}:::decision Q3 -->|"yes"| A3["Patch in place through rings"]:::core Q3 -->|"no"| Q4{"Live patch?"}:::decision Q4 -->|"yes"| A4["Live patch now, restart later"]:::core Q4 -->|"no"| Q5{"Workload can move?"}:::decision Q5 -->|"yes"| A5["Drain, patch, return"]:::core Q5 -->|"no"| A6["Unpatchable plan: isolate, replace"]:::risk classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029 classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029 ``` Text equivalent: With no fix yet, apply an emergency mitigation and re-check. With a fix, rebuild and redeploy assets built from images; otherwise patch in place through rings if the asset can restart in its window; if not, live-patch when possible and restart later; else drain, patch and return the workload if it can move; failing that, follow an unpatchable plan: isolate and plan replacement. ::: Assumptions: a "fix" is a patch or a fixed image; "built from an image" means an automated, tested rebuild; "restart in window" includes verifying health afterwards. ### Maturity model This is this Atlas's model, not an industry standard. | Level | Observable behaviour | Exit criterion | |---|---|---| | 0. Ad hoc | Patching when someone has time | Inventory of patchable assets with owners | | 1. Scheduled | Monthly cycle, one window, manual approvals | Maintenance groups for ≥ 90 % of assets | | 2. Ringed | Canary and pilot rings; deadlines and grace periods; emergency route | On-time effective coverage measured per group | | 3. Measured | Honest denominators; restart tracking; expiring exceptions; quarterly reviews | Emergency lead time rehearsed within target | | 4. Engineered | Automatic gates on functional probes; frozen tested sets; live patching; rebuilds where possible | Sustained targets with falling patch-caused incidents | ### Adoption roadmap: 30/60/90 days | Period | Actions | Milestone | |---|---|---| | Days 1–30 | Reconcile inventory with check-ins; list vendor cadences; define 3–5 maintenance groups; name a ring captain; agree the emergency route with [ITOps](../itops/) | One group's cycle with an honest coverage figure | | Days 31–60 | Ring 0 with one asset per role and build; functional probes; deadlines and grace periods; frozen tested set; exception register | Gate decisions recorded for every ring | | Days 61–90 | All groups; monthly report; emergency drill; hand-off record agreed with [VulnerabilityOps](../vulnerabilityops/) | Drill completed within the target lead time | ## Toolkit ### Checklists **Quick start (one maintenance group, one cycle)** - [ ] Inventory reconciled with agent check-ins; unreporting assets listed with owners - [ ] Vendor cadences and emergency lanes written down - [ ] Ring 0 holds at least one asset per role, build and configuration, with functional probes - [ ] Deadlines, grace periods and windows set; reboot behaviour tested on ring 0 - [ ] Report counts only active fixes against the whole group **Monthly cycle preflight (release day)** - [ ] Release notes and known issues read; applicability confirmed - [ ] Tested set frozen; ring calendar sent to owners and the service desk - [ ] Ring captain, backup and halt authority named - [ ] Rollback or mitigation path known for each major role **Quarterly governance audit** - [ ] Every asset in a maintenance group; no "miscellaneous" group - [ ] No exception past expiry; each has an owner and compensating control - [ ] Ring membership still representative; patch-system access reviewed - [ ] End-of-support dates for the next 18 months listed with plans ### Templates **Maintenance plan record** (one per maintenance group; this Atlas's template) ```text MAINTENANCE PLAN group: linux-servers-prod owner: platform-team Assets: ~420 Ubuntu 24.04 / RHEL 9 VMs (tag PatchGroup=linux-servers-prod) Business limits: restarts Tue-Thu 01:00-05:00 UTC; no restarts in the quarter-end freeze except via the emergency route Routine: rings ring0 (8) -> ring1 (40) -> ring2 (180) -> ring3 (192) start release+2d; deadline 14d; tested set frozen per cycle Emergency: canary 2h; all rings within 3d; emergency change route Mitigation: web application firewall (WAF) rules; service disable runbooks; isolation VLAN Unpatchable members: none (move any to group legacy-isolated) Verification: effective = installed + restarted; weekly authenticated scan Review: quarterly; last reviewed 2026-10-01 by ``` **Exception record** ```text EXCEPTION id: PX-2026-031 group: critical asset: crit-03 Fix not applied: October 2026 cumulative update Reason: vendor-certified application not yet validated on this build Compensating control: inbound restricted to app subnet; endpoint detection and response (EDR) agent in block mode Risk owner: approved: 2026-10-14 expires: 2026-11-20 Exit plan: vendor certification due 2026-11-10; patch in next window Review: weekly status to PatchOps lead; auto-escalate on expiry ``` **Emergency patch change** (one page) ```text EMERGENCY PATCH change: CHG- due (from VulnerabilityOps): Item: scope: exposure: Plan: canary for ; then rings by Restart impact: Fallback: ; Verification: ; evidence to Approvers: , halt authority: ``` **Supplier patch-support questions** (adapted from the questionnaire in NIST SP 800-40 Rev. 4, section 3.7[^sp800-40]) ```text 1. Will you release security updates for this product, and for how many years? 2. How many updates per year do you expect, how fast do you fix a vulnerability, and how often do you release emergency updates? 3. Do you release on a schedule (weekly, monthly, quarterly), as needed, or both? 4. Do you run a vulnerability disclosure and incident response programme? 5. When no patch exists yet, what mitigation do you recommend? 6. What does an update disrupt: a service restart, a reboot, downtime? 7. How do you test updates, how often do customers hit problems, and can an update be rolled back? 8. Which update channels (agent, content, firmware) can customers stage in rings? ``` NIST's eight questions are condensed into questions 1–7 (its questions 1 and 3 become question 1 here); question 8 is this Atlas's addition, prompted by the July 2024 content-update incident[^cs-hub]. **Monthly report** (one page per cycle; values illustrative) ```text CYCLE 2026-10 release 2026-10-13 report date 2026-10-30 Group in-scope on-time active days to active restart>3d unreporting (median) workstations 4,100 91 % 3.5 55 120 linux-servers 420 88 % 6.5 4 9 critical 140 81 % 12.0 0 0 Exceptions: 14 open, 1 past expiry (escalated) Patch-caused incidents: 1 Gate halts: 1 (ring1, RDS sign-in probe; resumed with OOB release) ``` ### Deep comparisons Alternatives to in-place patching; neighbours are compared [earlier](#how-patchops-relates-to-other-ops). | Approach | Optimises | Wins for | Costs | |---|---|---|---| | In-place patching in rings (this page) | Keeping long-lived assets current | Stateful servers, user devices | Maintenance windows, restarts, compliance tracking | | Rebuild and redeploy ([ContainerOps](../containerops/), [InfraOps](../infraops/)) | Identical, reproducible assets | Stateless services, autoscaled fleets | Pipeline discipline; still needs deadlines | | SaaS or managed service | Transferring patching to a provider[^sp800-40] | Commodity capabilities | Less control over timing | | Isolation as a plan | Containing what cannot be fixed | End-of-support or certified systems | Ongoing mitigation; replacement still due | ### FAQ **Should we test every patch before deploying it?** Test on real assets first. NIST calls phased deployment to canaries the way patching "gets tested"[^sp800-40]. A lab helps for high-risk systems but should not delay the canary ring. **How long should each ring soak?** Long enough for probes and users to surface problems, short enough for the risk deadline. Autopatch group defaults promote within days[^autopatch-groups]; NIST's example rounds take days, then weeks[^sp800-40]. Measure which ring catches problems, then tune. **"We redeploy containers, so we do not patch." Right?** Rebuilding *is* patching: it needs a cadence, a deadline and proof that the running workload uses the new image. The build practice belongs to [ContainerOps](../containerops/) and [ServerlessOps](../serverlessops/). **What about servers that can never reboot?** Use live patching, move the workload to a partner node, or give it an unpatchable-asset plan with isolation and a replacement date. A server that cannot restart cannot recover from a crash either. **Is WSUS dead?** Deprecated, not removed: as of October 2026, Microsoft's 2024 statement stands (no new features, continued updates, no current plan to remove it from in-market Windows Server, including 2025)[^wsus]. ## Going further ### Advanced topics and emerging direction - **Boot-chain and firmware updates are routine work now.** Microsoft's 2011 Secure Boot certificates carry 2026 expiry dates: 24 June for the Key Enrollment Key (KEK) certificate authority (CA), which signs updates to the boot databases; 27 June for the third-party UEFI CA; 19 October for the Windows Production PCA, the CA that signs the Windows boot loader. Devices without the 2023 certificates "will continue to start and operate normally" but can no longer receive new early-boot protections; most get the new certificates automatically, and many OEMs provide firmware updates when needed[^secureboot]. Run it as a firmware programme with its own rings. - **End-of-support waves.** Windows 10 reached end of support on 14 October 2025[^win10-eos], CentOS Linux 7 on 30 June 2024[^centos-eol]; Ubuntu 22.04's standard security maintenance ends in May 2027[^ubuntu-cycle]. Each wave needs an upgrade, paid extended support or an unpatchable-asset plan. - **Component-level inventory.** NIST notes that suppliers' software bills of materials (SBOMs) can augment inventories[^sp800-40]. ### Learning path | Stage | What to do | |---|---| | 1-day primer | NIST SP 800-40 Rev. 4 sections 2–3[^sp800-40]; Lab 1 on a test VM | | 1 week | Design groups and rings for a lab estate; Lab 2 with your own gates | | 30 days | Two real cycles for one group with an honest report (Lab 3 logic) | | Advanced | Frozen tested sets, functional probes, an emergency drill, firmware rings | | Portfolio | A versioned policy file, a calendar generator and a monthly report from real data | | Next pages | [VulnerabilityOps](../vulnerabilityops/) (security side); [ITOps](../itops/), [EndpointOps](../endpointops/) (operations) | ## Wrap-up ### Executive recap and cheat sheet - **Decide in advance:** maintenance groups, a plan per scenario, ring membership, deadlines, windows, and who may halt. - **Practices:** representative canaries with functional probes; frozen tested sets; restarts in windows; forced installs; expiring exceptions. - **Metrics:** on-time *effective* coverage over the whole inventory; time to effective patch; unreporting rate; restart age; patch-caused incidents; emergency lead time. - **Watch-outs:** tool reboot defaults, upgrade classifications, update channels outside your rings, dashboards that drop silent assets. - **Next actions:** reconcile inventory; build ring 0 properly; publish one honest report. ### A judgement call: automatic approval with short rings, or manual approval after lab testing? **Situation (illustrative).** A hospital group runs 9,000 Windows devices and 600 servers. Since a bad update broke a clinical application two years ago, every monthly update waits for a 14-day lab test and change-board approval; median time to effective patch is 31 days. The CISO wants automatic approval with rings measured in days. **Option A: keep manual approval and lab testing.** Trusted by the clinical lead. But the lab has never had the clinical application's newest version, the 14 days add exposure to every fix, and the board approves almost everything anyway. **Option B: automatic approval with short rings.** Matches vendor guidance[^autopatch-wqu] and fits NIST's phased model, although NIST's example rounds run days, then weeks[^sp800-40]: short rings are a choice, not NIST's prescription. But ring 0 has no clinical workstation, so the next regression would reach wards by day 2. **Decision.** Assumptions: the application runs on about 1,200 devices; its vendor certifies monthly updates within 10 days; one ward can tolerate half a day of disruption. Choose **B for everything except a "clinical" maintenance group**, which gets its own rings (IT clinical test devices, then one volunteer ward, then the rest) and a deadline tied to the vendor's certification. **Why the attractive choice can be wrong.** A *feels* safe because patching harm is visible and self-inflicted, while delay harm stays invisible until an incident; yet, as NIST notes, patching disruptions are "largely controllable" and incident disruptions "largely uncontrollable"[^sp800-40]. **Uncertainty:** if the vendor certifies slowly, the problem becomes supplier management. ### Learning objectives and mastery check After this page you can **explain** where the label fits, **design** maintenance groups, rings and gates, **operate** a cycle with an emergency lane, **measure** effective coverage honestly, and **decide** when to patch in place, rebuild, live-patch or isolate. ::: answer 1. A report says 97 % of servers are compliant. What three questions do you ask first? Is the denominator all inventory or only reporting assets? Does "compliant" mean installed or active after restart? Are exceptions counted, and are any expired? Lab 3 shows one data set giving 53 % and 100 %. ::: ::: answer 2. Ring 0 is 40 IT laptops. Why might it miss a regression that breaks Remote Desktop hosts? It holds no RDS host. Canaries must represent roles, builds and configurations; the September 2026 RDS issue was role-specific[^rh-w11]. ::: ::: answer 3. Ring 2 installed three packages ring 0 never received. What went wrong? The approved set was not frozen. Freeze it per cycle (repository snapshot, explicit list or cut-off date), remembering the Windows Server supersedence trap: re-approve out-of-band releases and verify by build number. ::: ::: answer 4. An exploited flaw has no patch yet on an internet-facing appliance. Which NIST scenario applies? Emergency mitigation: disable the vulnerable feature or isolate the asset, and schedule both the patch and the mitigation's removal. ::: ::: answer 5. Why is "we can always roll back" a weak plan for Windows security updates? Uninstalling reopens the vulnerabilities, Known Issue Rollback covers only non-security changes[^kir], and pausing does not undo installs[^autopatch-wqu]. Plan containment per role, a time-boxed exception and a roll-forward release. ::: ### Conclusion PatchOps is a new name for an old, unglamorous job: getting other people's fixes onto your systems without breaking them. The label adds nothing the standards lack; the practice decides whether known flaws stay open for days or months. Done well, it is decided in advance, and each month is execution. If you remember one thing: **a patch counts when it is active on an asset you can see, not when a console says it was sent.** ## Sources and further reading [^sp800-40]: [NIST SP 800-40 Rev. 4: Guide to Enterprise Patch Management Planning](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-40r4.pdf) — NIST, April 2022. Checked 2026-10-17. [^amrita]: [PatchOps: An Automated Vulnerability Detection and Patch Management System for Windows Environments](https://www.amrita.edu/publication/patchops-an-automated-vulnerability-detection-and-patch-management-system-for-windows-environments/) — IEEE i-PACT 2025 (abstract page), Amrita Vishwa Vidyapeetham. Checked 2026-10-17. [^cis7]: [CIS Control 7, Safeguards 7.3 and 7.4](https://cas.docs.cisecurity.org/en/latest/source/Controls7/) — CIS Controls Assessment Specification v8.1. Checked 2026-10-17. [^iec-62443-2-3]: [IEC TR 62443-2-3:2015](https://webstore.iec.ch/en/publication/22811) — IEC, 2015. Checked 2026-10-16. [^bod2604]: [BOD 26-04: Prioritizing Security Updates Based on Risk](https://www.cisa.gov/news-events/directives/bod-26-04-prioritizing-security-updates-based-risk) — CISA, 2026-06-10. Checked 2026-10-16. [^gao-equifax]: [GAO-18-559: Actions Taken by Equifax and Federal Agencies in Response to the 2017 Breach](https://www.gao.gov/assets/gao-18-559.pdf) — US GAO, August 2018. Checked 2026-10-17. [^nao-wannacry]: [Investigation: WannaCry cyber attack and the NHS](https://www.nao.org.uk/reports/investigation-wannacry-cyber-attack-and-the-nhs/) — National Audit Office (UK), 2017-10-27. Checked 2026-10-16. [^keepers]: [Keepers of the Machines](https://faculty.cc.gatech.edu/~frankli/papers/li_soups2019.pdf) — Li, Rogers, Mathur, Malkin, Chetty, SOUPS 2019. Checked 2026-10-16. [^ms-cycle]: [Update release cycle for Windows clients](https://learn.microsoft.com/en-us/windows/deployment/update/release-cycle) — Microsoft Learn, updated 2026-09-29. Checked 2026-10-17. [^autopatch-wqu]: [Windows quality updates and .NET Framework updates](https://learn.microsoft.com/en-us/windows/deployment/windows-autopatch/manage/windows-autopatch-windows-quality-update-overview) — Microsoft Learn, updated 2026-09-01. Checked 2026-10-16. [^autopatch-groups]: [Autopatch group policies](https://learn.microsoft.com/en-us/windows/deployment/windows-autopatch/manage/windows-autopatch-groups-policies) — Microsoft Learn, updated 2026-07-02. Checked 2026-10-17. [^autopatch-ux]: [Windows quality update end user experience](https://learn.microsoft.com/en-us/windows/deployment/windows-autopatch/manage/windows-autopatch-windows-quality-update-end-user-exp) — Microsoft Learn, updated 2025-03-31. Checked 2026-10-17. [^intune-expedite]: [Manage Windows quality updates (Intune)](https://learn.microsoft.com/en-us/intune/device-updates/windows/manage-quality-updates) — Microsoft Learn, updated 2026-01-06. Checked 2026-10-17. [^wsus]: [Windows Server Update Services (WSUS) deprecation](https://techcommunity.microsoft.com/t5/windows-it-pro-blog/windows-server-update-services-wsus-deprecation/ba-p/4250436) — Microsoft, 2024-09-20. Checked 2026-10-16. [^kir]: [Known Issue Rollback](https://learn.microsoft.com/en-us/troubleshoot/windows-server/installing-updates-features-roles/known-issue-rollback) — Microsoft Learn, updated 2026-02-12. Checked 2026-10-16. [^hotpatch]: [Hotpatch for Windows Server](https://learn.microsoft.com/en-us/windows-server/get-started/hotpatch) — Microsoft Learn, updated 2026-09-08. Checked 2026-10-16. [^rh-ws2022]: [Windows Server 2022 resolved issues](https://learn.microsoft.com/en-us/windows/release-health/resolved-issues-windows-server-2022) — Microsoft release health (entry opened 2024-11-09). Checked 2026-10-17. [^rh-w11]: [Windows 11, version 24H2 resolved issues](https://learn.microsoft.com/en-us/windows/release-health/resolved-issues-windows-11-24h2) — Microsoft release health (September 2026 entries). Checked 2026-10-17. [^secureboot]: [Windows Secure Boot certificate expiration and CA updates](https://support.microsoft.com/en-us/servicing/os/secure-boot/2025/06/windows-secure-boot-certificate-expiration-and-ca-updates) — Microsoft Support, revised 2026-05-18. Checked 2026-10-17. [^win10-eos]: [Windows 10 Enterprise and Education lifecycle](https://learn.microsoft.com/en-us/lifecycle/products/windows-10-enterprise-and-education) — Microsoft. Checked 2026-10-16. [^ws-channels]: [Comparison of Windows Server servicing channels](https://learn.microsoft.com/en-us/windows-server/get-started/servicing-channels-comparison) — Microsoft Learn. Checked 2026-10-16. [^azure-um]: [About Azure Update Manager](https://learn.microsoft.com/en-us/azure/update-manager/overview) — Microsoft Learn. Checked 2026-10-16. [^aws-policies]: [Patch policy configurations in Quick Setup](https://docs.aws.amazon.com/systems-manager/latest/userguide/patch-manager-policies.html) — AWS. Checked 2026-10-16. [^aws-rpb]: [AWS-RunPatchBaseline](https://docs.aws.amazon.com/systems-manager/latest/userguide/patch-manager-aws-runpatchbaseline.html) — AWS. Checked 2026-10-17. [^aws-rule]: [AWS::SSM::PatchBaseline](https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-resource-ssm-patchbaseline.html); [PatchBaseline Rule](https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-properties-ssm-patchbaseline-rule.html) — AWS. Checked 2026-10-17. [^aws-predef]: [Predefined and custom patch baselines](https://docs.aws.amazon.com/systems-manager/latest/userguide/patch-manager-predefined-and-custom-patch-baselines.html) — AWS. Checked 2026-10-17. [^aws-mw]: [AWS::SSM::MaintenanceWindow](https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-resource-ssm-maintenancewindow.html); [cron and rate expressions](https://docs.aws.amazon.com/systems-manager/latest/userguide/reference-cron-and-rate-expressions.html) — AWS. Checked 2026-10-16. [^aws-mwtask]: [AWS::SSM::MaintenanceWindowTask](https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-resource-ssm-maintenancewindowtask.html); [MaintenanceWindowTarget](https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-resource-ssm-maintenancewindowtarget.html) — AWS. Checked 2026-10-16. [^gcp-patch]: [About Patch (VM Manager)](https://docs.cloud.google.com/compute/vm-manager/docs/patch) — Google Cloud. Checked 2026-10-17. [^ubuntu-auto]: [Automatic updates](https://ubuntu.com/server/docs/how-to/software/automatic-updates/) — Ubuntu Server documentation, Canonical. Checked 2026-10-17. [^apt-get]: [apt-get(8)](https://manpages.ubuntu.com/manpages/noble/en/man8/apt-get.8.html) — Ubuntu Manpages. Checked 2026-10-16. [^ubuntu-cycle]: [The Ubuntu lifecycle and release cadence](https://ubuntu.com/about/release-cycle) — Canonical. Checked 2026-10-16. [^dnf-auto]: [DNF Automatic](https://dnf.readthedocs.io/en/latest/automatic.html) — DNF 4 documentation. Checked 2026-10-17. [^needs-restarting]: [DNF needs-restarting plugin](https://dnf-plugins-core.readthedocs.io/en/latest/needs_restarting.html) — dnf-plugins-core. Checked 2026-10-17. [^dnf-cmd]: [DNF Command Reference](https://dnf.readthedocs.io/en/latest/command_ref.html) — DNF 4 documentation. Checked 2026-10-16. [^centos-eol]: [CentOS Linux end of life](https://www.redhat.com/en/topics/linux/centos-linux-eol) — Red Hat. Checked 2026-10-16. [^livepatch]: [Livepatch documentation](https://ubuntu.com/security/livepatch/docs) — Canonical, updated 2026-06-18. Checked 2026-10-16. [^kured]: [kured](https://github.com/kubereboot/kured); [configuration](https://kured.dev/docs/configuration/) — Kured project (CNCF Sandbox). Checked 2026-10-17. [^lvfs]: [Linux Vendor Firmware Service](https://fwupd.org/); [introduction](https://lvfs.readthedocs.io/en/latest/intro.html) — LVFS. Checked 2026-10-16. [^oracle-cpu]: [Critical Patch Updates, Critical Security Patch Updates, Security Alerts and Bulletins](https://www.oracle.com/security-alerts/) — Oracle. Checked 2026-10-17. [^cs-hub]: [Falcon Content Update Remediation and Guidance Hub](https://www.crowdstrike.com/falcon-content-update-remediation-and-guidance-hub/) — CrowdStrike, 2024. Checked 2026-10-17. [^needrestart-conf]: [needrestart.conf, upstream default configuration](https://github.com/liske/needrestart/blob/master/ex/needrestart.conf) — needrestart project, last changed 2026-09-06. Checked 2026-10-17. [^chrome-updates]: [Manage Chrome updates (Windows)](https://support.google.com/chrome/a/answer/6350036) — Google, Chrome Enterprise and Education Help. Checked 2026-10-17. [^m365-channels]: [Overview of update channels for Microsoft 365 Apps](https://learn.microsoft.com/en-us/microsoft-365-apps/updates/overview-update-channels) — Microsoft Learn, updated 2026-05-27. Checked 2026-10-17. [^intune-eam]: [Microsoft Intune Enterprise Application Management](https://learn.microsoft.com/en-us/intune/app-management/deployment/enterprise-app-management) — Microsoft Learn, updated 2026-06-03. Checked 2026-10-17. [^apple-updates]: [Install and enforce software updates for Apple devices](https://support.apple.com/guide/deployment/install-and-enforce-software-updates-depd30715cbb/1/web/1.0) — Apple Platform Deployment, Apple, published 2025-09-24. Checked 2026-10-17. **Further reading:** [NIST SP 1800-31, Improving Enterprise Patching for General IT Systems](https://csrc.nist.gov/pubs/sp/1800/31/final) (NCCoE, April 2022), the technology companion to SP 800-40. Evidence ledger: `research/patchops-ledger.md`. **Changelog:** 2026-10-16, revision 0 — first complete draft. 2026-10-17, revision 1 — answered review r1 (3 major, 20 minor); four sources added. 2026-10-17, revision 2 — answered review r2 (1 major, 8 minor) and the first score's S2–S4; one source added. 2026-10-17, post-score polish: date-proof Secure Boot wording, shorter figure 4, use-case trade-offs, dated name search, trims to the length band.