---
title: "DetectionOps"
slug: detectionops
category: security
term_status: overlapping-synonym
evidence_status: verified
summary: "DetectionOps is a rare label for detection engineering run like software delivery: detections are written as code from threat-informed requirements, tested, peer-reviewed, deployed through a pipeline, measured in production, tuned and retired."
audience: [Detection engineers, SOC engineers and analysts, Purple teams, Security architects, Security leaders, Beginners]
last_verified: 2026-10-17
key_takeaways:
- "\"DetectionOps\" is a rare label: every documented use found comes from the European Commission's OpenTIDE project. The practice behind it is detection engineering run as detection-as-code; search for those names."
- "Treat every detection as a small product: a requirement, a data dependency, logic in version control, tests that prove it fires and stays quiet, an owner, and a date to review or retire it."
- "A green pipeline proves logic against fixtures, not that a rule fires in production. Check rule runs and live data, and emulate periodically in an authorised lab."
- "An ATT&CK tag is not coverage. Count validated coverage per technique implementation and judge detection quality (robustness and precision), as MITRE's Summiting the Pyramid advises."
- "Every filter buys precision with a blind spot. Record accepted gaps as tests so they stay visible."
- "Frameworks and toolchains move (ATT&CK v19 renamed tactics; pySigma 2.0 outpaced sigma-cli). Pin toolchains and maintain mappings as part of each detection."
related: [threatops, soc-ops, secdataops, aiops-for-security, responseops, devsecops, secops]
---
## What DetectionOps is
### In 30 seconds
**DetectionOps is detection engineering run like software delivery.** A detection (a query in a SIEM, the platform that collects and
searches security logs; a rule in an endpoint detection and response (EDR) tool; a Sigma rule) starts from a written requirement:
which threat, on which assets, seen in which data. It lives in version control with tests proving it fires on malicious examples and
stays quiet on benign ones, passes review, is deployed through staging by a pipeline, is judged by analysts' verdicts, and is tuned or
retired on purpose. The page serves detection and SOC engineers, purple teams, and leaders who must say what their detections cover.
::: note Term status: overlapping synonym. Read this first
"DetectionOps" is not a standard term. Every documented use this Atlas found on 2026-10-16 comes from one open-source project, the
European Commission's OpenTIDE (now OpenTide), which calls itself "the DetectionOps engine for detection-as-code"[^opentide-gh]. Its
maintainers describe the idea as mapping "all Detection Engineering tasks on a DevOps-like loop"[^hacklu25].
The practice is well documented under better-known names: **detection engineering** (the discipline) and **detection-as-code** (the
way of working). SANS surveys it[^sans-2025]; Elastic and MITRE's Center for Threat-Informed Defense (CTID) publish models and
methods[^debmm][^stp-workflow]. This page teaches that practice under the manifest's label.
:::
### At a glance
| Question | Short answer |
|---|---|
| Goal | A versioned, tested library of detections whose coverage and alert quality are known |
| Main users | Detection engineers, SOC engineers, purple teams; analysts as customers |
| Inputs | Threat intelligence, hunt findings, incident lessons, SOC feedback, asset priorities |
| Outputs | Production detections with tests and documentation, coverage evidence, tuning and retirement records |
| Prerequisites | Working log collection ([SecDataOps](../secdataops/)), a SOC that works alerts ([SOC Ops](../soc-ops/)), basic Git |
| Out of scope | Triage, response, intelligence, telemetry pipelines, ML detection |
| Term status | Overlapping synonym for detection engineering and detection-as-code |
### The problem it solves
Detections created by hand in a console cause three problems. Nobody can say what the library covers, because rules carry no
requirement or evidence. Changes are invisible: a 2 a.m. exclusion creates an unknown blind spot. And rules decay silently: a parser
update renames a field, and the query runs but never matches again.
Palantir's incident response team described the result in its Alerting and Detection Strategy (ADS) framework: a "lack of rigor" meant
"sub-optimal alerts going to production without documentation or peer-review", which "led to fatigue, alerting apathy, or additional
engineering time and resources"[^ads]. SANS's summary of its 2025 survey (264 respondents) says "Only
45% of organizations have a way to reliably measure the effectiveness of their detection engineering efforts"[^sans-2025]. The
sponsor's summary of the 2026 survey (307 practitioners) reports 62% using version control for rules, 58% peer review and 42% CI/CD
integration[^sans-2026].
### What it is not
- **Not the SOC, hunting or incident response.** Triage belongs to [SOC Ops](../soc-ops/), intelligence and hunting to
[ThreatOps](../threatops/), confirmed incidents to [ResponseOps](../responseops/).
- **Not log engineering.** Onboarding, parsing and retention belong to [SecDataOps](../secdataops/); this page only names the
data dependency.
- **Not machine-learning detection.** Anomaly detection, user and entity behaviour analytics (UEBA) and LLM triage are evaluated in
[AIOps for Security](../aiops-for-security/).
## Where it comes from and where it sits
### Origins and evolution of the label
- **2013: behaviour over artefacts.** David Bianco's Pyramid of Pain (revised in 2014, when hash values became its bottom level) ranks
indicators up to TTPs (tactics, techniques and procedures), where defenders work "directly on adversary behaviors, not against their
tools"[^pyramid].
- **2017: documentation and portable rules.** Palantir published the ADS framework[^ads]; Sigma, "for log files what Snort is for
network traffic and YARA is for files"[^sigma-about], was presented at Hack.lu[^sigmahq-readme].
- **2021–2024: maturity models.** Kyle Bailey's Detection Engineering Maturity Matrix (2021) treated detection-as-code as a
technology dimension[^de-matrix]; Elastic's DEBMM (2024) defined five tiers[^debmm].
- **2022–2026: the label.** The European Commission's security operations centre put OpenTIDE into operation in September 2022 and
released it as open source in 2024[^ec-ospo]. Its maintainers' 2024 Hack.lu workshop opened with an "Introduction to DetectionOps with
OpenTIDE"[^hacklu24]; a 2026 workshop builds an "Operational DetectionOps Platform" (due 23 October 2026)[^hacklu26].
- **2023–2026: measurement matures.** CTID's Summiting the Pyramid (2023; version 5.0 in 2026) defined robustness levels
and coverage measures beyond technique tags[^stp-changelog].
### Glossary and mental model
| Term | Meaning on this page | Example |
|---|---|---|
| Detection (rule, analytic) | Logic that turns telemetry into an alert or signal | Query for tasks whose action runs from `C:\Users\Public\` |
| Query languages | KQL (Microsoft Sentinel), SPL (Splunk), ES\|QL (Elastic), YARA-L (Google SecOps) | — |
| Sigma; backend; pipeline | Open YAML rule format; a backend converts it to a query language; a pipeline maps fields and log sources[^sigma-cli] | `-t splunk -p sysmon` |
| Detection strategy (ATT&CK) | ATT&CK's approach for detecting a technique through platform-specific analytics[^attack-objects], with tunable "mutable elements"[^det0441] | DET0441 for scheduled tasks |
| Disposition | The SOC's verdict: true positive (malicious), benign positive (rule worked, activity legitimate), false positive (logic or data wrong), undetermined[^ms-close] | The ChatApp updater: benign positive |
| False negative | Malicious activity that raised no alert | A task created via PowerShell, missed by rule A |
| Precision and recall | Low probability of alerting on benign activity; high probability of detecting malicious events[^stp-defs]. Formulas: the Atlas's, see Metrics | 6 of 40 alerts true → precision 15% |
| Robustness | How hard a signal is to avoid while still achieving the goal[^stp-quality] | Task-registration events beat a tool's command line |
| Implementation | A behaviourally distinct way to perform a technique[^stp-impl] | PowerShell, a registry write, schtasks or XML for a scheduled task |
| Emulation; purple team | Reproducing adversary behaviour safely in an authorised lab; attackers and defenders doing it together | An Atomic Red Team test |
| Filter | Logic that suppresses known benign matches, always creating a blind spot | Exclude one updater |
**Mental model: a detection is a small product with a contract.** For *this* behaviour, on *these* assets, using *this* data, it
alerts within *this* time at roughly *this* precision, with *these* gaps accepted. Git stores the contract, tests check it, production
measures it, and review dates force renewal or cancellation.
### How DetectionOps relates to other Ops
These boundaries follow the flow of security work; they are **editorial distinctions**, not a standard.
| Neighbour | What that page owns | Handoff with DetectionOps |
|---|---|---|
| [SecDataOps](../secdataops/) | Onboarding, parsing, normalisation, retention, cost | Data gaps go there to be fixed |
| [ThreatOps](../threatops/) | Intelligence lifecycle, hunting | A validated hunt finding becomes a detection request |
| [SOC Ops](../soc-ops/) | Triage, tiers, shifts, managed detection and response (MDR) | Dispositions and tuning requests flow back |
| [ResponseOps](../responseops/) | Containment, eradication, evidence | Missed or late detections become requests |
| [AIOps for Security](../aiops-for-security/) | ML, UEBA, LLM assistance, autonomy limits | ML is evaluated there; rules here |
| [DevSecOps](../devsecops/) | Security in software delivery | Pipeline practice; application logs |
| [SecOps](../secops/) | Security–IT collaboration | Operations announces changes that break detections |
::: figure Where DetectionOps sits in the flow of security work
```mermaid
flowchart TB
TO["ThreatOps"]
RO["ResponseOps"]
SD["SecDataOps"]
DO["DetectionOps"]
SOC["SOC Ops"]
TO -->|"requests"| DO
RO -->|"misses"| DO
SD -->|"events"| DO
DO -->|"data gaps"| SD
DO -->|"alerts, runbooks"| SOC
SOC -->|"dispositions"| DO
SOC -->|"incidents"| RO
classDef core fill:#e8f3f2,stroke:#16756f,color:#122029
classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029
class DO core
class TO,RO,SD,SOC neutral
```
Text equivalent: detection requests arrive from ThreatOps (intelligence, hunts) and ResponseOps (missed detections), normalised
events from SecDataOps (data gaps go back), alerts with runbooks go to SOC Ops, dispositions return, and
confirmed incidents go from SOC Ops to ResponseOps.
:::
## Principles and operating model
### Charter and principles
No body publishes a DetectionOps manifesto. This charter is **the Atlas's proposal**, distilled from the ADS framework, the SANS life
cycle, Summiting the Pyramid and the maturity models cited here.
**Mission.** Detections for the threats that matter here, with known coverage, noise and gaps, kept true as things change.
1. **Threat-informed, asset-aware.** Every detection traces to a threat that can reach something you run.
2. **Behaviour over artefacts.** Prefer signals an adversary cannot easily avoid; hashes and tool names are short-lived
additions[^pyramid][^stp-quality].
3. **Everything as code, everything reviewed.** Rules, tests, filters and mappings live in version control; no console-only edits.
4. **Tests are part of the rule.** A rule without a positive and a negative test is a draft; accepted blind spots are tests too.
5. **Measure in production.** Judge rules by analysts' dispositions and by emulation results, not by whether they compile.
6. **Every exclusion is a decision** with an owner, an expiry and a record of what it hides.
7. **Retire on purpose**, with a reason, rather than silently disabling.
**Non-goals.** DetectionOps does not promise to detect everything, replace prevention, or own the SOC's queue.
### Roles, ownership and RACI
| Activity | Detection engineer | Detection lead | SOC | ThreatOps | SecDataOps | Platform owner |
|---|---|---|---|---|---|---|
| Accept and prioritise a request | C | A | C | R (intel input) | C | I |
| Write rule, tests and strategy | R | A | C | C | C | I |
| Peer review | R (second engineer) | A | C | I | C (fields) | I |
| Deploy to production | R | A | I | I | I | C (change window) |
| Triage alerts, record dispositions | I | I | A/R | I | I | I |
| Tune or add a filter | R | A | C (evidence) | I | I | I |
| Fix a data gap | C | I | I | I | A/R | C |
| Retire a rule | R | A | C | C | I | I |
R = responsible, A = accountable, C = consulted, I = informed. Small teams merge roles; each decision keeps one accountable person.
### The detection lifecycle
The SANS poster's life cycle runs Requirements Discovery, Triage, Investigation, Development, Testing, Deployment[^sans-poster]; CTID
stresses that its workflow steps "are not strictly linear"[^stp-workflow]; Sigma's `status` marks maturity (`experimental`, `test`,
`stable`), replacement (`deprecated`) or a rule that cannot be used as it stands (`unsupported`)[^sigma-spec]. The Atlas adds the
states needed after deployment.
::: figure Lifecycle of one detection, from request to retirement
```mermaid
stateDiagram-v2
direction TB
[*] --> Requested
Requested --> Parked: no data yet
Parked --> Requested: data arrives
Requested --> Rejected: low priority
Requested --> Building: accepted
Building --> Testing: rule and tests
Testing --> Building: test fails
Testing --> Staged: CI and review pass
Staged --> Production: volume as expected
Staged --> Building: noisy or silent
Production --> Tuning: verdicts or miss
Tuning --> Production: fix merged
Production --> Deprecated: replaced, obsolete or costly
Deprecated --> [*]
Rejected --> [*]
```
Text equivalent: a request without data is parked until data arrives; a low-priority one is rejected; others are built. Failing tests
return it to building; passing CI and review stages it. Expected volume promotes it; noise or silence sends it back. Dispositions or an
emulation miss send it to tuning; replaced, obsolete or costly rules are deprecated.
:::
**Staging deserves its own discipline.** Run new and changed rules where they cannot page anyone (a test tenant, a separate index,
alerting off) for about seven days and compare volume and samples with the request. OpenTide stages from the pull
request and deploys production on merge[^opentide-site].
### Operating model and governance
| Cadence | What happens | Output |
|---|---|---|
| Per change | Pull request with rule, tests, strategy; CI; one reviewer; staged run | Merged change, staging evidence |
| Weekly | Backlog triage with the SOC lead; review of the noisiest rules | Ranked backlog, tuning tickets |
| Monthly | Health review: silent rules, failing checks, expired filters | Fix or retire decisions |
| Quarterly | Coverage review against the threat model and emulation results | Coverage report, priorities |
| On framework change | Remap tags after ATT&CK or schema releases; re-pin toolchain | Updated mappings, green CI |
**Change classes (Atlas proposal).** *Standard* follows the full path. *Emergency* (active campaign) may skip staging, never tests or
review, and is re-reviewed within five working days. *Filter* changes need SOC evidence and an expiry. Align with [ITOps](../itops/)
change routes where they apply.
## Architecture and workflow
### Reference architecture and components
The parts are ordinary (repository, CI runner, converter, platform APIs); what makes it DetectionOps is that every arrow is automated
or recorded.
::: figure Reference architecture for detection-as-code
```mermaid
flowchart TB
IN["Inputs: intel, hunts, incident lessons, SOC requests"]
REPO["Repository: rules, tests, filters, strategies, mappings"]
CI["Pipeline: lint, unit tests, convert per platform"]
STG["Staging: shadow mode, no paging"]
PRD["Production SIEM and EDR"]
HM["Health monitoring: rule runs, data presence, canaries"]
FB["Feedback: dispositions, emulation results"]
COV["Coverage store: implementations, evidence, gaps"]
IN --> REPO
REPO --> CI
CI --> STG
STG --> PRD
PRD --> HM
HM --> FB
FB --> COV
FB -.->|"changes as pull requests"| REPO
classDef core fill:#e8f3f2,stroke:#16756f,color:#122029
classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029
class REPO,CI core
class IN,STG,PRD,HM,FB,COV neutral
```
Text equivalent: requests feed a repository of rules, tests, filters, strategies and mappings. A pipeline lints, tests and converts
them for staging in shadow mode, then production SIEM and EDR. Health checks (rule runs, data, canaries), dispositions and emulation
results form the feedback, which fills a coverage store and returns changes as pull requests.
:::
| Component | Responsibility | Design choices and trade-offs |
|---|---|---|
| Repository | Source of truth for rules, tests, filters, mappings, docs | Private (see Quality, security and compliance) |
| Rule format | Sigma for portability, or native KQL, SPL, ES\|QL, YARA-L | See the [judgement call](#a-judgement-call-sigma-as-the-source-of-truth-or-native-queries-as-code) |
| Tests | Positive, negative and known-gap fixtures; regression samples | Synthetic is fast; real samples catch parser reality |
| Pipeline | Lint, validate, test, convert, deploy | Pin versions; fail on validation issues |
| Health monitoring | Find rules that stop running, seeing data or matching | Rule-run logs; data-presence and canary checks per rule |
| Coverage store | Rules mapped to techniques, implementations and assets | Evidence from emulation, not tags |
### End-to-end workflow
::: figure From detection request to production: the gates and who passes them
```mermaid
flowchart TB
REQ["Detection request: hunt finding, intel or incident"]
LIB["Search SigmaHQ and vendor content first"]
PR["Pull request: rule, tests, strategy"]
REV["Second engineer reviews"]
STG["Staging: shadow mode, no paging"]
TAG["Maintainer tags release-* on main"]
PRD["Production: alerts reach the SOC"]
REQ -->|"accepted"| LIB
LIB --> PR
PR -->|"CI green; no secrets used"| REV
REV -->|"approve; merge to protected main"| STG
STG -->|"7 days; promotion checklist passed"| TAG
TAG -->|"production reviewer approves, not the author"| PRD
PRD -.->|"dispositions, tuning requests"| REQ
classDef core fill:#e8f3f2,stroke:#16756f,color:#122029
classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029
classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029
class PR,STG,PRD core
class REV,TAG decision
class REQ,LIB neutral
```
Text equivalent: an accepted request starts with a search of existing content. A pull request must pass CI without secrets; a second
engineer approves the merge to protected `main`, which deploys to staging. After seven days and the promotion checklist, a maintainer
tags `release-*` on `main` and a reviewer other than the author approves production. SOC dispositions start the next change.
:::
**Exception paths.**
- **No data.** Park the request as a data gap for [SecDataOps](../secdataops/), naming the fields required; never write a rule
against data you do not have.
- **Tests or conversion fail.** The pull request cannot merge. Conversion failures usually mean an unsupported modifier or unmapped
field: fix the rule or pipeline, not the gate.
- **Staging is noisy.** Sample matches with an analyst before adding any filter.
- **Staging is silent.** Replay a positive sample through the platform; a rule that cannot be shown to fire stays out of production.
## Practices, standards and tools
### Core practices and playbooks
| Practice | Trigger | Owner | Output |
|---|---|---|---|
| Requirement intake | Intel, hunt, incident, audit finding | Detection lead | Request: threat, assets, data, priority |
| Search existing content | Accepted request | Engineer | Community or vendor rule to adapt, run against your fixtures |
| Strategy before query | Accepted request | Engineer | ADS-style strategy document |
| Test-first rule writing | Strategy agreed | Engineer | Failing positive test, then rule, then negatives |
| Review and staged release | Pull request, merge | Second engineer, pipeline | Approval; shadow-mode evidence |
| Disposition feedback | Every closed alert | SOC | True, benign or false positive, or undetermined, per rule |
| Health checks | Daily | Pipeline | Run status, data presence and canary result per rule |
| Emulation campaign | Quarterly | Purple team | Detected or missed, per implementation |
| Retirement | Obsolete, replaced, too costly | Engineer | `deprecated` status with reason and replacement |
**Search the community library first.** For this page's behaviour SigmaHQ already ships "Schedule Task Creation From Env Variable Or
Potentially Suspicious Path Via Schtasks.EXE" and an event-4698 rule, "Suspicious Scheduled Task Creation"[^sigmahq-sched]. Run your
fixtures against them, keep their filters where they fit and record how your version differs; the labs start from scratch only to
teach each step.
**Write the strategy before the query.** Palantir's ADS template is a compact checklist: Goal, Categorization, Strategy
Abstract, Technical Context, Blind Spots and Assumptions, False Positives, Validation, Priority, Response[^ads]. Blind spots and
validation keep coverage claims honest.
**Write the failing test first.** Panther recommends at least two tests per detection, "one false positive and one true
positive"[^panther]. Add a **known-gap test** for malicious activity the rule deliberately misses, so any change to the gap forces the
documentation to change ([Lab 2](#lab-2-intermediate-test-first-tuning-and-the-filter-trade-off)).
**Treat tuning as a change, not a click.** A filter needs evidence, a scope (one rule, or a Sigma filter for many[^sigma-filters]), an
owner and an expiry; CTID warns that filters "should be evaluated for the blind spots they may create"[^stp-quality].
### Methods, standards and frameworks
| Source | Type | What it gives DetectionOps |
|---|---|---|
| Sigma Rules Specification v2.1.0 (August 2025)[^sigma-spec] | Normative for Sigma files | Rule structure, status and levels, conditions, correlation and filters |
| MITRE ATT&CK v20 (October 2026)[^attack-v20] | Knowledge base | Techniques to map; Detection Strategies, Analytics, Data Components |
| Summiting the Pyramid (CTID)[^stp-defs] | Method | Robustness levels, detection quality, implementation coverage, mapping rules |
| SANS detection engineering life cycle[^sans-poster] | Training reference | Six stages; seven criteria for judging requests |
**Sigma in one paragraph.** A rule needs a `title`, a `logsource` and a `detection` with a `condition`; the optional `id` should be a
random version 4 UUID. "All values are treated as case-insensitive strings", while regular expressions are case-sensitive by default;
`informational` means "No case or alerting should be triggered"[^sigma-spec].
### Coverage mapping and its limits
Every coverage report should carry CTID's sentence "An ATT&CK tag alone is not evidence of detection coverage"[^stp-mapping]. Use
three measures instead of one heatmap:
1. **Implementation coverage:** how many of a technique's implementations your rules detect, say "2/8 implementations"[^stp-impl].
2. **Detection quality:** how robust and precise is each covering rule[^stp-quality]?
3. **Validation evidence:** was the behaviour emulated recently, and did the rule fire?
**Mappings decay.** ATT&CK v18 (October 2025) replaced per-technique detection text with Detection Strategies and Analytics and
deprecated Data Sources[^attack-v18]; v19 (April 2026) split Defense Evasion into **Stealth** (TA0005) and **Defense Impairment**
(TA0112)[^attack-v19][^attack-tactics]. On 16 October 2026 the Sigma tags appendix still mapped `defense-evasion` to
TA0005[^sigma-tags], while a SigmaHQ rule already carried `attack.stealth`[^sigmahq-tag]. Tactic counts can drop or double after a
release with no change in what you detect; "maintaining those mappings is part of maintaining the detection"[^stp-mapping].
::: caution Tag techniques, derive tactics
Map rules to technique IDs and derive tactics from the ATT&CK release you report against; hand-kept tactic tags drift first. The lab
rules carry technique tags only; SigmaHQ rules also carry tactic tags[^sigmahq-sched], so lint those against the current release on
import.
:::
### Tooling landscape
Named tools are examples verified on 2026-10-16, not recommendations.
| Category | Job | Verified examples (October 2026) | Selection notes |
|---|---|---|---|
| Native languages | Full platform features | Google SecOps YARA-L 2.0[^yaral]; KQL, SPL, ES\|QL | Best fidelity; lock-in |
| Portable format and converter | One rule, many platforms | Sigma with sigma-cli 3.1.0 and pySigma backends[^sigma-cli][^pypi] | Conversion gaps; pin versions |
| Rule collections | Starting content | SigmaHQ (3,000+ rules)[^sigmahq-readme]; Elastic detection-rules[^elastic-dr] | Needs local tests and tuning |
| Vendor DaC tooling | Repository sync and tests | Elastic DaC reference[^elastic-dac]; Sentinel repositories[^sentinel-cicd]; Panther tests[^panther]; Splunk contentctl[^contentctl] | Product prerequisites |
| Threat-to-rule framework | Link threats, objectives, rules | OpenTide (EUPL 1.2)[^opentide-gh] | Worth it when traceability is the pain |
| Tests and emulation | Prove detections fire | Atomic Red Team[^art]; SigmaHQ regression samples[^sigma-regression] | Authorised, isolated labs only |
**Facts that change decisions (October 2026; for Lab 3 and the judgement call).** Sentinel repository connections need the
Owner role on the workspace's resource group; only Defender XDR custom detection rules (Preview) add a Microsoft 365 E5 or
equivalent licence and Defender-portal onboarding. Sentinel in the Azure portal is unsupported after 31 March 2027[^sentinel-cicd],
and Microsoft now recommends custom detections for new rules[^ms-ingest]. Splunk's contentctl is "no longer accepting new pull
requests or feature requests" as investment moves to Detection Studio[^contentctl].
::: caution Pin the Sigma toolchain
pySigma 2.0.0 (4 October 2026) and 2.0.1 (11 October) are out, but sigma-cli 3.1.0 and the Splunk backend 2.1.0 require pySigma below
2[^pypi]. Unpinned CI installs can break or resolve to untested combinations: pin everything and upgrade on purpose, with the full test
suite.
:::
### Automation and AI
Automate the deterministic steps (lint, tests, conversion, deployment, health checks, coverage reports); keep judgement (accepting a
request, a filter or a gap) with people, on the record. SANS's 2025 summary says 45% of organisations used AI for detection[^sans-2025];
the sponsor's 2026 summary says 83% of practitioners use AI tools but only 42% "trust them for core work like tuning
detections"[^sans-2026]. OpenTide argues that agents made writing a query cheap, but not knowing whether it "is valid, maps to a real
threat and is deployed where it should be"[^opentide-site]; generated rules pass the same tests, review and staging.
## DetectionOps in practice
### Reference implementation
The reference implementation covers one behaviour: **persistence through a Windows scheduled task whose action runs from a
user-writable folder** (ATT&CK T1053.005). It is safe to reproduce; ATT&CK lists a logon task running
`C:\Users\Public\test.exe`[^attack-t1053], and DET0441 lists the telemetry: Security events 4698 (task created) and 4702 (task updated) and
Sysmon process, file and registry events[^det0441]. It also shows robustness in miniature: watching the `schtasks.exe` command line "might be
easier for an adversary to evade than tracking task scheduler event IDs"[^stp-defs]. So the repository holds a quick command-line rule
(A) and a sturdier rule on task-creation and task-update events (B), each with tests and a strategy document.
_Example status: structure and files executed in Labs 1–2 on 2026-10-17; deployment adapter not shown._
```text
detections/
rules/windows/
proc_creation_win_schtasks_user_writable_action.yml # rule A (command line)
win_security_task_user_writable_action.yml # rule B (events 4698, 4702)
tests/
d0fcbc79-de80-42d4-ab35-6f19f14047df.yml # cases for rule A
e71410f2-d333-4f3d-bb72-343d74241685.yml # cases for rule B
docs/strategies/ # one ADS-style file per rule
tools/lint_rules.py tools/run_tests.py tools/deploy.py
.github/workflows/detections.yml
CODEOWNERS # rules/ -> detection team
```
Rule A's strategy document follows the ADS sections[^ads].
_Example status: illustrative template, filled in for rule A._
```text
Goal: Detect persistence through scheduled tasks whose action runs
from a user-writable folder.
Categorization: ATT&CK T1053.005 (Execution, Persistence, Privilege Escalation).
Strategy: Sysmon Event ID 1 / process creation: schtasks.exe (by image or
PE OriginalFileName) with /create or -create and an action
under \Users\Public\, \AppData\, \ProgramData\, \Temp\ or
%Public%-style environment variables (a teaching subset of
the folders in the SigmaHQ rules).
Context: Legitimate software rarely registers tasks that run from these
folders; per-user updaters are the known exception.
Blind spots: Tasks created through PowerShell, COM, the API or XML import,
and existing tasks repointed with /change or the API (both
covered by rule B, events 4698 and 4702); direct registry
writes, which need SYSTEM and raise neither rule (see Lab 4);
forward-slash paths such as C:/Users/Public/ (accepted; see
Lab 2, step 4); malware written to the ChatApp updater path
that registers its own task (accepted, see filter_chatapp).
False pos.: Per-user updaters (benign positives); administrator testing.
Validation: In an authorised lab host, create a run-once task whose action
is a copy of a harmless built-in program in C:\Users\Public;
confirm one alert; delete the task and the copy.
Priority: Medium; High when the user is privileged or the host is a server.
Response: Runbook SOC-RB-031: check the task, the binary's signer and
the user's recent logons; escalate to ResponseOps if unknown.
```
### Hands-on labs
Labs 1–2 need only Python 3.8+ and PyYAML; Lab 3 needs GitHub and a SIEM; Lab 4 is a planning exercise.
::: warning Lab safety
Use synthetic events (Labs 1–2) or isolated, authorised lab systems (Labs 3–4). Never emulate on production without written
approval, a named owner and the SOC informed.
:::
#### Lab 1 (beginner): lint rules against house rules
**Objective.** Catch malformed rules before review and enforce house rules Sigma leaves optional. **Prerequisites.** Python 3.8+,
PyYAML (`python3 -m pip install pyyaml`). **Cost and risk.** None.
1. Create `rules/windows/`, `tests/` and `tools/`.
2. Save the linter as `tools/lint_rules.py`:
_Example status: executed on 2026-10-17 (Python 3.8.10, PyYAML 5.3.1)._
```python
"""House-rule linter for Sigma rules (teaching subset; also run sigma check)."""
import pathlib
import re
import sys
import uuid
import yaml
STATUS = {"stable", "test", "experimental", "deprecated", "unsupported"}
LEVEL = {"informational", "low", "medium", "high", "critical"}
TAG = re.compile(r"^[a-z0-9_-]+(\.[a-z0-9_.-]+)+$")
TECHNIQUE = re.compile(r"^attack\.t\d{4}(\.\d{3})?$")
DATE = re.compile(r"^\d{4}-\d{2}-\d{2}$")
def problems(rule, path, tests_dir):
out = []
if not rule.get("title") or len(rule["title"]) > 256:
out.append("title missing or longer than 256 characters")
try:
if uuid.UUID(str(rule.get("id"))).version != 4:
out.append("id is not a version 4 UUID")
except ValueError:
out.append("id missing or not a UUID")
if rule.get("status") not in STATUS:
out.append("status must be in %s" % sorted(STATUS))
if rule.get("level") not in LEVEL:
out.append("level must be in %s" % sorted(LEVEL))
if not DATE.match(str(rule.get("date", ""))):
out.append("date must be YYYY-MM-DD")
src = rule.get("logsource") or {}
if not any(k in src for k in ("category", "product", "service")):
out.append("logsource needs category, product or service")
if "condition" not in (rule.get("detection") or {}):
out.append("detection.condition is missing")
tags = rule.get("tags") or []
bad = [t for t in tags if not TAG.match(t)]
if bad:
out.append("malformed tags: %s" % bad)
if not any(TECHNIQUE.match(t) for t in tags):
out.append("house rule: at least one attack.tNNNN technique tag")
if not rule.get("falsepositives"):
out.append("house rule: list known false positives (or 'Unknown')")
if rule.get("status") == "deprecated" and not rule.get("related"):
out.append("deprecated rule must name its replacement in related")
test_file = tests_dir / ("%s.yml" % rule.get("id"))
needs_tests = rule.get("status") in {"test", "stable", "experimental"}
if needs_tests and not test_file.exists():
out.append("house rule: no test file %s" % test_file.name)
return out
def main(rules_dir="rules", tests_dir="tests"):
failed = 0
for path in sorted(pathlib.Path(rules_dir).rglob("*.yml")):
rule = yaml.safe_load(path.read_text())
found = problems(rule, path, pathlib.Path(tests_dir))
print("%s %s" % ("FAIL" if found else "PASS", path))
for item in found:
print(" - " + item)
failed += bool(found)
print("%d rule(s) failed" % failed)
return 1 if failed else 0
if __name__ == "__main__":
sys.exit(main(*sys.argv[1:]))
```
3. Save rule A, version 1, in `rules/windows/` under the file name given in its first line:
_Example status: executed on 2026-10-17 as input to Labs 1 and 2._
```yaml
# file: proc_creation_win_schtasks_user_writable_action.yml
title: Scheduled Task Created With An Action In A User-Writable Folder
id: d0fcbc79-de80-42d4-ab35-6f19f14047df
status: experimental
description: >-
Detects schtasks.exe creating a task whose action runs from a folder that
any user can write to, a common persistence pattern.
references:
- https://attack.mitre.org/techniques/T1053/005/
author: Detection team
date: 2026-10-16
tags:
- attack.t1053.005
logsource:
category: process_creation
product: windows
detection:
selection_img:
Image|endswith: '\schtasks.exe'
selection_create:
CommandLine|contains: '/create'
selection_path:
CommandLine|contains:
- '\Users\Public\'
- '\Temp\'
condition: all of selection_*
falsepositives:
- Per-user software updaters that install under the user profile
level: medium
```
4. Save a deliberately broken rule as `rules/windows/broken_example.yml` (`id: 1234`, `status: production`,
`date: 16/10/2026`, `level: severe`, the tag `ATT&CK T1053`, no `falsepositives`) and run `python3 tools/lint_rules.py`.
**Expected output** (executed):
```text
FAIL rules/windows/broken_example.yml
- id missing or not a UUID
- status must be in ['deprecated', 'experimental', 'stable', 'test', 'unsupported']
- level must be in ['critical', 'high', 'informational', 'low', 'medium']
- date must be YYYY-MM-DD
- malformed tags: ['ATT&CK T1053']
- house rule: at least one attack.tNNNN technique tag
- house rule: list known false positives (or 'Unknown')
FAIL rules/windows/proc_creation_win_schtasks_user_writable_action.yml
- house rule: no test file d0fcbc79-de80-42d4-ab35-6f19f14047df.yml
2 rule(s) failed
```
5. Delete the broken rule. Rule A still fails until it has tests.
**Validation.** Exit code 1 while any rule fails, 0 once all pass (after Lab 2 step 1). **Troubleshooting.** `ScannerError`: use single
quotes for Windows paths. **Teardown.** Keep the folder for Lab 2.
#### Lab 2 (intermediate): test-first tuning and the filter trade-off
**Objective.** Write positive, negative, out-of-scope and known-gap tests, watch version 1 fail, improve it, and measure a filter's
precision–recall trade-off. **Prerequisites.** Lab 1. **Cost and risk.** None; synthetic events.
1. Save the cases as `tests/d0fcbc79-de80-42d4-ab35-6f19f14047df.yml` (the first follows an ATT&CK procedure example[^attack-t1053]):
_Example status: executed on 2026-10-17._
```yaml
rule_id: d0fcbc79-de80-42d4-ab35-6f19f14047df
cases:
- name: logon task runs from Users\Public
expect: match
event:
Image: 'C:\Windows\System32\schtasks.exe'
OriginalFileName: schtasks.exe
CommandLine: >-
schtasks /create /tn "mysc" /tr C:\Users\Public\test.exe
/sc ONLOGON /ru "System"
- name: action in a roaming profile
expect: match
event:
Image: 'C:\Windows\System32\schtasks.exe'
OriginalFileName: schtasks.exe
CommandLine: >-
schtasks /CREATE /tn Sync /sc daily
/tr C:\Users\alice\AppData\Roaming\sync\s.exe
- name: renamed copy of schtasks
expect: match
event:
Image: 'C:\Users\Public\tsk.exe'
OriginalFileName: schtasks.exe
CommandLine: 'tsk.exe /create /tn upd /tr C:\Users\Public\u.exe /sc onlogon'
- name: dash switches
expect: match
event:
Image: 'C:\Windows\System32\schtasks.exe'
OriginalFileName: schtasks.exe
CommandLine: 'schtasks -create -tn upd -tr C:\Users\Public\u.exe -sc onlogon'
- name: unexpanded environment variable
expect: match
event:
Image: 'C:\Windows\System32\schtasks.exe'
OriginalFileName: schtasks.exe
CommandLine: 'schtasks /create /tn upd /tr %PUBLIC%\u.exe /sc onlogon'
- name: action under ProgramData
expect: match
event:
Image: 'C:\Windows\System32\schtasks.exe'
OriginalFileName: schtasks.exe
CommandLine: 'schtasks /create /tn upd /tr C:\ProgramData\u.exe /sc onlogon'
- name: hides behind the updater path
expect: match
event:
ParentImage: 'C:\Windows\System32\cmd.exe'
Image: 'C:\Windows\System32\schtasks.exe'
OriginalFileName: schtasks.exe
CommandLine: >-
schtasks /create /sc onlogon /tr C:\Users\Public\u.exe
/tn C:\Users\eve\AppData\Local\ChatApp\Update.exe
- name: vendor updater in Program Files
expect: no_match
event:
Image: 'C:\Windows\System32\schtasks.exe'
OriginalFileName: schtasks.exe
CommandLine: >-
schtasks /create /tn "Vendor\Update" /sc daily
/tr "C:\Program Files\Vendor\update.exe"
- name: administrator lists tasks
expect: no_match
event:
Image: 'C:\Windows\System32\schtasks.exe'
OriginalFileName: schtasks.exe
CommandLine: 'schtasks /query /fo LIST /v'
- name: chat app per-user updater
expect: no_match
event:
ParentImage: 'C:\Users\bob\AppData\Local\ChatApp\Update.exe'
Image: 'C:\Windows\System32\schtasks.exe'
OriginalFileName: schtasks.exe
CommandLine: >-
schtasks /create /tn "ChatApp Update" /sc daily
/tr "C:\Users\bob\AppData\Local\ChatApp\Update.exe --check" /f
- name: malware at the updater path
expect: known_gap
owner: filter_chatapp
event:
ParentImage: 'C:\Users\eve\AppData\Local\ChatApp\Update.exe'
Image: 'C:\Windows\System32\schtasks.exe'
OriginalFileName: schtasks.exe
CommandLine: >-
schtasks /create /tn "ChatApp Update" /sc onlogon
/tr C:\Users\eve\AppData\Local\ChatApp\Update.exe
- name: task registered from PowerShell
expect: out_of_scope
owner: rule e71410f2
event:
Image: 'C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe'
OriginalFileName: PowerShell.EXE
CommandLine: 'powershell.exe -File C:\Users\Public\register.ps1'
- name: task repointed with /change
expect: out_of_scope
owner: rule e71410f2
event:
Image: 'C:\Windows\System32\schtasks.exe'
OriginalFileName: schtasks.exe
CommandLine: 'schtasks /change /tn Existing /tr C:\Users\Public\u.exe'
```
2. Save the runner as `tools/run_tests.py`: a teaching subset of Sigma with case-insensitive matching as the specification
requires[^sigma-spec]. It ignores `logsource`, which real backends map through pipelines.
_Example status: executed on 2026-10-17 (Python 3.8.10, PyYAML 5.3.1). Teaching code, not a replacement for pySigma._
```python
"""Run labelled events against Sigma rules (teaching subset of Sigma).
Supports: maps and lists of maps; modifiers contains, startswith, endswith, all,
windash; * and ? wildcards; conditions with and/or/not, brackets, '1 of x*',
'all of x*'. Matching is case-insensitive, as the Sigma specification requires.
"""
import fnmatch
import itertools
import pathlib
import re
import sys
import yaml
DASHES = ("-", "/", "\u2013", "\u2014", "\u2015") # windash characters
def windash(value):
"""Every variant of value with each flag's leading - or / swapped."""
parts = re.split(r"\B[-/]\b", value)
return ["".join(p + d for p, d in zip(parts, combo + ("",)))
for combo in itertools.product(DASHES, repeat=len(parts) - 1)]
def value_matches(actual, expected, mods):
if "windash" in mods:
rest = [m for m in mods if m != "windash"]
return any(value_matches(actual, v, rest) for v in windash(str(expected)))
a, e = str(actual).lower(), str(expected).lower()
if "contains" in mods:
return e in a
if "startswith" in mods:
return a.startswith(e)
if "endswith" in mods:
return a.endswith(e)
return fnmatch.fnmatchcase(a, e)
def map_matches(event, mapping):
for key, expected in mapping.items():
field, *mods = key.split("|")
if field not in event:
return False
values = expected if isinstance(expected, list) else [expected]
hits = [value_matches(event[field], v, mods) for v in values]
if not (all(hits) if "all" in mods else any(hits)):
return False
return True
def search_matches(event, search):
if isinstance(search, list): # list of maps = OR
return any(map_matches(event, item) for item in search)
return map_matches(event, search)
TOKEN = re.compile(r"\(|\)|1 of [\w*]+|all of [\w*]+|\band\b|\bor\b|\bnot\b|[\w*]+")
def evaluate(condition, results):
tokens = TOKEN.findall(condition)
pos = [0]
def peek():
return tokens[pos[0]] if pos[0] < len(tokens) else None
def take():
pos[0] += 1
return tokens[pos[0] - 1]
def group(tok):
quant, pattern = tok.split(" of ")
names = [n for n in results if fnmatch.fnmatchcase(n, pattern)]
hits = [results[n] for n in names]
return all(hits) if quant == "all" else any(hits)
def atom():
tok = take()
if tok == "(":
val = expr()
take() # ")"
return val
if tok == "not":
return not atom()
if " of " in tok:
return group(tok)
return results[tok]
def term():
val = atom()
while peek() == "and":
take()
rhs = atom()
val = val and rhs
return val
def expr():
val = term()
while peek() == "or":
take()
rhs = term()
val = val or rhs
return val
return expr()
def rule_matches(rule, event):
det = rule["detection"]
results = {k: search_matches(event, v)
for k, v in det.items() if k != "condition"}
return evaluate(det["condition"], results)
def main(rules_dir="rules", tests_dir="tests"):
rules = {}
for path in pathlib.Path(rules_dir).rglob("*.yml"):
rule = yaml.safe_load(path.read_text())
rules[rule["id"]] = rule
tp = fp = fn = tn = failures = 0
for test_path in sorted(pathlib.Path(tests_dir).glob("*.yml")):
suite = yaml.safe_load(test_path.read_text())
rule = rules[suite["rule_id"]]
print("%s (%s)" % (rule["title"], rule["status"]))
for case in suite["cases"]:
fired = rule_matches(rule, case["event"])
want = case["expect"]
if want == "out_of_scope":
print(" SKIP %-34s fired=%s owner=%s"
% (case["name"], fired, case["owner"]))
continue
if want == "known_gap": # malicious, accepted blind spot
fn += not fired
failures += fired # gap closed: update the test
print(" GAP %-34s fired=%s accepted=%s"
% (case["name"], fired, case["owner"]))
continue
ok = fired == (want == "match")
failures += not ok
tp += fired and want == "match"
fp += fired and want == "no_match"
fn += (not fired) and want == "match"
tn += (not fired) and want == "no_match"
print(" %s %-34s expect=%-8s fired=%s"
% ("PASS" if ok else "FAIL", case["name"], want, fired))
precision = tp / (tp + fp) if tp + fp else float("nan")
recall = tp / (tp + fn) if tp + fn else float("nan")
print("TP=%d FP=%d FN=%d TN=%d precision=%.2f recall=%.2f failures=%d"
% (tp, fp, fn, tn, precision, recall, failures))
return 1 if failures else 0
if __name__ == "__main__":
sys.exit(main(*sys.argv[1:]))
```
3. Run `python3 tools/run_tests.py` against version 1. **Expected output** (executed; exit code 1):
```text
Scheduled Task Created With An Action In A User-Writable Folder (experimental)
PASS logon task runs from Users\Public expect=match fired=True
FAIL action in a roaming profile expect=match fired=False
FAIL renamed copy of schtasks expect=match fired=False
FAIL dash switches expect=match fired=False
FAIL unexpanded environment variable expect=match fired=False
FAIL action under ProgramData expect=match fired=False
PASS hides behind the updater path expect=match fired=True
PASS vendor updater in Program Files expect=no_match fired=False
PASS administrator lists tasks expect=no_match fired=False
PASS chat app per-user updater expect=no_match fired=False
GAP malware at the updater path fired=False accepted=filter_chatapp
SKIP task registered from PowerShell fired=False owner=rule e71410f2
SKIP task repointed with /change fired=False owner=rule e71410f2
TP=2 FP=0 FN=6 TN=3 precision=1.00 recall=0.25 failures=5
```
4. Replace the `detection` block with version 2. It accepts the name compiled into the executable's header (`OriginalFileName`),
adds `\AppData\`, `\ProgramData\` and environment variables, and uses `windash` because "Windows command line flags can often be indicated by both
characters"[^sigma-mod]. Its filter for the updater the SOC reported checks two fields, parent process and action path; an `endswith`
on the whole command line would hide the "hides behind the updater path" case and miss this updater's quoted `/tr`. The path
lists are a teaching subset: production rules add folders such as `\Perflogs\`, `\Downloads\` and `\Desktop\` (compare the
SigmaHQ rules[^sigmahq-sched]); forward-slash spellings (`C:/Users/Public/`) match neither rule and are an accepted gap in the
strategy.
_Example status: executed on 2026-10-17._
```yaml
detection:
selection_img:
- Image|endswith: '\schtasks.exe'
- OriginalFileName: 'schtasks.exe'
selection_create:
CommandLine|contains|windash: '/create'
selection_path:
CommandLine|contains:
- '\Users\Public\'
- '\AppData\'
- '\ProgramData\'
- '\Temp\'
- '%Public%'
- '%AppData%'
- '%LocalAppData%'
- '%Temp%'
- '%TMP%'
filter_chatapp:
ParentImage|endswith: '\AppData\Local\ChatApp\Update.exe'
CommandLine|contains: '\AppData\Local\ChatApp\Update.exe'
condition: all of selection_* and not filter_chatapp
```
5. Run the tests again. **Expected output** (executed; exit code 0):
```text
Scheduled Task Created With An Action In A User-Writable Folder (experimental)
PASS logon task runs from Users\Public expect=match fired=True
PASS action in a roaming profile expect=match fired=True
PASS renamed copy of schtasks expect=match fired=True
PASS dash switches expect=match fired=True
PASS unexpanded environment variable expect=match fired=True
PASS action under ProgramData expect=match fired=True
PASS hides behind the updater path expect=match fired=True
PASS vendor updater in Program Files expect=no_match fired=False
PASS administrator lists tasks expect=no_match fired=False
PASS chat app per-user updater expect=no_match fired=False
GAP malware at the updater path fired=False accepted=filter_chatapp
SKIP task registered from PowerShell fired=False owner=rule e71410f2
SKIP task repointed with /change fired=False owner=rule e71410f2
TP=7 FP=0 FN=1 TN=3 precision=1.00 recall=0.88 failures=0
```
6. **See the trade-off.** Remove `and not filter_chatapp` and rerun: on 2026-10-17 the result was
`TP=7 FP=1 FN=0 TN=2 precision=0.88 recall=1.00 failures=2`. Without the filter the updater alerts (in production, a benign
positive); with it, malware written to that path is invisible, and the known-gap test records the decision. Restore the filter.
7. **Cover the out-of-scope cases with rule B.** Save the rule on events 4698 and 4702 and its tests, then run a third time.
_Example status: executed on 2026-10-17._
```yaml
title: Scheduled Task Registered Or Updated To Run From A User-Writable Folder
id: e71410f2-d333-4f3d-bb72-343d74241685
status: experimental
description: >-
Detects any scheduled task registration or update (schtasks, PowerShell,
COM or API) whose action runs from a user-writable folder, using Security
events 4698 (task created) and 4702 (task updated).
references:
- https://attack.mitre.org/detectionstrategies/DET0441/
author: Detection team
date: 2026-10-16
tags:
- attack.t1053.005
logsource:
product: windows
service: security
definition: Requires the audit subcategory "Audit Other Object Access Events"
detection:
selection_created:
EventID: 4698
TaskContent|contains:
- '\Users\Public\'
- '\AppData\'
- '\ProgramData\'
- '\Temp\'
- '%Public%'
- '%AppData%'
- '%LocalAppData%'
- '%Temp%'
- '%TMP%'
selection_updated:
EventID: 4702
TaskContentNew|contains:
- '\Users\Public\'
- '\AppData\'
- '\ProgramData\'
- '\Temp\'
- '%Public%'
- '%AppData%'
- '%LocalAppData%'
- '%Temp%'
- '%TMP%'
condition: 1 of selection_*
falsepositives:
- Per-user software updaters that install under the user profile
level: medium
```
```yaml
rule_id: e71410f2-d333-4f3d-bb72-343d74241685
cases:
- name: task registered from PowerShell
expect: match
event:
EventID: 4698
TaskName: '\Updater'
TaskContent: 'C:\Users\Public\u.exe'
- name: action as an environment variable
expect: match
event:
EventID: 4698
TaskName: '\Upd'
TaskContent: '%PUBLIC%\u.exe'
- name: existing task repointed to TMP
expect: match
event:
EventID: 4702
TaskName: '\Microsoft\StartListener'
TaskContentNew: '%TMP%\listener.exe'
- name: vendor task in Program Files
expect: no_match
event:
EventID: 4698
TaskName: '\Vendor\Update'
TaskContent: 'C:\Program Files\Vendor\update.exe'
- name: vendor update in Program Files
expect: no_match
event:
EventID: 4702
TaskName: '\Vendor\Update'
TaskContentNew: 'C:\Program Files\Vendor\update.exe'
- name: inline encoded PowerShell action
expect: known_gap
owner: backlog DR-0142
event:
EventID: 4698
TaskName: '\Sync'
TaskContent: >-
powershell.exe
-enc BASE64_PAYLOAD
```
**Expected output of the third run** (executed; last lines, exit code 0):
```text
Scheduled Task Registered Or Updated To Run From A User-Writable Folder (experimental)
PASS task registered from PowerShell expect=match fired=True
PASS action as an environment variable expect=match fired=True
PASS existing task repointed to TMP expect=match fired=True
PASS vendor task in Program Files expect=no_match fired=False
PASS vendor update in Program Files expect=no_match fired=False
GAP inline encoded PowerShell action fired=False accepted=backlog DR-0142
TP=10 FP=0 FN=2 TN=5 precision=1.00 recall=0.83 failures=0
```
**What to notice.** Event 4698 exists only if "Audit Other Object Access Events" is enabled[^ms-4698], and Sigma's `definition` is
documentation, not configuration[^sigma-spec]: put the dependency in the request and a health check. Rule B ignores which tool
registered the task, provided Task Scheduler logs it, and sees a task repointed with `schtasks /change` or the API through event
4702, whose new definition is in `TaskContentNew`, not `TaskContent`[^ms-4702]. A task written straight into the registry (needs SYSTEM and a service restart)
"can bypass the generation of scheduled task creation event logs, such as 4698 and 106"[^ghosttask]; look for it in DET0441's Sysmon
registry events[^det0441]. **Validation.** Break one positive fixture and confirm the run fails. **Troubleshooting.** `KeyError`: a
test's `rule_id` matches no rule `id`. **Teardown.** Keep the folder for Lab 3.
#### Lab 3 (advanced): a CI pipeline with staged deployment
**Objective.** Validate every pull request, deploy merges in shadow mode, and promote only from a reviewed release tag on `main`.
**Prerequisites.** A private GitHub repository with the
Lab 2 folder; a SIEM API with a least-privileged token per environment; your own `tools/deploy.py` adapter (for Splunk, around the
`saved/searches` REST endpoint; not shown). **Cost and risk.** CI minutes; a bad production deploy can flood the SOC.
_Example status: syntax-checked, not executed (needs GitHub, a SIEM and Python 3.10 or later for sigma-cli 3.1.0). Action majors are
current as of October 2026[^actions]; pin each action to a full commit SHA in production._
```yaml
name: detections
on:
pull_request:
paths: ["rules/**", "tests/**", "tools/**", ".github/workflows/**"]
push:
branches: [main]
tags: ["release-*"]
permissions:
contents: read
jobs:
validate:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v7 # pin to a full commit SHA in production
- uses: actions/setup-python@v7
with:
python-version: "3.12"
- name: Install the pinned toolchain
run: >-
python -m pip install "sigma-cli==3.1.0" "pysigma==1.5.1"
"pysigma-backend-splunk==2.1.0" "pysigma-pipeline-sysmon==2.0.0"
"pysigma-pipeline-windows==2.0.1" "pyyaml==6.0.3"
- name: House rules and unit tests
run: |
python tools/lint_rules.py
python tools/run_tests.py
- name: Sigma validators (fail on issues, not only on errors)
run: |
mkdir -p build
sigma check --fail-on-issues --junitxml build/sigma-check.xml rules/
- name: Convert for Splunk
run: >-
sigma convert -t splunk -p sysmon -p windows-logsources
-f savedsearches -o build/savedsearches.conf rules/
- uses: actions/upload-artifact@v7
with:
name: converted
path: build/
deploy-staging:
if: github.ref == 'refs/heads/main'
needs: validate
runs-on: ubuntu-24.04
environment: detections-staging # deployment branch rule: main only
steps:
- uses: actions/checkout@v7
- uses: actions/download-artifact@v8
with:
name: converted
path: build/
- name: Deploy in shadow mode (no notable events, no paging)
env:
SIEM_TOKEN: ${{ secrets.SIEM_TOKEN }}
run: python tools/deploy.py --mode shadow build/savedsearches.conf
deploy-production:
if: startsWith(github.ref, 'refs/tags/release-')
needs: validate
runs-on: ubuntu-24.04
environment: detections-production # tag rule release-*, required reviewer
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0 # all branches, for the next step
- name: Refuse a tag whose commit is not on main
run: git merge-base --is-ancestor "$GITHUB_SHA" origin/main
- uses: actions/download-artifact@v8
with:
name: converted
path: build/
- name: Promote to alerting
env:
SIEM_TOKEN: ${{ secrets.SIEM_TOKEN }}
run: python tools/deploy.py --mode alert build/savedsearches.conf
```
**Why each line is there.**
- `sigma check` passes on validation issues unless you add `--fail-on-issues`[^sigma-check].
- `paths` includes the workflow folder, so a pin bump runs `validate` instead of shipping untested or, if `validate` is a required
check, never merging.
- Conversion follows the sigma-cli README[^sigma-cli]. The Sysmon pipeline rewrites only generic categories such as `process_creation`;
`windows-logsources` adds rule B's Windows `Channel` condition[^pipeline-windows]. Check the converted rule B for it.
- **Who can reach each SIEM token.** The validate job runs unreviewed pull-request code: `contents: read`, no environment. Keep each
token only as an environment secret, never a repository or organisation secret: such secrets "are only available to workflow jobs
that reference the environment"[^gh-envs], on any branch unless restricted. Limit `detections-staging` to `main` with a deployment
branch rule, and protect `main` with required review and status checks; the `if:` lines are convenience, not control.
- **What makes a release reviewed.** A deployment tag rule matches only the name in `GITHUB_REF`[^gh-envs]; any writer can push
`release-x` on any commit unless a tag ruleset restricts creating and updating `release-*` to maintainers[^gh-rulesets]. The ancestry
step refuses tags outside `main`. On `detections-production`, require reviewers, prevent self-review and disallow the default
administrator bypass; a job awaiting approval cannot read the environment's secrets.
- **Plan limits (October 2026).** In private repositories, environments, their secrets and branch and tag rules need GitHub Pro, Team
or Enterprise[^gh-manage-envs]; rulesets need Pro, Team or Enterprise Cloud[^gh-rulesets]; required reviewers need Enterprise[^gh-envs][^gh-manage-envs],
and self-hosted runners are not isolated[^gh-envs]. Each token may only create and update detections in one app or workspace.
**Expected output.** A pull request runs only `validate`; a merge to `main` runs `deploy-staging`; a `release-*` tag on `main` waits
for a reviewer, then deploys; a tag elsewhere fails the ancestry step. **Validation.** A broken fixture fails the pull request and
deploys nothing; a branch workflow naming `detections-staging` is refused. **Troubleshooting.** `ResolutionImpossible` from pip: one
package moved to pySigma 2 early; keep pins together. **Teardown.** Delete the test tag and staging searches; revoke lab tokens.
#### Lab 4 (advanced, planning exercise): design one validation run
**Objective.** Turn "we detect scheduled-task persistence" into evidence. **Prerequisites.** An authorised, isolated lab host
sending telemetry to staging; the SOC informed. **Status.** A worksheet, nothing executed. Its Expected output, Validation and
Teardown lines are the acceptance criteria:
```text
Behaviour: T1053.005 Scheduled Task (persistence)
Implementations: 1 schtasks.exe 2 PowerShell cmdlet 3 COM/API 4 XML import
5 direct registry write (needs SYSTEM)
6 existing task repointed (schtasks /change or the API)
Expected signal: impl 1 -> rules A and B; impl 2-4 -> rule B only (4698);
impl 5 -> neither: known gap (look at Sysmon 13/14, DET0441);
impl 6 -> rule B only (4702)
Test source: Atomic Red Team test chosen by GUID (GUIDs do not change)
Harmless payload: copy of a built-in program; no network, no credentials
Expected output: each implementation raises its expected alert within 15 min
Validation: alert and raw event fall inside the run's window, per GUID
Evidence kept: alert IDs, raw events, test GUID, time window, engineer
Teardown: delete the task and the copied file; confirm with a query
Stop condition: any unexpected impact on the host or the SOC queue
```
Atomic Red Team tests are designed to run "in five minutes or less", many with cleanup commands[^art-site]; select them by GUID
because GUIDs "are guaranteed to not change"[^art-exec]; the detected-or-missed record per implementation feeds validated coverage.
### Use cases and scenarios
**1. Sourced: the European Commission's security operations centre.** Before OpenTide, threat information went into spreadsheets whose
"rows could not be linked to the rules they deployed", analyst Claus Houmann recalls. Since September 2022, Remi Seguy's team has used
it to "manage hundreds of threat vectors, each linked to detection objectives and deployed detection rules"; Seguy says "Every rule is
validated, greatly reducing the risk of errors"[^ec-ospo]. *Lesson:* traceability was the problem; tooling followed. *Caveat:* the
Commission's own account; no independent evaluation found.
::: example Illustrative scenario: a managed detection provider with many tenants
A provider serves 60 customers on two SIEMs. Rules are written once in Sigma with per-platform pipelines; tenant exclusions live in
Sigma filter files[^sigma-filters] with owners and expiries. *Success:* precision per rule and tenant; time from report to staged
rule. *Trade-off:* conversion gaps force native overrides. *Pitfall:* one tenant's filter silently applied to all.
:::
::: example Illustrative scenario: a three-person team relying on vendor content
A mid-sized company runs vendor EDR and SIEM content. It keeps its 25 custom rules and all exclusions in Git, reviews filters monthly
with the SOC, and emulates its ten priority techniques quarterly. *Success:* every exclusion owned and dated; evidence
of which priorities alert. *Pitfall:* trusting a vendor's green coverage map.
:::
## Measure, govern, pay
### Metrics and KPIs
Measure the library, not people; the formulas are the Atlas's proposals. Precision uses [SOC Ops](../soc-ops/) dispositions:
benign positives count against it like false positives, since both cost analyst time, but need a different fix (context or a scoped
filter); undetermined alerts are reported separately. Teams judging logic, not analyst load, may count benign positives as
correct: Magonia classes expected activity as a true positive with "No Business Impact", defines precision as "how often the rule
fires correctly" and adds a separate impact rate[^magonia]. Label each report's convention.
| KPI | Formula | Data source | Pitfall |
|---|---|---|---|
| Rule precision | TP ÷ (TP + benign positives + FP), per rule, 30 days | SOC dispositions | Undetermined or unlabelled alerts |
| Validated coverage | prioritised implementations detected in emulation within 90 days ÷ prioritised implementations | Emulation records | Counting tags, not runs |
| Test completeness | rules with ≥ 1 positive and ≥ 1 negative test ÷ production rules | Repository | Tests that match by construction |
| Rule health | rules whose runs, data and canary checks passed in 7 days ÷ production rules | Health jobs, rule-run logs | Checking that a query runs, not that it can match |
| Detection latency | median and p95 of (alert time − event time) per rule | Emulation runs, true positives | Ingestion delay hidden by event-time filters |
| Detection lead time | median days from accepted request to production | Backlog, Git | Splitting work to look fast |
| Change failure rate | deployments rolled back or hot-fixed within 7 days ÷ deployments | Git, change log | Silent fix-forwards |
| Filter debt | filters past expiry or without owner ÷ filters | Repository | Filters hidden in consoles |
Test completeness, rule health and filter debt **lead**; validated coverage **lags** but is closest to the truth; lead time and change
failure rate come from [DevOps](../devops/). The sponsor's 2026 summary reports 59% of teams track false-positive rates but only 14%
prioritise reducing them[^sans-2026].
**Worked calculation (illustrative numbers).** In September a team has 120 production rules.
- Rule A: 40 alerts, 6 true, 30 benign and 4 false positives. Precision = 6 ÷ (6 + 30 + 4) = **15%**: noisy-rule review.
- 84 rules have a positive and a negative test. Test completeness = 84 ÷ 120 = **70%**.
- Emulation in 90 days detected 18 of 30 prioritised implementations. Validated coverage = 18 ÷ 30 = **60%**; a tag heatmap shows
100%.
- 114 rules passed health checks. Rule health = 114 ÷ 120 = **95%**.
- 3 of 50 deployments were rolled back within a week. Change failure rate = 3 ÷ 50 = **6%**.
Set targets per rule class: a rule that pages on-call may need precision above 50%; a low-severity rule feeding a correlation can
tolerate less.
### Quality, security and compliance
- **The repository is sensitive.** It lists what you watch and, through filters and known gaps, what you do not. Palantir notes the
"operational security considerations around publicly acknowledging and documenting internal alerts"[^ads]; the Atlas adds: keep it
private, use code owners, and require review for filter changes.
- **Pipeline credentials** are scoped per environment ([Lab 3](#lab-3-advanced-a-ci-pipeline-with-staged-deployment)). ATT&CK's
Defense Impairment tactic covers attempts to "break security mechanisms, pipelines, and tooling"[^attack-tactics], yours included.
- **Test data.** Prefer synthetic or sanitised samples; real events carry user and host names ([PrivacyOps](../privacyops/)).
- **Auditability.** Git history shows who changed which detection and why. Report against NIST CSF 2.0 outcomes DE.CM and
DE.AE[^nist-csf]; CSF is voluntary unless mandated; sector monitoring duties belong to [ComplianceOps](../complianceops/).
### Cost, capacity and sustainability
**Telemetry:** Microsoft warns that Sysmon ProcessAccess events "can generate significant
amounts of logging"[^sysmon]; requests state their data needs, and [SecDataOps](../secdataops/) decides ingest. **Compute:** long
look-backs, correlations and real-time rules consume SIEM capacity. **People:** SANS's 2025 summary lists high false-positive rates in
vendor detection tools (64%) among the most-cited technology challenges, and 71% cite resource and time constraints as their primary
obstacle[^sans-2025]. A useful unit is **analyst minutes per true positive**: (alerts × average triage
minutes) ÷ true positives. Rule A's 40 alerts at 6 minutes for 6 true positives cost 40 minutes each. Retirement is the cheapest
capacity.
### Assurance of the operating model
- **Sampled review:** each month, check five random production rules for requirement, strategy, tests, owner and review date.
- **Deployed equals repository:** a nightly job reports drift between platforms and Git (console edits, disabled rules).
- **Gap register:** known-gap tests and expired filters appear in the quarterly coverage report.
- **Independent validation:** an annual exercise by a red team tests implementations the detection team did not choose.
## When things go wrong
### Common failures and troubleshooting
| Symptom | Likely cause | Diagnosis | Remediation and prevention |
|---|---|---|---|
| Rule silent, data present | Rule disabled, runs failing or skipped, deployed query drifted from Git | Rule-health log[^ms-health]; diff with Git | Redeploy from Git; alert on failed runs and drift |
| Rule stopped firing, no errors | Field renamed, parser or audit policy changed | Positive fixture against live raw events | Fix mapping; hunt the gap window |
| Late events never alert | Look-back equals the run interval; late-ingested events fall between runs | Event versus ingestion time per source | Widen the look-back by the delay; deduplicate the overlap[^ms-ingest] |
| Alert flood after a deploy | Logic error, missing filter, wrong index | Diff with the previous version | Roll back via the pipeline; enforce staging |
| Weeks of noise, analysts ignore it | No tuning loop or owner | Precision per rule from dispositions | Tune, downgrade to informational, or retire |
| CI breaks with no rule change | Unpinned toolchain | Package versions versus the last green run | Pin together; planned upgrades |
| Passes tests, misses in emulation | Fixtures unlike real telemetry, or an uncovered implementation | Capture the run's raw events | Add real regression samples[^sigma-regression] |
### Worked runbook: the rule that went silent
**Situation (illustrative).** Rule B (events 4698 and 4702) usually raises 2–5 alerts a week; the weekly review notices eleven silent days.
1. **Did the deployed rule run?** Check the rule-run log and diff the deployed query with Git. In Microsoft Sentinel, with health
monitoring on, the SentinelHealth table records for each run "if they succeeded or failed, and if they failed, why", and shows auto-disabled rules
and skipped windows[^ms-health]. Result: all runs succeeded; the query matches Git.
2. **Is the data arriving?** Count events 4698 and 4702 per day and host for 14 days. Result: it stopped on 80% of hosts on one day.
3. **What changed?** Check the change calendar ([SecOps](../secops/)). Result: a Group Policy change rebuilt the workstation audit
policy without "Audit Other Object Access Events", the subcategory that produces both events[^ms-4698][^ms-4702].
4. **Contain the blind spot.** Open a data-gap incident with [SecDataOps](../secdataops/) and the policy owner; meanwhile raise rule
A's priority, knowing it sees only the `schtasks.exe` path.
5. **Hunt the gap window.** Ask [ThreatOps](../threatops/) to hunt the eleven days with DET0441's other telemetry (Sysmon process and
registry events)[^det0441].
6. **Restore and verify through the deployed rule.** After the fix, repeat the Lab 4 validation on one lab host; the alert must come
from the production rule, not a fixture.
7. **Prevent recurrence.** Alert when a host sends Security events but no 4698 for 72 hours, and on failed or skipped runs; add the
audit subcategory to the rule's dependencies and to the audit-policy change checklist.
**Exit criteria.** Runs healthy; 4698 on all monitored hosts; validation alert from the production rule; gap hunt closed.
::: figure Runbook flow when a detection goes silent
```mermaid
flowchart TB
R["Silent rule: ran, and matches Git?"]
B["Raw events arriving on time?"]
D["Fixture matches live fields?"]
F["Canary fires through the deployed rule?"]
Q["Quiet is real: record it"]
RX["Fix the run or redeploy"]
C["Data gap or delay: SecDataOps"]
E["Fix mapping or rule"]
R -->|"yes"| B
B -->|"yes"| D
D -->|"yes"| F
F -->|"yes"| Q
R -->|"no"| RX
B -->|"no"| C
D -->|"no"| E
F -->|"no"| E
RX ~~~ C
C ~~~ E
classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029
classDef core fill:#e8f3f2,stroke:#16756f,color:#122029
classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029
class R,B,D,F decision
class Q core
class RX,C,E risk
```
Text equivalent: check in order that the deployed rule ran and matches Git (else fix or redeploy); that raw events arrive in time
(else a data gap or delay for SecDataOps); that the fixture matches live fields (else fix mapping or rule); and that a canary fires
through the deployed rule (else fix the rule). Only then is the quiet real.
:::
### Anti-patterns and myths
- **"We cover 80% of ATT&CK."** A tag count, not coverage; report implementations and validation evidence[^stp-mapping].
- **Console hot-fixes.** A night-time exclusion in the SIEM UI never reaches Git; the next deploy erases it.
- **Tests written from the rule.** Fixtures derived from the query pass by construction; derive them from the threat and real benign
activity.
- **Importing a pack wholesale.** SigmaHQ's 3,000+ rules[^sigmahq-readme] and vendor content are starting points to adapt and test
one by one; enabling everything buys noise and false confidence.
- **Indicator rules as detections.** Hash and IP matches sit on the two lowest levels of the Pyramid of Pain[^pyramid]; give them expiry dates and
behavioural replacements.
- **Myth: detection-as-code needs a big platform.** Labs 1–3 need Git, two scripts, a CI runner and one API.
## Deciding and adopting
### When to use and when not to use
**Adopt the full practice** when you write or change custom detections, more than one person edits them, the SOC reports noise or
misses, or nobody can answer "what do we detect?" with evidence; multi-platform and multi-tenant teams gain most. **Use a light
version** with mainly vendor or MDR content: custom rules and exclusions in Git, filter reviews with the provider, periodic emulation.
**Do not start here when:**
- **The data is missing.** Start with [SecDataOps](../secdataops/); SANS's 2025 summary says 47% of respondents report inadequate access
to the data feeds needed[^sans-2025].
- **Nobody works the alerts.** Better detections become better-ignored alerts; start with [SOC Ops](../soc-ops/).
- **Prevention basics are missing.** The same effort may reduce more risk in [VulnerabilityOps](../vulnerabilityops/) or
[IAMOps](../iamops/).
- **The platform has no content API.** Keep the repository as documentation and reconcile by export.
Hidden responsibilities: someone must own the toolchain (pins, upgrades, remapping), health checks and the emulation lab.
### Decision framework
::: figure Should you adopt detection-as-code, and how much of it?
```mermaid
flowchart TB
Q1["Custom detections?"]
Q2["Telemetry parsed?"]
Q3["Alerts triaged?"]
Q4["Several platforms?"]
L1["Light version"]
L2["Fix data first"]
L3["Fix triage first"]
L4["Native queries as code"]
L5["Sigma or threat model"]
Q1 -->|"no"| L1
Q1 -->|"yes"| Q2
Q2 -->|"no"| L2
Q2 -->|"yes"| Q3
Q3 -->|"no"| L3
Q3 -->|"yes"| Q4
Q4 -->|"no"| L4
Q4 -->|"yes"| L5
L1 ~~~ L2
L2 ~~~ L3
L3 ~~~ L4
classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029
classDef core fill:#e8f3f2,stroke:#16756f,color:#122029
classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029
class Q1,Q2,Q3,Q4 decision
class L1,L4,L5 core
class L2,L3 risk
```
Text equivalent: four questions in order. Do you write or change custom detections? If not, use the light version. Is telemetry
collected and parsed? If not, fix data first with SecDataOps. Is anyone triaging the alerts? If not, fix triage first with SOC Ops.
Several platforms or tenants? If not, keep native queries as code; if so, use Sigma or a threat-to-rule model.
:::
For tool choices, score candidates 1–5 against weights you can defend; an example for a single-SIEM team:
| Criterion | Weight | Why |
|---|---|---|
| Content API or repository connector | 25% | Otherwise deployment stays manual |
| Fidelity to the platform features you use | 20% | Lost features become blind spots |
| Testability: fixtures, replay, regression samples | 20% | Tests are the core of the practice |
| Toolchain stability and licence | 15% | See the pySigma and contentctl notes |
| Team skills | 10% | SANS 2026 (sponsor summary): 13% of detection engineers report high software-engineering proficiency[^sans-2026] |
| Portability | 10% | A SIEM migration means rewriting every native rule |
### Maturity model
This is **the Atlas's model**, not a standard. It parallels Elastic's five DEBMM tiers[^debmm] and the Detection Engineering Maturity
Matrix[^de-matrix], with exit criteria you can check in a repository.
| Level | Observable behaviour | Exit criteria (evidence) |
|---|---|---|
| 0 Ad hoc | Rules created and edited in consoles; no owner | — |
| 1 Versioned | Custom rules and filters in Git with owner, status and technique tags | Nightly drift report shows zero drift |
| 2 Tested | Positive and negative tests; CI lints, tests and converts; reviews required | Test completeness ≥ 90%; no merge without green CI |
| 3 Validated | Staging for every change; precision per rule; quarterly emulation | Validated coverage reported per implementation |
| 4 Managed portfolio | Threat-ranked backlog; health checks; expiring filters; routine retirement | Rule health ≥ 95%; filter debt near zero; coverage used in risk decisions |
Elastic warns that rushing tiers "without proper budget and staff can lead to burnout"[^debmm]. Reach level 3 for your top threats
before level 2 everywhere.
### Adoption roadmap: 30/60/90 days
| Window | Actions | Milestone and metric |
|---|---|---|
| Days 1–30 | Export custom rules and exclusions to a private repository; add owner, status, technique tags; Lab 1 linting in CI | Inventory complete; drift report running |
| Days 31–60 | Tests for the 20 noisiest or most severe rules; conversion and `sigma check --fail-on-issues` (or the native validator); record dispositions | Test completeness 100% for the top 20; their precision measured |
| Days 61–90 | Shadow mode for all changes; first emulation of five priority techniques; health checks for them; first coverage report | Validated coverage for five techniques; change failure rate tracked |
Quick wins: visible exclusions and evidence-based tuning. Dependencies: SecDataOps (data), SOC Ops (dispositions), ThreatOps
(priorities), platform owners (API tokens).
## Toolkit
### Checklists
**Pre-merge checklist (every pull request)**
- [ ] Request linked; threat, assets and data stated
- [ ] Strategy document updated, blind spots included
- [ ] At least one positive and one negative test; known gaps written as tests
- [ ] Technique IDs checked against the current ATT&CK release
- [ ] Filters have reason, owner and expiry
- [ ] CI green: lint, tests, `sigma check --fail-on-issues` or native validator, conversion
- [ ] Reviewer is not the author
**Promotion checklist (staging to production)**
- [ ] Staging period completed; volume within the estimate; matches sampled with an analyst
- [ ] Runbook linked
- [ ] Severity agreed with the SOC lead
- [ ] Health checks defined (rule runs, data presence, canary)
- [ ] Release tag created by a maintainer on `main`
**Monthly health review**
- [ ] Silent rules investigated (runs, data or logic)
- [ ] Expired filters renewed with evidence or removed
- [ ] Rules past their review date renewed or deprecated
- [ ] Deployed-versus-repository drift is zero
- [ ] Toolchain pins reviewed
**Quarterly governance review**
- [ ] Coverage by implementation, with evidence and gap register, reviewed with risk owners; mappings current
### Templates
**Detection request (intake form)**
_Example status: template._
```text
Request ID: DR-____
Requested by: (team, person) Date:
Source: intel report / hunt / incident / audit / SOC
Behaviour: (plain words + ATT&CK technique or sub-technique)
Implementations: (which ways of doing it must be detected)
Assets in scope: (systems, criticality)
Data needed: (log source, event IDs, fields; available? Y/N)
Evidence: (hunt query, incident timeline, report link)
Expected volume: (alerts per week, best guess)
Response: (what the SOC should do; runbook exists? Y/N)
Priority: (impact x likelihood, with reasons)
Decision: accepted / parked (data gap) / rejected By: Date:
```
**Filter record (one per exclusion)**
_Example status: template._
```text
Filter ID: FLT-____ Applies to rules: (IDs)
Reason: (alert samples, analyst evidence)
Logic: (fields and values excluded)
Blind spot: (what an attacker could hide behind this filter)
Known-gap test: (test file and case name)
Owner: (person) Expiry: YYYY-MM-DD Reviewed by:
```
**Retirement record**
_Example status: template._
```text
Rule ID / title: (id, title) Status -> deprecated
Reason: replaced / obsolete technique / data source retired /
cost exceeds value (precision, analyst minutes per TP)
Replacement: (rule IDs, in Sigma "related" with type)
Coverage impact: (implementations no longer detected, if any)
Approved by: (detection lead) SOC informed: (date)
```
### Deep comparisons
**Sigma, native queries, or a threat-to-rule model**
| Approach | Strengths | Limits | Choose when |
|---|---|---|---|
| Native queries as code (KQL, SPL, ES\|QL, YARA-L) | Full platform features; no conversion | Lock-in | One platform, deep feature use |
| Sigma as source of truth | Portable; community library; validators | Conversion gaps; pipelines; toolchain churn | Several platforms or tenants |
| Threat-to-rule model (e.g. OpenTide) | Threat → objective → rule traceability[^opentide-site] | Another schema and tool | Traceability is the main pain |
### FAQ
**Is DetectionOps the same as detection engineering?** In practice yes: the label names detection engineering run as a DevOps-style
loop[^hacklu25]. Hire under the standard names.
**Do we need Sigma?** No. It helps with several platforms or community content; one platform can keep native queries as code with
the same gates.
**How many tests does a rule need?** One that must fire and one that must not[^panther], a known-gap test per accepted blind spot, and
a real regression sample when you can[^sigma-regression].
**Our vendor claims 90% ATT&CK coverage. Is that coverage?** It is a mapping. Ask which implementations, on which assets, at what
precision, validated when[^stp-mapping].
**Should detection content be open source?** Generic rules often, yes; your filters, gaps and priorities, no, since they map your
blind spots[^ads].
## Going further
### Advanced topics and emerging direction
- **ATT&CK detection strategies (established since v18).** v20 lists 703 Enterprise detection strategies and 1,770
analytics[^attack-v20].
- **Coverage measurement (maturing).** Summiting the Pyramid 5.0 (September 2026) added a Detection Coverage Calculator[^stp-changelog];
Sigma has an `stp` tag namespace for robustness scores[^sigma-tags].
- **Correlation as code (uneven support).** Sigma correlation rules express counts, distinct values and temporal order[^sigma-corr];
test conversion per backend.
### Learning path
| Stage | Do this | Output |
|---|---|---|
| Prerequisites | Read [SOC Ops](../soc-ops/) and [SecDataOps](../secdataops/); learn Git and one query language | You know where alerts come from |
| One day | Read the Sigma specification[^sigma-spec] and the Pyramid of Pain[^pyramid]; do Lab 1 | A linted rule |
| One week | Lab 2; an ADS strategy for a rule you run; CTID's quality and mapping pages[^stp-quality][^stp-mapping] | A tested rule with a known gap |
| 30 days | Lab 3 on a test SIEM; precision for your five noisiest rules; plan Lab 4 | A pipeline and first metrics |
| Specialise | Emulation engineering, coverage analytics, cloud or identity detection ([CloudSecOps](../cloudsecops/), [IdentityOps](../identityops/)) | A portfolio repository with tests and a coverage report |
## Wrap-up
### Executive recap and cheat sheet
| Topic | Remember |
|---|---|
| Definition | Detection engineering run like software delivery; "DetectionOps" is a rare label for it |
| Unit of work | A detection with requirement, data dependency, code, tests, owner, review date |
| Practices | Strategy before query; test-first; review; staging; dispositions; health checks; emulation; retirement |
| Metrics | Precision per rule; validated coverage; test completeness; rule health; latency |
| Watch-outs | Tag-based coverage, console hot-fixes, unpinned toolchains, filters without expiry, silent rules |
### A judgement call: Sigma as the source of truth, or native queries as code?
**Situation (illustrative).** A 1,500-person company runs Microsoft Sentinel; two engineers maintain about 150 KQL analytics in the
portal. The CISO wants detection-as-code and has heard Sigma makes rules portable; a SIEM migration is "possible within three years".
**Option A: everything in Sigma, KQL generated.** Attractive: portability, community rules, validators, a hedge against migration.
Costs: every rule must survive conversion; KQL features the team relies on (watchlist joins, functions, enrichment) may not map;
analysts debug queries they did not write; and the toolchain is in flux (pinning caution above).
**Option B: KQL as the source of truth, as code.** Deploy the analytics rules through Sentinel's repository connection[^sentinel-cicd],
with the same tests, review, staging and metrics. Microsoft now recommends custom detections for new rules[^ms-ingest]: a dated
assumption (October 2026) to revisit, not a reason to rewrite 150 working analytics. Cost: lock-in; a migration means rewriting.
**Deciding assumptions.** Migration probability and timing; the share of rules using platform-specific features; whether you will
share or consume Sigma content; capacity to own a conversion toolchain.
**Reasoning.** With one platform, two engineers and an uncertain migration, the expensive part of any migration is re-validating
behaviour on new telemetry, not syntax. Requirements, tests and strategy documents survive it; generated queries help little if field
semantics change. So **Option B now, with cheap portability**: keep requirements, strategies and test cases platform-neutral, and
author in Sigma only simple single-event rules and community imports. Revisit if a migration is scheduled, a
second platform appears, or a partner requires Sigma. "Sigma everywhere" optimises for a migration that may never come and pays
daily in conversion gaps.
### Learning objectives and mastery check
After this page you can explain the label, write a Sigma rule with positive, negative and known-gap tests, design a staged pipeline
with scoped credentials, measure precision, coverage and rule health, and decide how much of the practice to adopt.
::: answer 1. Why is "share of ATT&CK techniques with a tagged rule" a weak coverage measure?
A tag says nothing about which implementations are detected, how robust or precise the rule is, or whether it fired in a recent
emulation. Use implementation coverage, detection quality and validation evidence instead.
:::
::: answer 2. In Lab 2 the filter raised precision from 0.88 to 1.00 and cut recall from 1.00 to 0.88. What should the team do?
Decide explicitly: keep the filter only if the updater's benign positives cost more than the risk of malware at that exact path, with
an owner, an expiry and the known-gap test. Better: narrow it with context an attacker cannot easily copy (such as the binary's
signer, if logged), or cover the gap with another rule.
:::
::: answer 3. A rule passed CI and staging but missed the behaviour in an emulation. Give two likely causes.
Fixtures unlike real telemetry (field names, encoding, mapping), or an implementation the rule never covered (PowerShell instead of
`schtasks.exe`). Turn the run's raw events into a regression sample.
:::
::: answer 4. Why does the Lab 3 validate job get no secrets, and what stops a writer reaching them anyway?
It runs unreviewed pull-request code. Tokens live only as environment secrets; staging accepts only `main`, production only
maintainers' `release-*` tags on `main`, approved by someone other than the author (plan permitting).
:::
::: answer 5. After an ATT&CK release your dashboard shows Defense Evasion coverage collapsing. Why, and what is the fix?
The release renamed or split the tactic (v19 did this), so hand-kept tactic tags no longer match. Detection did not change;
derive tactics from technique IDs.
:::
### Conclusion
"DetectionOps" is a small label for a large idea: treat each detection as a tested, owned and measured product rather than a saved
query. Tools matter less than that contract, above all tests for what you chose not to see. Keep it, and "what do we detect, and how
do we know?" has an answer better than a heatmap.
## Sources and further reading
Sources were opened on 2026-10-16 or 2026-10-17; survey figures come from summary pages, not full reports.
Ledger: `research/detectionops-ledger.md`.
[^opentide-gh]: [OpenTideHQ (GitHub organisation)](https://github.com/OpenTideHQ) — OpenTide project, live page. Checked 2026-10-16.
[^opentide-site]: [opentide · spec-driven development for detection engineering](https://opentide.org/) — OpenTide project (v0.15.0 banner). Checked 2026-10-16.
[^hacklu24]: [Unleashing the power of purple teaming with OpenTIDE](https://pretalx.com/hack-lu-2024/talk/UXQXUT/) — Hack.lu 2024 workshop, Remi Seguy and Amine Besson, 2024-10-23. Checked 2026-10-16.
[^hacklu25]: [When TI made actionable drives your Threat Detection (slides)](https://archive.hack.lu/media/hack-lu-2025/submissions/NMTJJE/resources/Hack.lu_2025__fJtiGVM.pdf) — Hack.lu 2025, Amine Besson, Remi Seguy, Claus Houmann, 2025-10-23. Checked 2026-10-16.
[^hacklu26]: [OpenTIDE Workshop – Build Your Own Operational DetectionOps Platform in 90 Minutes](https://pretalx.com/hack-lu-2026/talk/J9HKFX/) — Hack.lu 2026, scheduled 2026-10-23. Checked 2026-10-16.
[^ec-ospo]: [Taming the flood](https://interoperable-europe.ec.europa.eu/collection/ec-ospo/news/taming-flood) — European Commission Open Source Programme Office, 2025 (updated 2025-05-21). Checked 2026-10-16.
[^sans-2025]: [2025 SANS Detection Engineering Survey: Evolving Practices in Modern Security Operations](https://www.sans.org/white-papers/2025-sans-detection-engineering-survey-evolving-practices-modern-security-operations) — SANS Institute (summary page), 2025-02-24. Checked 2026-10-16.
[^sans-2026]: [The State of Detection Engineering Report 2026](https://www.anvilogic.com/report/state-of-detection-engineering) — Anvilogic, sponsor's summary of a SANS survey report, 2026. Checked 2026-10-16.
[^sans-poster]: [Detection Engineering (poster)](https://www.sans.org/posters/detection-engineering) — SANS Institute, Kathryn Hedley and Nick Mitropoulos, 2026-02-09. Checked 2026-10-16.
[^debmm]: [Elastic releases the Detection Engineering Behavior Maturity Model](https://www.elastic.co/security-labs/elastic-releases-debmm) — Elastic Security Labs, Mika Ayenson, Terrance DeJesus, Samir Bousseaden, 2024-09-06. Checked 2026-10-16.
[^de-matrix]: [Detection Engineering Maturity Matrix](https://detectionengineering.io/) — Kyle Bailey, 2021 (updated 2022-09). Checked 2026-10-16.
[^pyramid]: [The Pyramid of Pain](http://detect-respond.blogspot.com/2013/03/the-pyramid-of-pain.html) — David J. Bianco, Enterprise Detection & Response, 2013-03-01 (updated 2014-01-17). Checked 2026-10-16.
[^ads]: [Alerting and Detection Strategies Framework](https://github.com/palantir/alerting-detection-strategy-framework) — Palantir, 2017. Checked 2026-10-16.
[^nist-csf]: [The NIST Cybersecurity Framework (CSF) 2.0, NIST CSWP 29](https://nvlpubs.nist.gov/nistpubs/CSWP/NIST.CSWP.29.pdf) — NIST, 2024-02-26. Checked 2026-10-16.
[^sigma-spec]: [Sigma Rules Specification, version 2.1.0](https://github.com/SigmaHQ/sigma-specification/blob/main/specification/sigma-rules-specification.md) — SigmaHQ, 2025-08-02. Checked 2026-10-16.
[^sigma-filters]: [Sigma Filters Specification, version 2.1.0](https://github.com/SigmaHQ/sigma-specification/blob/main/specification/sigma-filters-specification.md) — SigmaHQ, 2025-08-02. Checked 2026-10-16.
[^sigma-corr]: [Sigma Correlation Rules Specification](https://github.com/SigmaHQ/sigma-specification/blob/main/specification/sigma-correlation-rules-specification.md) — SigmaHQ. Checked 2026-10-16.
[^sigma-tags]: [Sigma Tags Appendix](https://github.com/SigmaHQ/sigma-specification/blob/main/specification/sigma-appendix-tags.md) — SigmaHQ (history to v2.1.0, 2025-08-02). Checked 2026-10-16.
[^sigmahq-tag]: [SigmaHQ rule file carrying the attack.stealth tag](https://github.com/SigmaHQ/sigma/blob/master/rules/windows/process_creation/proc_creation_win_rundll32_process_dump_via_comsvcs.yml) — SigmaHQ rules repository (file modified 2025-02-23). Checked 2026-10-16.
[^sigma-about]: [About Sigma](https://sigmahq.io/docs/guide/about.html) — SigmaHQ documentation. Checked 2026-10-16.
[^sigmahq-readme]: [SigmaHQ/sigma: main rule repository](https://github.com/SigmaHQ/sigma) — SigmaHQ README. Checked 2026-10-16.
[^sigma-regression]: [Sigma Regression Testing](https://github.com/SigmaHQ/sigma/blob/master/regression_data/README.md) — SigmaHQ README. Checked 2026-10-16.
[^sigma-cli]: [sigma-cli README](https://github.com/SigmaHQ/sigma-cli) — SigmaHQ. Checked 2026-10-16.
[^sigma-check]: [sigma-cli source: check command](https://github.com/SigmaHQ/sigma-cli/blob/main/sigma/cli/check.py) — SigmaHQ, main branch. Checked 2026-10-16.
[^pypi]: [sigma-cli on PyPI](https://pypi.org/project/sigma-cli/) (3.1.0, 2026-07-07), [pySigma](https://pypi.org/project/pySigma/) (2.0.1, 2026-10-11), [pySigma-backend-splunk](https://pypi.org/project/pySigma-backend-splunk/) (2.1.0), [pySigma-pipeline-windows](https://pypi.org/project/pySigma-pipeline-windows/) (2.0.1) — Python Package Index. Checked 2026-10-17.
[^sigma-mod]: [Sigma Modifiers Appendix](https://github.com/SigmaHQ/sigma-specification/blob/main/specification/sigma-appendix-modifiers.md) — SigmaHQ (`windash`). Checked 2026-10-17.
[^sigmahq-sched]: SigmaHQ rules [Schedule Task Creation From Env Variable Or Potentially Suspicious Path Via Schtasks.EXE](https://github.com/SigmaHQ/sigma/blob/master/rules/windows/process_creation/proc_creation_win_schtasks_env_folder.yml) (modified 2025-10-07) and [Suspicious Scheduled Task Creation](https://github.com/SigmaHQ/sigma/blob/master/rules/windows/builtin/security/win_security_susp_scheduled_task_creation.yml) — SigmaHQ rules repository. Checked 2026-10-17.
[^pipeline-windows]: [pySigma-pipeline-windows](https://github.com/SigmaHQ/pySigma-pipeline-windows) — SigmaHQ, source on the main branch (pipeline `windows-logsources`). Checked 2026-10-17.
[^attack-v20]: [ATT&CK Updates – October 2026 (v20)](https://attack.mitre.org/resources/updates/) — MITRE, 2026-10-13. Checked 2026-10-16.
[^attack-v19]: [ATT&CK Updates – April 2026 (v19)](https://attack.mitre.org/resources/updates/updates-april-2026/) — MITRE, 2026-04-28. Checked 2026-10-16.
[^attack-v18]: [ATT&CK Updates – October 2025 (v18)](https://attack.mitre.org/resources/updates/updates-october-2025/) — MITRE, 2025-10-28. Checked 2026-10-16.
[^attack-objects]: [ATT&CK Detection Strategies](https://attack.mitre.org/detectionstrategies/), [Analytics](https://attack.mitre.org/analytics/) and [Data Components](https://attack.mitre.org/datacomponents/) — MITRE. Checked 2026-10-16.
[^attack-tactics]: [ATT&CK Enterprise Tactics](https://attack.mitre.org/tactics/enterprise/) — MITRE (v20). Checked 2026-10-16.
[^attack-t1053]: [Scheduled Task, T1053.005](https://attack.mitre.org/techniques/T1053/005/) — MITRE ATT&CK, technique version 1.8. Checked 2026-10-16.
[^det0441]: [DET0441: Detection of Suspicious Scheduled Task Creation and Execution on Windows](https://attack.mitre.org/detectionstrategies/DET0441/) — MITRE ATT&CK, modified 2026-05-12. Checked 2026-10-16.
[^stp-defs]: [Summiting the Pyramid: Definitions](https://center-for-threat-informed-defense.github.io/summiting-the-pyramid/overview/definitions/) — MITRE Center for Threat-Informed Defense. Checked 2026-10-16.
[^stp-quality]: [Summiting the Pyramid: Detection Quality](https://center-for-threat-informed-defense.github.io/summiting-the-pyramid/detection-ttv/detectioncoverage/detectionquality/) — MITRE CTID. Checked 2026-10-16.
[^stp-impl]: [Summiting the Pyramid: Implementation Coverage](https://center-for-threat-informed-defense.github.io/summiting-the-pyramid/detection-ttv/detectioncoverage/implementationcoverage/) — MITRE CTID. Checked 2026-10-16.
[^stp-mapping]: [Summiting the Pyramid: Mapping Detections to ATT&CK](https://center-for-threat-informed-defense.github.io/summiting-the-pyramid/analytic-design/mappingdetections/) — MITRE CTID. Checked 2026-10-16.
[^stp-workflow]: [Summiting the Pyramid: Detection Engineering Workflow](https://center-for-threat-informed-defense.github.io/summiting-the-pyramid/analytic-design/deworkflow/) — MITRE CTID. Checked 2026-10-16.
[^stp-changelog]: [Summiting the Pyramid: Changelog](https://center-for-threat-informed-defense.github.io/summiting-the-pyramid/changelog/) — MITRE CTID (version 5.0, 2026-09-10). Checked 2026-10-16.
[^art]: [Atomic Red Team](https://github.com/redcanaryco/atomic-red-team) — Red Canary README. Checked 2026-10-16.
[^art-site]: [Atomic Red Team website](https://atomicredteam.io/) — Red Canary. Checked 2026-10-16.
[^art-exec]: [Execute Atomic Tests (Local)](https://github.com/redcanaryco/invoke-atomicredteam/wiki/Execute-Atomic-Tests-(Local)) — Invoke-AtomicRedTeam wiki, edited 2023-09-18. Checked 2026-10-16.
[^panther]: [Testing (detections)](https://docs.panther.com/detections/testing) — Panther documentation. Checked 2026-10-16.
[^elastic-dr]: [elastic/detection-rules](https://github.com/elastic/detection-rules) — Elastic README. Checked 2026-10-16.
[^elastic-dac]: [DaC Reference: Managing Elastic Security Detection Rules Using DaC](https://dac-reference.readthedocs.io/en/latest/) — Elastic (version 0.4.0). Checked 2026-10-17.
[^contentctl]: [splunk/contentctl](https://github.com/splunk/contentctl) — Splunk Threat Research Team README. Checked 2026-10-16.
[^sentinel-cicd]: [Deploy content as code from your repository](https://learn.microsoft.com/en-us/azure/sentinel/ci-cd) — Microsoft Learn, updated 2026-07-01. Checked 2026-10-17.
[^ms-ingest]: [Handle ingestion delay in Microsoft Sentinel](https://learn.microsoft.com/en-us/azure/sentinel/ingestion-delay) — Microsoft Learn, updated 2026-07-02. Checked 2026-10-17.
[^ms-health]: [Monitor the health and audit the integrity of your Microsoft Sentinel analytics rules](https://learn.microsoft.com/en-us/azure/sentinel/monitor-analytics-rule-integrity) — Microsoft Learn, updated 2026-07-02. Checked 2026-10-17.
[^magonia]: [What Framing Security Alerts as a Binary True or False Positive is Costing You](https://www.magonia.io/research/what-framing-security-alerts-as-a-binary-true-or-false-positive-is-costing-you/) — Magonia Research, signalblur, 2025-08-07. Checked 2026-10-17.
[^ms-close]: [Investigate incidents with Microsoft Sentinel (legacy)](https://learn.microsoft.com/en-us/azure/sentinel/investigate-cases) — Microsoft Learn, updated 2026-07-02 (closing classifications). Checked 2026-10-17.
[^yaral]: [Single and multiple event rules in YARA-L](https://docs.cloud.google.com/chronicle/docs/yara-l/yara-l-2-0-examples) — Google Security Operations documentation, updated 2026-10-13. Checked 2026-10-17.
[^ms-4698]: [4698(S): A scheduled task was created](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-10/security/threat-protection/auditing/event-4698) — Microsoft Learn, 2021-09-07. Checked 2026-10-16.
[^ms-4702]: [4702(S): A scheduled task was updated](https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-10/security/threat-protection/auditing/event-4702) — Microsoft Learn, 2021-09-07 (field `TaskContentNew`). Checked 2026-10-17.
[^sysmon]: [Sysmon v15.22](https://learn.microsoft.com/en-us/sysinternals/downloads/sysmon) — Microsoft Sysinternals, updated 2026-09-10. Checked 2026-10-16.
[^gh-envs]: [Deployments and environments](https://docs.github.com/en/actions/reference/workflows-and-actions/deployments-and-environments) — GitHub Docs. Checked 2026-10-17.
[^gh-manage-envs]: [Managing environments for deployment](https://docs.github.com/en/actions/how-tos/deploy/configure-and-manage-deployments/manage-environments) — GitHub Docs. Checked 2026-10-17.
[^gh-rulesets]: [About rulesets](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/about-rulesets) and [Available rules for rulesets](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/available-rules-for-rulesets) — GitHub Docs. Checked 2026-10-17.
[^ghosttask]: [GhostTask README](https://github.com/netero1010/GhostTask) — netero1010, GitHub (last push 2025-01-02). Checked 2026-10-17.
[^actions]: [actions/checkout](https://github.com/actions/checkout), [actions/setup-python](https://github.com/actions/setup-python), [actions/upload-artifact](https://github.com/actions/upload-artifact) and [actions/download-artifact](https://github.com/actions/download-artifact) — GitHub, live READMEs (v7, v7, v7, v8). Checked 2026-10-16.
**Changelog**
- 2026-10-16: first version (labs 1–2 executed, lab 3 syntax-checked); term status overlapping synonym: the label comes from one
project, and the practice has better-known names.
- 2026-10-17: revised after independent review: Lab 3 token reach and plan limits; Lab 2 evasion cases and an anchored filter
(re-run); SigmaHQ starting points; Microsoft rule types; SOC Ops' dispositions; rule-run and ingestion-delay checks; latency KPI;
registry-only tasks as a known gap.
- 2026-10-17, second review pass: rule B extended to event 4702 (`TaskContentNew`) with fixtures and Lab 4 implementation 6;
`\ProgramData\` and `%TMP%` in both rules, path lists declared a teaching subset; Labs 1–2 re-run; workflow `paths` covers itself;
about 110 words of restatement cut.
- 2026-10-17, post-score polish: benign-positive precision convention (Magonia); lab output ≤ 90 characters (Labs 1–2 re-run);
tooling signpost; footnotes re-checked; CSF wording; restatements cut.