--- title: "ResearchDataOps" slug: researchdataops category: data term_status: ambiguous evidence_status: verified summary: "ResearchDataOps is this Atlas's name for running research data management as an operational service: planning, describing, identifying, depositing, protecting and preserving research data so that others can find, cite and reuse it for decades." audience: [Research data stewards and librarians, Principal investigators, Research software engineers, Research computing staff, Research office and compliance leads, Beginners] last_verified: 2026-10-18 key_takeaways: - "No published definition of \"ResearchDataOps\" was found. The established field is research data management (RDM) and data stewardship; this page uses the label for the day-to-day operations that keep RDM working." - "The goal is longevity and reuse, not freshness: a stranger must be able to find, understand, cite and reuse the data years later without asking the people who made it." - "FAIR is not the same as open. The FAIR principles ask for persistent identifiers, rich metadata, licences and provenance; access may still require authentication and approval." - "Leaving data with its authors fails: in one study the odds of a dataset still existing fell by 17% per year; in another, 93% of authors who promised data on request did not respond or declined." - "Funder rules are becoming structured and checkable: NIH's new plan format (active awards that generate scientific data move to it from FY2027) and NSF's Research.gov form; Horizon Europe has long required a living plan by month 6. Check each funder's current rules." - "A DOI promises a landing page and open metadata, not usable files. Repositories keep bits; understandability (formats, README, codebook) stays your job." - "Stewardship lasts decades but is paid with project money: NIH allows preservation fees only if they are spent during the award, and EPSRC's 10-year retention clock restarts when someone requests the data." related: [dataops, bioops, labops, researchops, privacyops, datasecops, backupops, knowledgeops] --- ## What ResearchDataOps is ### In 30 seconds **ResearchDataOps** is not an established term. This Atlas uses it for **the operational side of research data management (RDM)**: the recurring work that turns data-sharing rules into datasets others can find, understand, cite and reuse long after the project, the grant and the PhD student have gone: plans, metadata and identifiers, deposit and versions, controlled access, preservation, and the final retention or disposal decision. **In one line: DataOps asks "is the data correct and on time?"; ResearchDataOps asks "can someone else still find, trust and reuse it in 2046?"** The reference test is the archival idea of data that is *independently understandable*: understandable by its intended users "without having to resort to special resources not widely available, including named individuals".[^oais] **For** data stewards and librarians, principal investigators (PIs), research software engineers, research computing staff and research office or compliance leads. No prerequisites; [DataOps](../dataops/) helps. ### At a glance | Item | Summary | |---|---| | Term status | **Ambiguous.** No definition of "ResearchDataOps" was found (searches on 2026-10-17); "research data operations" also names a clinical-study data team (below). The practice, RDM, is established. | | Goals | FAIR data that meets funder, journal and ethics conditions and is preserved as long as promised. | | In scope | Data management plans (DMPs); metadata and DOIs; deposit and versions; licences; controlled access; links to code; storage tiers and transfers; preservation; retention; reuse metrics. | | Out of scope | Pipelines → [DataOps](../dataops/); instruments, LIMS → [LabOps](../labops/); compute → [BioOps](../bioops/); workflow engines → [WorkflowOps](../workflowops/); storage systems → [StorageOps](../storageops/); privacy law → [PrivacyOps](../privacyops/); access enforcement → [DataSecOps](../datasecops/); UX research → [ResearchOps](../researchops/). | | Key outputs | A living DMP; a checked deposit package; a findable DOI; a data availability statement; access and retention decisions. | ### The problem it solves Research data has a short natural half-life when it stays with its creators. Vines and colleagues requested data from 516 articles published 2 to 22 years earlier. Where authors reported the data's status, "the odds of a data set being extant fell by 17% per year", and they concluded that "research data cannot be reliably preserved by individual researchers".[^vines2014] A promise is not a process either: of 1,792 BioMed Central articles whose data availability statement said the authors would share, "1669 (93%) authors either did not respond or declined to share their data with us", and "only 123 (6.8%) provided the requested data".[^gabelica2022] | Without operational RDM | With it | |---|---| | Data on a laptop or a departed student's account | Deposited in a repository with a DOI and fixity checks | | "Available on reasonable request" | A landing page that works without the author | | Spreadsheet with unexplained columns | README, codebook, open formats, licence | | Consent forms nobody can map to a request | Access conditions recorded and machine-readable | The costs: unverifiable findings, funder non-compliance (an NIH plan becomes a term of the award[^nih2021]), repeated data collection, and data kept forever because nobody dares delete it. ### What it is not - **Not DataOps renamed.** DataOps optimises flow and freshness; this page optimises persistence, citation and reuse. - **Not "make everything open".** FAIR principle A1.2 says the access protocol "allows for an authentication and authorization procedure, where necessary";[^fair2016] the GO FAIR Foundation states that following FAIR "is not equal to making all data 'open'".[^gofaira12] - **Not [ResearchOps](../researchops/)**, which runs user-experience research (participants, insight repositories). - **Not clinical data management**, although the name collides: at the Population Health Research Institute (PHRI, Canada), "Research Data Operations" brings together "database development, clinical data operations, and data standards", including randomisation systems for its studies.[^phri] - **Not a standard, certification or job title.** Data stewards, data librarians and research computing staff do this work under those names. ## Where it comes from and where it sits ### Origins and evolution | Year | Milestone | What it changed | |---|---|---| | 2002 | OAIS reference model first issued by CCSDS (January); third issue December 2024, also published as ISO 14721:2025[^oais][^oaismsi] | A shared vocabulary for archives | | 2011 | NSF will "not permit submission of a proposal that is missing a Data Management Plan" (a supplementary document of up to two pages), from 18 January[^nsf2011] | Plans become a condition of applying | | 2014 | Joint Declaration of Data Citation Principles (FORCE11): "Data should be considered legitimate, citable products of research"[^jddcp] | Data citation as a scholarly norm | | 2016 | FAIR Guiding Principles published in *Scientific Data*[^fair2016] | A shared vocabulary for "reusable" | | 2022 | US OSTP memo: publications "and their supporting data" accessible "without an embargo"; agency policies due by 31 December 2025 (how each agency complied is not assessed here)[^ostp2022] | Sharing at publication as US policy direction | | 2023 | NIH Data Management and Sharing (DMS) Policy effective 25 January[^nih2021] | The plan becomes a term of the award | | 2026 | NIH's structured plan format (applications due from 25 May; all active awards move to it from FY2027); NSF's Research.gov plan form (27 April); DataCite Metadata Schema 4.7 (3 March)[^nih2026][^nih2026b][^nsf2026][^dcschema] | Plans become forms that can be checked | ### Glossary and mental model | Term | Meaning on this page | |---|---| | **DMP / DMSP** | Data management (and sharing) plan, written at proposal time and kept current. | | **Scientific data (NIH)** | "The recorded factual material commonly accepted in the scientific community as of sufficient quality to validate and replicate research findings"; excludes lab notebooks, drafts and physical specimens.[^nih2021] | | **PID, DOI** | Persistent identifier; a DOI is the usual PID for a dataset, registered with DataCite by repositories such as Zenodo.[^zdev] | | **ORCID iD, ROR ID** | ORCID: "a unique, persistent identifier free of charge to researchers".[^orcid] ROR, the Research Organization Registry, identifies research organisations.[^ror] | | **Version DOI / concept DOI** | A DOI for one version; a DOI that represents all versions (Zenodo's terms).[^zver] | | **Fixity** | Evidence that content "has not been altered in an undocumented manner" (OAIS).[^oais] | | **Designated Community** | OAIS: the potential users who should be able to understand the archived information; the archive defines it.[^oais] | | **Tombstone** | The page a DOI still resolves to after withdrawal: citation, identifier and why the item is unavailable.[^dctomb] | | **DAC, DAS** | Data access committee (approves controlled-access requests); a paper's data availability statement. | **Mental model: the project ends, the promise does not.** Three spans rarely line up: the **project span** (grant years, when money and people exist), the **promise span** (what the plan, journal and consent promised: share at publication, retain for 10 years) and the **reuse window** (when someone needs the data, often after the project). ResearchDataOps keeps the promise after the project has gone. ### How ResearchDataOps relates to other Ops The boundaries below are an **editorial distinction** of this Atlas, not an industry standard. ResearchDataOps owns the sharing and retention promise from the proposal onwards and takes custody of the data at curation and publication. ::: figure The research data path and the Atlas pages that own each stage ```mermaid flowchart TB LAB["LabOps: samples, instruments"]:::neutral DOPS["DataOps: tested pipelines"]:::neutral BIO["BioOps: compute pipelines"]:::neutral RDO["ResearchDataOps: plan, describe, deposit, preserve"]:::core RE["Reuse: citation, replication"]:::core PRIV["PrivacyOps: legal limits"]:::decision DSEC["DataSecOps: access controls"]:::decision ROPS["ResearchOps: UX research, not on this path"]:::risk FLD["Field, survey, simulation data"]:::neutral LAB -->|"data leaves the instrument"| BIO BIO -->|"derived data"| RDO DOPS -->|"tested outputs"| RDO FLD -->|"direct"| RDO RDO -->|"DOI, licence, access route"| RE RDO -.-|"bounded by"| PRIV RDO -.-|"enforced by"| DSEC RE ~~~ ROPS classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029 classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029 classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029 ``` Text equivalent: Instrument data (LabOps) passes through compute pipelines (BioOps); tested pipeline outputs (DataOps) and field, survey and simulation data arrive directly. ResearchDataOps plans, describes, deposits and preserves them and hands reusers a DOI, licence and access route. Dotted links: PrivacyOps sets legal limits and DataSecOps enforces access. ResearchOps (user-experience research) is not on this path despite the similar name. ::: | Neighbour | Shared scope | Its focus | This page's focus | |---|---|---|---| | [DataOps](../dataops/) | Data quality, versioning | Flow, tests, freshness | Persistence, citation, reuse | | [BioOps](../bioops/) | Provenance, reproducible workflows | Running pipelines | Publishing their outputs | | [LabOps](../labops/) | Data integrity, metadata at source | Sample to record | Record to repository | | [PrivacyOps](../privacyops/) | Consent, personal data | Lawful processing, breaches | Access routes that honour consent | | [DataSecOps](../datasecops/) | Access control, audit | Policy enforcement in platforms | Who may reuse, under which terms | | [BackupOps](../backupops/) | Copies, restores | Recovering systems | Preserving meaning for decades | | [KnowledgeOps](../knowledgeops/) | Findability, metadata | Organisational knowledge | Citable research datasets | An adjacent framing, **SciOps**, applies DevOps to scientific operations ([maturity model](#maturity-model)).[^sciops] ## Principles and operating model ### A proposed charter and principles There is no official manifesto; this charter is **this Atlas's proposal**. **Mission:** every dataset the organisation promises to share can be found, understood without its authors, accessed on the promised conditions, and kept as long as promised. 1. **Write for the reuser you will never meet.** If understanding needs a named person, the dataset is not finished.[^oais] 2. **Metadata outlives data.** FAIR principle A2: metadata stays accessible "even when the data are no longer available"; the data citation principles say identifiers and metadata "should persist — even beyond the lifespan of the data they describe".[^fair2016][^jddcp] 3. **As open as possible, as closed as necessary** (Horizon Europe's phrase):[^hepg] every restriction has a written reason and an access route. 4. **Identify what will be cited:** dataset versions, people and organisations (DOI, ORCID, ROR).[^dccreator] 5. **Version, never overwrite.** Published files are immutable; a correction is a new version, and a removal leaves a tombstone. 6. **Prove integrity at every handoff** with checksums.[^bagit] 7. **Fund the tail.** A retention promise needs an owner and money after the grant ends. **Non-goals:** a perfect automated FAIR score; keeping everything forever; releasing what consent does not cover. ### Building blocks Eight blocks recur, each with its artefacts: **plan** (a DMP, machine-actionable where possible[^madmp]); **describe** (README, codebook, DataCite metadata, a community standard such as BIDS[^bids2016]); **identify** (version and concept DOIs, ORCID iDs, ROR IDs); **package** (a BagIt bag with SHA-512 manifests, RO-Crate metadata[^bagit][^rocrate]); **deposit** (a repository record); **control access** (conditions, a committee, Data Use Ontology tags[^duo]); **preserve** (fixity checks, open formats, a certified repository[^cts]); and **retire** (retention review, disposal record, tombstone[^dctomb]). ### Roles, ownership and RACI RDMkit calls data stewardship "a relatively new profession and a catch-all term for numerous support functions, roles and activities"; stewards may be **embedded** in a group or **central** (for example in a library), and policy-, research- or infrastructure-oriented.[^rdmkitds] The Campus Research Computing Consortium (CaRCC) describes **data-facing** roles covering "research data curation, storage, backup, preservation, and transfer; and research data policy compliance", and stresses that facings "are roles and not persons": in a small institution one person covers several.[^carcc] **RACI** (R responsible, A accountable, C consulted, I informed; DPO: data protection officer). An illustrative split; institutions differ. | Activity | PI | Researcher | Data steward | Repository or library | Ethics / DPO | Research office | |---|---|---|---|---|---|---| | Write and update the DMP | A | R | C | C | C | I | | Document data (README, codebook) | A | R | C | I | — | — | | Decide the access level | A | C | R | C | C | I | | Pre-deposit checks | I | R | A | C | — | — | | Curate, register DOI, publish | I | C | C | R/A | — | I | | Decide access requests (restricted data) | C | — | R | I | C | I | | Fixity and preservation | — | — | I | R/A | — | — | | Retention review and disposal | A | C | R | R | C | I | | Funder reporting on sharing | A | C | C | I | — | R | ### Lifecycle RDMkit organises research data management into planning, collecting, processing, analysing, preserving, sharing and reusing.[^rdmkitlc] The operational view groups these into phases with exit criteria: | Phase | Activities | Exit criteria | |---|---|---| | **Day 0: plan** | DMP with the funder's template; consent wording that allows the intended sharing; repository and cost estimate | Plan accepted; consent forms approved; preservation costs budgeted | | **Day 1: set up** | Storage tier per promise; folder and file naming; metadata templates; code repository; access groups | First data lands in the right place with a README | | **Day 2: share** | Pre-deposit checks; deposit; DOI in the paper's DAS; access requests; corrections as versions | DOI resolves; DAS matches the record; requests answered | | **Tail: steward** | Fixity audits; format checks; retention reviews; disposal or transfer | Review date set or disposal recorded | ### Operating model and governance - **Intake:** one deposit request per dataset ([template](#templates)), linked to the DMP and ethics approval. - **Gates:** automated [checks](#reference-implementation), then human review of sensitivity and licence. - **Change control:** published files are never edited; corrections create a version with a change note. - **Escalation:** steward → access or ethics committee → DPO for a possible personal data breach ([runbook](#worked-runbook-sensitive-data-published-by-mistake)). - **Cadence:** monthly queue review; annual review of every dataset whose retention or embargo date falls in the next year. ::: decision Governance proportionality A single lab needs a current DMP, a README template, a repository account and calendar entries for embargo and retention dates. A university needs the intake form, a data access committee, service levels and an owner for the retention register. Add ceremony only where an unowned promise would cost more than the ceremony. ::: ## Architecture and workflow ### Reference architecture The logical architecture has five zones. Record-level personal data stays inside the dashed **trust boundary**. Everything outside it is public or will become public, including embargoed files, which open automatically when the embargo ends.[^zpol] ::: figure Reference architecture: from active storage to long-term reuse ```mermaid flowchart TB subgraph Inside["Trust boundary: record-level data stays inside"] subgraph Active["1. Active research zone"] EL["Notebooks, pipelines, instruments"] ST["Project storage with backups"] end subgraph Curate["2. Curation zone"] PK["Package: README, metadata, checksums"] CK["Pre-deposit checks and human review"] end CTRL["3b. Controlled access: DAC, secure environment"] end subgraph Public["Public side"] OPEN["3a. Open or embargoed record"] DOI["4. DOI, landing page, open metadata"] IDS["4. ORCID, ROR, related identifiers"] FIND["5. Indexes, citations, usage metrics"] end EL --> ST ST --> PK PK --> CK CK -->|"restricted"| CTRL CK -->|"open or embargoed"| OPEN CTRL ~~~ OPEN OPEN --> DOI CTRL -->|"metadata only"| DOI DOI -.-|"linked to"| IDS DOI --> FIND classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029 classDef neutral fill:#f1f4f6,stroke:#5b7a87,color:#122029 class ST,EL neutral class PK,CK,OPEN,DOI,IDS,FIND core class CTRL risk style Inside stroke:#b4332b,stroke-width:2px,stroke-dasharray:6 4 ``` Text equivalent: Read top down. A dashed trust boundary at the top holds all record-level data: project storage fed by instruments, notebooks and pipelines (zone 1); packaging, then checks and human review (zone 2); and controlled access through a data access committee and secure environment (zone 3b, red). Below it, the public side holds open or embargoed records (zone 3a). Both routes get a DOI with a public landing page and open metadata (zone 4), linked to ORCID, ROR and related identifiers; the controlled route releases only metadata. Indexes, citations and usage metrics follow (zone 5). ::: Design rules: landing pages and metadata are public even when files are not, so keep sensitive details out of the metadata (DataCite says so explicitly);[^dcsens] the curation zone writes the checksums later zones rely on; and the controlled route records who received what, on which conditions. ### End-to-end workflow ::: figure Deposit workflow with exception paths ```mermaid flowchart TB T["Trigger: paper accepted or award ends"] --> R["Deposit request linked to DMP and ethics approval"] R --> P["Package and run pre-deposit checks"] P --> Q{"Blocking findings?"} Q -- "Yes" --> F["Fix and re-package"] F --> P Q -- "No" --> S{"Personal or restricted data?"} S -- "Yes" --> C["Controlled-access route and access conditions"] S -- "No" --> O["Open or embargoed record with licence"] C --> D["Publish: DOI resolves, metadata open"] O --> D D --> A["DOI goes into the data availability statement"] A --> M["Annual review: fixity, requests, retention date"] M -->|"error found"| V["New version with change note"] V --> D classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029 classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029 class T,R,P,O,D,A,M core class Q,S decision class F,C,V risk ``` Text equivalent: An accepted paper or the end of an award triggers a deposit request linked to the DMP and ethics approval. The dataset is packaged and checked; blocking findings send it back. Personal or restricted data takes a controlled-access route; other data gets an open or embargoed record with a licence. On publication the DOI resolves with open metadata and goes into the data availability statement. An annual review checks fixity, requests and the retention date; a later error produces a new version with a change note, never an edited file. ::: **Timing rules differ by funder** (as of October 2026): NIH says data should be shared "no later than the time of an associated publication, or the end of the award/support period, whichever comes first";[^nih2021] Horizon Europe asks for deposit "as soon as possible after data production and at the latest by the end of the project", and at publication for data underpinning a publication.[^hepg] ## Practices, standards and tools ### Core practices and playbooks | Practice | Trigger | Owner | Output | Verification | |---|---|---|---|---| | Living DMP | Proposal; scope, repository or consent change | PI | Current plan | Plan date later than the last material change | | README and codebook | First data file | Researcher | `README.md`, variable list | A colleague outside the project can explain each column | | Open, documented formats | Before deposit | Researcher | CSV, NetCDF, TIFF or domain formats beside proprietary originals | Opens with free software | | Package with checksums | Before any transfer | Data steward | Bag with SHA-512 manifest | Recomputed digests match | | Licence choice | Before an open release | PI with steward | Licence in metadata and README | Licence field present | | Access decision | Personal, commercial or third-party data | Steward with ethics/DPO | Written decision and conditions | Conditions on the landing page | | Correction | Error found after publication | PI | New version with change note | Old version still resolvable | | Retention review | Date reached | Steward | Keep, transfer or dispose decision | Decision recorded | **Choosing a data licence.** Open data usually gets CC0 or CC BY 4.0. The 4.0 licences also cover sui generis database rights, and Creative Commons advises against NonCommercial or NoDerivatives licences for databases "intended for scholarly or scientific use".[^ccfaq] CC0 avoids attribution stacking, where a derivative must credit every contributor to every upstream work; CC BY 4.0 eases it by accepting a link to an attribution page.[^dcclicence] Under CC0, credit rests on citation norms, as at Dryad.[^dryadreuse] #### Playbook: preparing a dataset for deposit Confirm what was promised; freeze the final files in a curation area (never curate in place); document; screen for personal and third-party data; package and run the [checks](#reference-implementation); choose the repository with the [decision framework](#decision-framework); deposit; put the version DOI in the DAS and the dates in the register ([checklist](#pre-deposit)). **Acceptance criteria:** the DOI resolves to a page with a licence or access conditions; the files verify against the manifest; an outsider can answer three questions about the data from the README alone. **Rollback:** before publication, discard the draft; after it, publish a corrected version or request withdrawal ([runbook](#worked-runbook-sensitive-data-published-by-mistake)). ### Reproducibility, active storage and transfer [BioOps](../bioops/) and [WorkflowOps](../workflowops/) own pipelines and engines; this section covers what must outlive them. **Link data to code and environment.** A dataset behind a figure is reusable only with the code that made the figure. - Deposit the analysis code as its own versioned record, linked both ways to the data (DataCite `relatedIdentifiers`). A `CITATION.cff` file holds "human- and machine-readable citation information for software (and datasets)", and Zenodo fills the record from it when a GitHub release is archived through its GitHub integration.[^cff] - Name the exact code: SWHIDs are "persistent, intrinsic identifiers for software source code artifacts" such as files, trees and commits, adopted as ISO/IEC 18670:2025; DataCite 4.7 accepts them as related identifiers.[^swhid][^dcschema] - Record the environment with the code: a lock file with exact versions or a container image digest, not "Python 3". - For workflow runs, Workflow Run RO-Crate profiles capture the provenance of an execution and bundle its "inputs, outputs, code, etc.".[^wrroc] **Active storage is not the retention promise.** NERSC, for example, tells users to "assume that any file not accessed within 8 weeks may be purged at any time" from its scratch file system (as of October 2026).[^nersc] | Tier | Built for | Risk to the promise | What it may hold | |---|---|---|---| | HPC scratch | Fast I/O during jobs | Purged by policy | Nothing that was promised | | Project storage | Active work, group sharing | Ends with the allocation or project | Working copies | | Archive or tape | Cheap bulk copies | Slow recall; unregistered copies get lost | A retention copy, if registered | | Repository | Publication and citation | Terms and limits change | The cited version | Day 1 rules: nothing promised lives only on scratch; the register records where each master copy is; and the [end-of-project checklist](#leaver-or-end-of-project) moves or disposes of data before the allocation ends. [StorageOps](../storageops/) and [BackupOps](../backupops/) own the systems. **Plan large transfers.** Estimate before promising a date: 12 TB is 96,000 gigabits, about 27 hours at a sustained 1 Gbit/s and longer in practice (illustrative arithmetic). For tens of terabytes, use data transfer nodes, servers "purpose-built and dedicated to the function of wide area data transfer" (ESnet),[^esnetdtn] and a service that verifies checksums, as Globus does by default.[^globus] Bag the data first so the receiver can verify it. ### Methods, standards and frameworks Mandatory rules come from funders, journals, law and consent; the standards below are voluntary unless a funder or repository requires them. | Standard or framework | Status | What it gives you | |---|---|---| | FAIR Guiding Principles (2016) | Principles; "not, themselves, a standard or a specification"[^fair2016] | 15 principles, F1 to R1.3 | | DataCite Metadata Schema 4.7 | Community schema (2026-03-03) | Six mandatory properties (Identifier, Creator, Title, Publisher, PublicationYear, ResourceType) plus recommended ones[^dcprops] | | Data Citation Principles (FORCE11, 2014) | Community declaration | Eight principles for citing data[^jddcp] | | OAIS (CCSDS 650.0-M-3, also ISO 14721:2025) | Standard reference model | Archive vocabulary: Designated Community, information packages, fixity[^oais][^oaismsi] | | CoreTrustSeal Requirements 2026–2028 | Certification of repositories | 16 requirements;[^ctshome] "all Requirements are mandatory and are equally weighted"[^cts] | | BagIt (RFC 8493) | Informational RFC | Packaging with manifests; tools "MUST support the SHA-256 and SHA-512 algorithms"[^bagit] | | RO-Crate 1.2 | Community specification | `ro-crate-metadata.json` (JSON-LD) describing a folder of research outputs[^rocrate] | | GA4GH Data Use Ontology (DUO) v1.0 | Approved standard (2021) | "a standard set of terms that can be used to tag datasets with use permissions"[^duo] | | Five Safes | Framework for safe access | Safe data, projects, people, settings and outputs[^fivesafes] | | CARE Principles for Indigenous Data Governance (2018) | Principles | Collective benefit, authority to control, responsibility, ethics;[^nsfbio] written to "complement the existing FAIR principles"[^care] | ### Tooling landscape Classes first, examples second; examples are verified to exist, not endorsed. | Job | Representative options | Selection criteria | Watch-outs | |---|---|---|---| | Plan | Funder templates; DMP tools (DMPTool, DMPonline); RDA DMP Common Standard for machine-actionable plans[^madmp] | Funder templates kept current | Plans that never change after award | | Find a repository | re3data, "a comprehensive registry of research data repositories"[^re3data] | Domain fit, certification, persistence policy | Choosing by convenience alone | | Generalist repository | Zenodo, Dryad, Dataverse installations, Figshare | Limits, curation, fees, access ([compared](#repositories-compared)) | Terms change | | Version large data | DataLad (Git with an annex for large files), Git LFS, DVC[^datalad] | File sizes, skills | Versioning is not depositing | | Package | BagIt tools, RO-Crate libraries | Repository accepts the format | Zip files hide content from previews | | Transfer | Globus between data transfer nodes; `rsync` with checksums for small copies | Volume, sites, verification | Unverified copies | | Assess FAIRness | F-UJI, "a web service to programatically assess FAIRness"[^fuji] | Machine-checkable signals | Scores measure metadata, not usefulness | | Measure reuse | Data Citation Corpus; COUNTER Code of Practice for Research Data[^mdc][^counter] | Open, documented methods | Citation counts lag by years | ### Automation and AI Automation pays off in the curation zone: checksums, metadata and licence checks, DOI reservation through repository APIs, and reminders for embargo and retention dates. Text mining already measures reuse: Make Data Count's open collection of article–dataset links was built with methods from a Kaggle competition.[^mdc] Large language models can draft READMEs and keywords, but invent plausible variable definitions and cannot judge consent: the data creator signs off every draft. Record-level personal data goes to an external AI service only when the controller, advised by the DPO, has a lawful basis, an Article 9 condition for special-category data such as health, a processing agreement (GDPR Article 28) and, for a third country, a Chapter V transfer route ([PrivacyOps](../privacyops/)).[^gdpr] The checker below finds obvious column names; it anonymises nothing. ## ResearchDataOps in practice ### Reference implementation **Design (illustrative):** each dataset is a **BagIt-style bag**: data and README under `data/`, a SHA-512 manifest, and under `metadata/` the DataCite fields (`datacite.json`) and the access decision with its conditions and dates (`deposit.json`). A tag manifest protects the metadata files too, as BagIt allows.[^bagit] ```text soil-2026/ bagit.txt BagIt-Version: 1.0 bag-info.txt Bagging-Date, Payload-Oxum manifest-sha512.txt checksum and path for every payload file tagmanifest-sha512.txt checksums of the other tag files, including metadata/ data/README.md files, variables, methods, licence, contact data/responses.csv the data metadata/datacite.json creators (with ORCID), titles, publisher, year, types... metadata/deposit.json {"access": "open", "personal_data": false, ...} ``` **What it checks.** Blocking: bag structure and fixity of payload and tag files (each file listed once with a matching digest; spaces or tabs as separators, as RFC 8493 allows); missing or invalid metadata files; DataCite mandatory properties except Identifier, which the repository registers;[^dcprops][^zdev] a known access value; a licence for open or embargoed data; an `embargo_date` or `access_conditions` where needed; and, for personal data, a `decision_record` plus an `anonymisation_record` if it would become public (**embargoed counts as public**). Warnings: missing recommended properties, formats outside a preferred list (an institutional choice), column names that suggest direct identifiers, missing README sections ("License" counts as "Licence") and a missing or past retention review date. Adapt the format list and patterns. _Example status: verified by execution (Python 3.8.10, Linux, 2026-10-17; re-run 2026-10-18) in Labs 1 and 3, plus edge cases (single-space and `*` manifests, missing or invalid JSON, embargoed and restricted personal data, a "License" heading)._ ```python #!/usr/bin/env python3 """Pre-deposit checks for a research dataset packaged as a BagIt-style bag. usage: rdo_check.py bag DIR write bagit.txt, bag-info.txt and both manifests rdo_check.py check DIR run the checks; exit 1 if any BLOCK finding Python 3.8+, standard library only. Screens, not guarantees: a person still decides. """ import csv import hashlib import json import re import sys from datetime import date from pathlib import Path # DataCite mandatory properties except Identifier: the repository assigns the DOI MANDATORY = ["creators", "titles", "publisher", "publicationYear", "types"] RECOMMENDED = ["subjects", "contributors", "dates", "relatedIdentifiers", "descriptions", "geoLocations"] ACCESS = ["open", "embargoed", "restricted", "closed"] PREFERRED = {".csv", ".tsv", ".txt", ".md", ".json", ".nc", ".h5", ".tif", ".pdf"} README_HEADINGS = ["files", "variables", "methods", "licence", "contact"] # or "license" ID_HINT = re.compile(r"(^|_)(name|email|phone|dob|birth|address|postcode|zip)($|_)", re.IGNORECASE) def sha512(path): h = hashlib.sha512() with open(path, "rb") as f: for chunk in iter(lambda: f.read(1 << 20), b""): h.update(chunk) return h.hexdigest() def payload(bag): return sorted(p for p in (bag / "data").rglob("*") if p.is_file()) def tag_files(bag): names = ["bagit.txt", "bag-info.txt", "manifest-sha512.txt"] meta = sorted(p.relative_to(bag).as_posix() for p in (bag / "metadata").rglob("*") if p.is_file()) return [n for n in names + meta if (bag / n).is_file()] def write_manifest(bag, name, rels): lines = [f"{sha512(bag / r)} {r}" for r in rels] (bag / name).write_text("\n".join(lines) + "\n", "utf-8") def make_bag(bag): files = payload(bag) write_manifest(bag, "manifest-sha512.txt", [p.relative_to(bag).as_posix() for p in files]) (bag / "bagit.txt").write_text( "BagIt-Version: 1.0\nTag-File-Character-Encoding: UTF-8\n", "utf-8") octets = sum(p.stat().st_size for p in files) (bag / "bag-info.txt").write_text( f"Bagging-Date: {date.today().isoformat()}\n" f"Payload-Oxum: {octets}.{len(files)}\n", "utf-8") write_manifest(bag, "tagmanifest-sha512.txt", tag_files(bag)) print(f"bagged {len(files)} payload files, {octets} bytes") def read_manifest(path, add): listed = {} for n, line in enumerate(path.read_text("utf-8").splitlines(), 1): if not line.strip(): continue parts = line.split(maxsplit=1) # RFC 8493: one or more spaces or tabs if len(parts) != 2: add("BLOCK", f"{path.name} line {n} is malformed") continue digest, rel = parts[0].lower(), parts[1].lstrip("*") if rel in listed: add("BLOCK", f"listed twice in {path.name}: {rel}") listed[rel] = digest return listed def compare(listed, on_disk, add): for rel in sorted(set(on_disk) - set(listed)): add("BLOCK", f"file not in manifest: {rel}") for rel in sorted(set(listed) - set(on_disk)): add("BLOCK", f"manifest lists a missing file: {rel}") for rel in sorted(set(listed) & set(on_disk)): if sha512(on_disk[rel]) != listed[rel]: add("BLOCK", f"checksum mismatch (fixity failure): {rel}") def check_fixity(bag, add): manifest = bag / "manifest-sha512.txt" if not (bag / "bagit.txt").is_file() or not manifest.is_file(): add("BLOCK", "not a bag: run 'bag' first") return compare(read_manifest(manifest, add), {p.relative_to(bag).as_posix(): p for p in payload(bag)}, add) tags = bag / "tagmanifest-sha512.txt" if not tags.is_file(): add("WARN", "no tag manifest: metadata files are not fixity-protected") return compare(read_manifest(tags, add), {r: bag / r for r in tag_files(bag)}, add) def load_json(bag, rel, add): try: return json.loads((bag / rel).read_text("utf-8")) except FileNotFoundError: add("BLOCK", f"{rel} is missing") except json.JSONDecodeError as err: add("BLOCK", f"{rel} is not valid JSON: {err}") return None def check_metadata(bag, add): meta = load_json(bag, "metadata/datacite.json", add) dep = load_json(bag, "metadata/deposit.json", add) if meta is None or dep is None: return for key in MANDATORY: if not meta.get(key): add("BLOCK", f"DataCite mandatory property missing: {key}") missing = [k for k in RECOMMENDED if not meta.get(k)] if missing: add("WARN", f"missing recommended: {', '.join(missing)}") access = dep.get("access") public = access in ("open", "embargoed") # embargoed files open by themselves if access not in ACCESS: add("BLOCK", f"access must be one of: {', '.join(ACCESS)}") if public and not meta.get("rightsList"): add("BLOCK", "open or embargoed data needs a licence in rightsList") if access == "embargoed" and not dep.get("embargo_date"): add("BLOCK", "embargoed data needs an embargo_date") if access == "restricted" and not dep.get("access_conditions"): add("BLOCK", "restricted data needs access_conditions") if dep.get("personal_data"): if public and not dep.get("anonymisation_record"): add("BLOCK", "personal data would go public without an anonymisation record") if not dep.get("decision_record"): add("BLOCK", "personal data needs a decision_record (who decided, and why)") review = dep.get("retention_review", "") if not review or review < date.today().isoformat(): add("WARN", "retention_review date missing or in the past") def check_payload(bag, add): for p in payload(bag): rel = p.relative_to(bag).as_posix() if p.suffix.lower() not in PREFERRED: add("WARN", f"format not on the preferred list: {rel}") if p.suffix.lower() in (".csv", ".tsv"): with open(p, newline="", encoding="utf-8") as f: delim = "\t" if p.suffix.lower() == ".tsv" else "," header = next(csv.reader(f, delimiter=delim), []) hits = [c for c in header if ID_HINT.search(c)] if hits: add("WARN", f"possible direct identifiers in {rel}: {', '.join(hits)}") readme = bag / "data" / "README.md" if not readme.is_file(): add("BLOCK", "data/README.md is missing") return text = readme.read_text("utf-8").lower().replace("license", "licence") heads = re.findall(r"^#+\s*(.+)$", text, re.MULTILINE) gaps = [w for w in README_HEADINGS if not any(w in h for h in heads)] if gaps: add("WARN", f"README lacks sections: {', '.join(gaps)}") def main(argv): if len(argv) != 3 or argv[1] not in ("bag", "check"): print(__doc__) return 2 bag = Path(argv[2]) if argv[1] == "bag": make_bag(bag) return 0 findings = [] add = lambda level, msg: findings.append((level, msg)) # noqa: E731 check_fixity(bag, add) check_metadata(bag, add) check_payload(bag, add) for level, msg in findings: print(f"{level:5} {msg}") blocks = sum(1 for level, _ in findings if level == "BLOCK") warns = len(findings) - blocks verdict = "NOT READY" if blocks else "READY" print(f"result: {verdict} ({blocks} blocking, {warns} warning(s))") return 1 if blocks else 0 if __name__ == "__main__": sys.exit(main(sys.argv)) ``` ### Hands-on labs #### Lab 1 (beginner): package and check a dataset **Objective:** turn a folder into a checked deposit package and fix what blocks it. **Prerequisites:** Python 3.8 or later, bash, coreutils; no account, cost or risk; about 30 minutes. Each block runs in a subshell, `( … )`, so its `set -e` cannot close your terminal when a command fails. 1. In an empty directory, save the checker above as `rdo_check.py`. 2. Create the example dataset (it contains deliberate problems): _Example status: verified by execution (bash 5.0, 2026-10-17), also pasted into an interactive shell._ ```bash ( set -euo pipefail mkdir -p soil-2026/data soil-2026/metadata && cd soil-2026 cat > data/README.md <<'MD' # Soil moisture survey 2026 (illustrative) ## Files responses.csv: one row per plot visit. ## Variables plot_id (text), visit_date (ISO 8601), moisture_pct (0-100), observer_email (text). ## Methods Capacitance probe at 10 cm, three readings averaged. ## Contact Data steward, example.org MD cat > data/responses.csv <<'CSV' plot_id,visit_date,moisture_pct,observer_email P01,2026-05-02,23.4,a.observer@example.org P02,2026-05-02,19.8,a.observer@example.org CSV cat > metadata/datacite.json <<'JSON' { "creators": [{"name": "Garcia, Sofia", "nameType": "Personal", "nameIdentifiers": [{"nameIdentifier": "https://orcid.org/0000-0000-0000-0001", "nameIdentifierScheme": "ORCID"}]}], "titles": [{"title": "Soil moisture survey 2026"}], "publicationYear": "2026", "types": {"resourceTypeGeneral": "Dataset"}, "descriptions": [{"description": "Plot-level soil moisture.", "descriptionType": "Abstract"}] } JSON cat > metadata/deposit.json <<'JSON' {"access": "open", "personal_data": false, "retention_review": "2036-06-30"} JSON ) ``` 3. Run `python3 rdo_check.py bag soil-2026 && python3 rdo_check.py check soil-2026; echo "exit=$?"`. 4. Fix the findings: drop the email column, add a publisher and a licence, add a Licence section to the README, then bag and check again. ```bash ( set -euo pipefail cd soil-2026 cut -d, -f1-3 data/responses.csv > data/r.tmp && mv data/r.tmp data/responses.csv sed -i 's/, observer_email (text)//' data/README.md printf '## Licence\nCC BY 4.0\n' >> data/README.md python3 - <<'PY' import json p = "metadata/datacite.json" m = json.load(open(p)) m["publisher"] = {"name": "Example University Research Data Repository"} m["rightsList"] = [{"rights": "Creative Commons Attribution 4.0 International", "rightsIdentifier": "CC-BY-4.0"}] json.dump(m, open(p, "w"), indent=2) PY ) python3 rdo_check.py bag soil-2026 && python3 rdo_check.py check soil-2026; echo "exit=$?" ``` **Expected output** (checked by execution). Step 3: ```text bagged 2 payload files, 422 bytes BLOCK DataCite mandatory property missing: publisher WARN missing recommended: subjects, contributors, dates, relatedIdentifiers, geoLocations BLOCK open or embargoed data needs a licence in rightsList WARN possible direct identifiers in data/responses.csv: observer_email WARN README lacks sections: licence result: NOT READY (2 blocking, 3 warning(s)) exit=1 ``` Step 4: `bagged 2 payload files, 359 bytes`, one warning (recommended properties), `result: READY (0 blocking, 1 warning(s))`, `exit=0`. **Validation:** `(cd soil-2026 && sha512sum --check manifest-sha512.txt)` prints `OK` for both files. Note what the checker caught: `deposit.json` said `"personal_data": false` while the file held email addresses; declarations need checking. **Troubleshooting:** `JSONDecodeError` usually means a trailing comma; on macOS use `sed -i ''`. **Teardown:** keep `soil-2026` if you continue to Labs 2 and 3; otherwise delete it. #### Lab 2 (intermediate): deposit to a sandbox and create a version **Objective:** practise the deposit API without minting real DOIs, with publishing kept in human hands. **Prerequisites:** the Lab 1 bag; `curl`, `zip`, Python 3.8+; a Zenodo **sandbox** account. The sandbox "requires a separate registration and separate access token", issues test DOIs with the `10.5072` prefix and "can be cleaned at anytime".[^zdev] **Least privilege.** Zenodo's `deposit:write` scope "does not allow publishing the upload"; publishing and new versions need `deposit:actions`.[^zdev] Automation therefore holds a write-only token and builds drafts; the reviewer holds the `deposit:actions` token (or publishes in the web interface). Keep tokens in a secret store, never in the bag or Git. The draft script maps the recorded access decision and licence, and refuses unprotected personal data. _Example status: `bash -n` passed; run on 2026-10-17 and 2026-10-18 against a local stand-in for `curl` returning responses shaped like Zenodo's documentation (open, CC0, embargoed, restricted, NOT READY and no-`rightsIdentifier` bags; publish, new version, wrong id). Not run against Zenodo._ ```bash #!/usr/bin/env bash # Create a Zenodo SANDBOX draft from a checked bag. This script never publishes. # Token: ZENODO_SANDBOX_TOKEN with the deposit:write scope ONLY. # Needs: bash 4+, curl, zip, python3 3.8+ and rdo_check.py in this directory. set -euo pipefail API="https://sandbox.zenodo.org/api" BAG="${1:?usage: zenodo_draft.sh BAG_DIR}" : "${ZENODO_SANDBOX_TOKEN:?export ZENODO_SANDBOX_TOKEN first (never commit it)}" H=(-H "Authorization: Bearer ${ZENODO_SANDBOX_TOKEN}") J=(-H "Content-Type: application/json") jget() { python3 -c 'import json, sys d = json.load(sys.stdin) for k in sys.argv[1:]: d = d[k] print(d)' "$@"; } python3 rdo_check.py check "$BAG" # stops here unless READY # Map DataCite fields and the recorded access decision to Zenodo metadata. meta=$(python3 - "$BAG" <<'PY' import json, sys bag = sys.argv[1] m = json.load(open(f"{bag}/metadata/datacite.json")) d = json.load(open(f"{bag}/metadata/deposit.json")) access = d["access"] if d.get("personal_data") and access != "restricted" \ and not d.get("anonymisation_record"): sys.exit("refused: personal data must be restricted or anonymised") creators = [] for c in m["creators"]: entry = {"name": c["name"]} for i in c.get("nameIdentifiers", []): if i.get("nameIdentifierScheme") == "ORCID": entry["orcid"] = i["nameIdentifier"].rsplit("/", 1)[-1] creators.append(entry) md = {"upload_type": "dataset", "title": m["titles"][0]["title"], "description": m["descriptions"][0]["description"], "creators": creators, "access_right": access} if access in ("open", "embargoed"): lic = m["rightsList"][0].get("rightsIdentifier") if not lic: sys.exit("refused: rightsList needs a rightsIdentifier, e.g. CC-BY-4.0") md["license"] = lic.lower() # Zenodo ids, e.g. cc-by-4.0, cc0-1.0 if access == "embargoed": md["embargo_date"] = d["embargo_date"] if access == "restricted": md["access_conditions"] = d["access_conditions"] print(json.dumps({"metadata": md})) PY ) zip -qr "${BAG}.zip" "$BAG" dep=$(curl -sSf "${H[@]}" "${J[@]}" -X POST "$API/deposit/depositions" -d '{}') id=$(jget id <<<"$dep") bucket=$(jget links bucket <<<"$dep") for f in "${BAG}.zip" "$BAG/data/README.md"; do curl -sSf "${H[@]}" --upload-file "$f" "$bucket/$(basename "$f")" >/dev/null done curl -sSf "${H[@]}" "${J[@]}" -X PUT "$API/deposit/depositions/$id" \ -d "$meta" >/dev/null echo "draft $id is ready for human review: $(jget links html <<<"$dep")" ``` The second script is run by the reviewer, never by CI: ```bash #!/usr/bin/env bash # Publish a reviewed draft, or open a new version. Run by a PERSON, not by CI. # Token: ZENODO_SANDBOX_PUBLISH_TOKEN with deposit:actions, held by that person. set -euo pipefail API="https://sandbox.zenodo.org/api" USAGE="usage: zenodo_release.sh publish|newversion DEPOSITION_ID" ACTION="${1:?$USAGE}" ID="${2:?$USAGE}" : "${ZENODO_SANDBOX_PUBLISH_TOKEN:?export ZENODO_SANDBOX_PUBLISH_TOKEN first}" case "$ACTION" in publish|newversion) ;; *) echo "$USAGE" >&2; exit 2 ;; esac read -r -p "Reviewed files, access and licence? Type the id to $ACTION it: " typed [[ "$typed" == "$ID" ]] || { echo "aborted" >&2; exit 1; } curl -sSf -H "Authorization: Bearer ${ZENODO_SANDBOX_PUBLISH_TOKEN}" \ -X POST "$API/deposit/depositions/$ID/actions/$ACTION" | python3 -c 'import json, sys r = json.load(sys.stdin) print(json.dumps({k: r.get(k) for k in ("doi", "record_id", "conceptrecid")})) print("latest draft:", r.get("links", {}).get("latest_draft"))' ``` **Steps:** (1) run `bash zenodo_draft.sh soil-2026` with the write-only token; (2) the reviewer checks the draft's files, access and licence at the printed URL and runs `bash zenodo_release.sh publish ID`; (3) the reviewer runs `newversion` with the same id, replaces the zip in the new draft with a corrected, re-checked bag, and publishes it. **Expected output** (from the API documentation, not observed): a draft URL; JSON whose `doi` starts with `10.5072/zenodo.`, with `record_id` and `conceptrecid`; then the new draft's URL. The concept identifier "represents all the versions".[^zver] **Validation:** version 1 still resolves after version 2 is published, and the draft token cannot publish. **Troubleshooting:** HTTP 400 names the rejected fields (for example `metadata.access_right`); `newversion` must use the latest version's id and returns the original record, not the draft.[^zdev] **Teardown:** a *published* deposition cannot be deleted (HTTP 403);[^zdev] leave sandbox records to be cleaned. Never practise on `zenodo.org`: production DOIs are permanent. #### Lab 3 (advanced): fixity audit and incident tabletop **Objective:** detect silent change, see why an embargo does not protect personal data, and rehearse an incident. **Prerequisites:** Lab 1; 60 minutes with a PI, a steward and, for step 3, the DPO or ethics office. **Risk:** none (local copies). 1. Copy the bag, make an undocumented edit, then a wrong declaration with a tempting "fix" (mark the data personal, embargo it): ```bash ( set -euo pipefail cp -r soil-2026 soil-audit echo "P03,2026-05-03,21.0" >> soil-audit/data/responses.csv python3 - <<'PY' import json p = "soil-audit/metadata/deposit.json" d = json.load(open(p)) d.update(personal_data=True, access="embargoed", embargo_date="2027-06-30") json.dump(d, open(p, "w")) PY ) python3 rdo_check.py check soil-audit; echo "exit=$?" (cd soil-audit && sha512sum --check manifest-sha512.txt); echo "exit=$?" ``` 2. Read the findings (below), then switch `access` to `restricted` and see which fields the checker now asks for. 3. Tabletop: assume version 1 was already public when the email column was discovered. Walk through the [runbook](#worked-runbook-sensitive-data-published-by-mistake) and write down who decides each step, and within what time. **Expected output** (checked by execution): ```text BLOCK checksum mismatch (fixity failure): data/responses.csv BLOCK checksum mismatch (fixity failure): metadata/deposit.json WARN missing recommended: subjects, contributors, dates, relatedIdentifiers, geoLocations BLOCK personal data would go public without an anonymisation record BLOCK personal data needs a decision_record (who decided, and why) result: NOT READY (4 blocking, 1 warning(s)) exit=1 ``` `sha512sum` prints `data/responses.csv: FAILED` and `exit=1`. The tag manifest caught the edited declaration; the embargo changed nothing. **Validation:** the tabletop produces a named decision-maker for withdrawal, notification and re-release. **Troubleshooting:** if no mismatch appears, you re-ran `bag` after editing, which silently blessed the change; bag only at deliberate release points. **Teardown:** `rm -r soil-audit`. ### Use cases and scenarios #### Case 1: UK Biobank, controlled access at scale (sourced) UK Biobank holds "over 30 petabytes of individual-level data stored centrally" and provides access through its cloud-based Research Analysis Platform (UKB-RAP); researchers pay an access fee and "will incur costs for using computational power, storing data and for transferring data out of the cloud".[^ukbrap] When it moved to platform-only access in 2024, one researcher told *The Transmitter* that re-running imaging analyses on a commercial cloud would be "extremely prohibitive"; chief executive Rory Collins called the transition "painful" but the way to secure "the long-term stability of the resource".[^transmitter] In 2026 the model was tested: "All access to UK Biobank’s Research Analysis Platform (UKB-RAP) was stopped in April due to clear breaches of the UK Biobank Material Transfer Agreement".[^ukbclosure] On 18 September UK Biobank expected re-opening to start no earlier than October, in phases, beginning at institutions that had confirmed deleting data downloaded for ended projects; the platform page still said "currently closed whilst we implement necessary changes" in the latest archived copy (11 October 2026).[^ukbreopen][^ukbrap] **Lessons:** a secure environment moves risk to agreements, training and output checks rather than removing it; costs move from the data holder to the analyst; and one closed platform halted reuse for more than five months. #### Case 2: BIDS and OpenNeuro, a community standard (sourced) In 2016 the BIDS authors wrote that "there has been no widely adopted standard for organizing and describing the data collected in an imaging experiment", which made sharing "difficult if not impossible".[^bids2016] BIDS defines file layout and metadata conventions with validators; the BIDS site lists more than 1,500 OpenNeuro public datasets (as of October 2026).[^bidssite] **Lesson:** where a domain standard exists (FAIR R1.3), the community validator is the cheapest deposit check, and every reuser repays the conversion. **Trade-off:** converting legacy data takes time, and the standard evolves. #### Case 3: a lab moves to NIH's 2026 plan format (illustrative) An NIH-funded lab writes its new application's plan in the 2026 format: yes/no answers on sharing and retention, up to 300 words on limitations, and a short table of data types and repositories.[^nih2026] Its other plans convert too: from FY2027 (which began on 1 October 2026) NIH requires "all existing awards and applications subject to the NIH DMS Policy" to use the format. Applications submitted before 25 May and not yet awarded convert at Just-in-Time; awards under the 2023 policy at their next progress report (RPPR), or the FY2027 one if the FY2026 RPPR is already in; and all active awards expected to generate scientific data, including those under the 2003 policy, with the FY2027 RPPR. The RPPR (section C.5.c) now also reports sharing progress and plan changes.[^nih2026b] Career development (K) awards no longer need a plan.[^nih2026c] For each paper the lab reserves a DOI before submission (Zenodo's `prereserve_doi`),[^zdev] runs the checks and publishes on acceptance. **Success:** each plan is converted before its report is due; data is public by the publication date. **Pitfall:** treating conversion as paperwork; later reports track the new plan. #### Case 4: a retention register (illustrative) A UK university holds EPSRC-funded data, which must be "securely preserved for at least 10 years from the date that any researcher ‘privileged access’ period expires or, if others have accessed the data, from the last date on which access to the data was requested".[^epsrc] The register therefore stores the last request date, not only the deposit date, and an annual job lists datasets due for review. **Trade-off:** popular data is kept indefinitely and needs a budget line. **Success:** every dataset has an owner and a review date. ## Measure, govern, pay ### Metrics and KPIs | KPI | Definition / formula | Pitfall | Cadence | |---|---|---|---| | DMP currency | Projects whose plan postdates their last material change ÷ data-generating projects | Plans never updated | Quarterly | | On-time sharing | Datasets public by their trigger date (publication or award end) ÷ datasets due | Counting deposits, not deadlines | Quarterly | | DAS resolution | Sampled papers whose DAS identifier resolves to the stated data ÷ papers sampled | "On request" counted as shared | Twice a year | | Metadata completeness | Recommended DataCite properties present ÷ applicable recommended properties (six) | Filler values | Per deposit | | Fixity pass rate | Files verifying against stored checksums ÷ files audited (target 100%) | Auditing only new files | Annual | | Time to DOI | Median days from complete request to findable DOI | Clock starts at a vague email | Monthly | | Access turnaround | Median days from complete access request to decision | Ignoring incomplete requests | Monthly | | Retention coverage | Datasets with an owner and a future review date ÷ datasets held | Owners who have left | Annual | | Reuse | Data citations and repository usage reports[^mdc][^counter] | Lags years; varies by field; bot traffic | Annual | Leading: DMP currency, completeness, retention coverage. Lagging: on-time sharing, DAS resolution, reuse. #### Worked calculation (illustrative) A department had 40 datasets due in a year; 31 were public by their trigger date: on-time sharing = 31 ÷ 40 = **77.5%**. Of 20 sampled papers, 17 had a DAS DOI that resolved to the right data: **85%**. Deposits averaged 3.2 of 6 recommended properties: **53%** completeness. The worst number is not the most urgent: the 9 late datasets and 3 broken statements are compliance exposures; completeness is a quality goal. Set targets one review cycle ahead (for example 90% on time) and investigate each miss. ### Quality, security, privacy and compliance Jurisdiction notes, as of October 2026; this is not legal advice ([PrivacyOps](../privacyops/) owns privacy law). - **EU and EEA (GDPR).** Research safeguards must "ensure respect for the principle of data minimisation" and "may include pseudonymisation" (Article 89(1)); pseudonymised data that could be re-attributed "should be considered to be information on an identifiable natural person" (Recital 26). A personal data breach must be notified "where feasible, not later than 72 hours" after the controller becomes aware of it, unless it "is unlikely to result in a risk to the rights and freedoms of natural persons" (Article 33), which the controller must be able to demonstrate (Recital 85). People affected are told when the risk is high (Article 34), and every breach is documented (Article 33(5)).[^gdpr] - **United States (NIH).** The 2026 plan asks how participant data is protected, "including whether any scientific data will be shared using access controls".[^nih2026] HIPAA belongs to [PrivacyOps](../privacyops/) and [HealthOps](../healthops/). - **UK (EPSRC).** Take "all reasonable steps to ensure that publicly funded data is not held in any jurisdiction where the available legal safeguards provide lower levels of protection than are available in the UK".[^epsrc] - **Consent is the ceiling** for every access route; DUO tags make permissions machine-readable, and the Five Safes frame the controls.[^duo][^fivesafes] Indigenous data also answers to the communities concerned (CARE).[^care] - **US research security.** NSF's biology directorate cites research security risks of bulk sensitive personal data, including health and human genomic data; its plan form records access restrictions (updated 12 August 2026).[^nsfbio] Security and export rules belong to [ComplianceOps](../complianceops/). ### Cost, capacity and sustainability **Funding mismatch.** Stewardship lasts decades; budgets last years. NIH allows preservation fees, "but funds for these activities must be spent during the performance period".[^nih2021] Decide early what to keep; prepay, or use a repository whose retention promise you can cite. **A simple model.** Lifetime cost ≈ curation hours × hourly rate + Σ over years (TB × copies × cost per TB-year) + repository fees + reuse costs borne by others (for example cloud compute in a secure environment, as in [Case 1](#case-1-uk-biobank-controlled-access-at-scale-sourced)). **Worked example (illustrative units, not prices).** Raw imaging: 12 TB kept in 2 copies for 10 years = 240 TB-years; at 60 units per TB-year that is 14,400, plus 40 curation hours at 50 = 2,000: **16,400**. Derived data: 0.4 TB deposited in a generalist repository, 25 curation hours = **1,250** plus any fee. Whether the raw data is worth 13 times more is a scientific question: can others re-analyse it, and will they? **Capacity limits shape architecture:** generalist repositories cap record size and file counts ([compared](#repositories-compared), as of October 2026), and Dryad charges a data publishing charge based on the costs of curating, publishing and preserving unless a partner sponsors it.[^dryadcost] Beyond those limits, use a domain or institutional archive with a DOI landing page. ### Assurance of the operating model Assurance reuses earlier parts: the [KPIs](#metrics-and-kpis) and the [annual review](#annual-review) (fixity, DAS sample, access reconciliation, as Case 1 shows is needed); CoreTrustSeal certification or a published preservation policy for each repository relied on;[^cts] and automated FAIR assessment such as F-UJI as a smoke test, not a verdict.[^fuji] ## When things go wrong ### Common failures and troubleshooting | Symptom | Likely cause | Diagnosis | Remediation | Prevention | |---|---|---|---|---| | DOI in a paper does not resolve | A draft was cited; drafts "are not DOIs"[^dcstates] | Check the DOI's state | Publish it or correct the paper | Publish before the paper appears | | Reusers cannot interpret the data | No codebook; proprietary format | Colleague test on three columns | New version with README, open formats | README template on Day 1 | | Checksum mismatch | Silent edit or storage fault | `sha512sum --check`; compare with repository copy | Restore from a verified copy; record the event | Bag only at release points; annual audits | | Access requests pile up | Committee or steward role vacant | Age of open requests | Name a deputy; publish turnaround target | Two named owners per dataset | | Data deleted too early, or never | No retention register; only copy on purged scratch | Find the dataset's promise (plan, funder) | Recover from backup if possible; record the decision | Review dates in the register | | Personal data in an open record | Screening skipped or wrong declaration | Inspect files; check download logs | [Runbook](#worked-runbook-sensitive-data-published-by-mistake) | Screens plus human review | | Repository changes terms or closes | Single home, no exit plan | Read its succession policy | Migrate; keep DOIs resolving | Prefer published succession plans[^zpol] | #### Worked runbook: sensitive data published by mistake **Scenario (illustrative).** Two weeks after publication, a reuser reports that `responses.csv` in an open record contains participant email addresses. Roles: data steward (incident lead), PI, DPO, repository support. ::: figure Runbook for personal data found in a published dataset (EU example) ```mermaid flowchart TB R["Report: personal data in open files"] --> C["Contain: ask the repository to restrict or withdraw the files"] C --> D["Same day: inform the DPO; start the breach record"] D --> A{"Can the controller show the breach is unlikely to risk people?"} A -- "Yes" --> L["Record the reasoning in the breach record"] A -- "No" --> N["Notify the authority: 72 h from awareness where feasible"] N --> H{"High risk to people?"} H -- "Yes" --> I["Inform the people affected without undue delay"] H -- "No" --> W I --> W L --> W["Files withdrawn from public view; deletion requested and confirmed"] W --> V["Clean the data; publish a corrected version"] V --> P["Post-incident review: fix screens and consent text"] classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029 classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029 class R,C,D,N,I risk class A,H decision class L,W,V,P core ``` Text equivalent: Contain first: ask the repository to restrict or withdraw the files; the same day, inform the DPO and start the breach record that every breach gets. If the controller (the institution) can show the breach is unlikely to result in a risk to people, it records that reasoning; if not, it notifies the supervisory authority (GDPR: within 72 hours of awareness where feasible) and, if the risk is high, informs the people affected. Either way the files are withdrawn from public view with deletion requested and confirmed, a corrected version is published, and a review fixes the screens and consent wording. ::: 1. **Contain (hour 0–2).** Ask repository support to restrict or withdraw the files. Withdrawal hides; it does not erase. Zenodo: "The DOI and the URL of the original object are retained"; for retracted records, "the data files and record are preserved".[^zpol] For personal data, request deletion of the files and get written confirmation. A findable DataCite DOI "can’t be deleted"; the Registered state hides its metadata from public APIs,[^dcstates] but it "can be retrieved via the Member API, so it is not fully restricted".[^dcsens] 2. **Escalate and record (same day).** The DPO advises the institution's assessment against the gates above; the 72-hour clock runs from the controller's awareness.[^gdpr] 3. **Assess** fields, people affected, time public and downloads (repository statistics). 4. **Tombstone:** the landing page keeps the citation and DOI and says why the item is unavailable.[^dctomb] 5. **Correct:** remove the column, re-run the [checks](#reference-implementation), publish a version with a change note. 6. **Learn:** update the screens and the consent or README template; close the breach record. Other jurisdictions set other thresholds; this is not legal advice ([PrivacyOps](../privacyops/)). **What you cannot do:** recall copies already downloaded from an open record. That is why the screen comes before publication. ### Anti-patterns and myths - **"Available on reasonable request."** Rarely honoured ([evidence](#the-problem-it-solves)); deposit instead. - **"FAIR means open" or "open means FAIR."** FAIR allows authentication;[^gofaira12] an open, undocumented spreadsheet is not reusable. - **"The DOI preserves it."** A DOI promises a landing page. Zenodo states: "Zenodo makes no promises of usability and understandability of deposited objects over time."[^zpol] - **Lab websites as archives.** NSF's biology directorate warns that plans "relying on self-publication on personal or laboratory websites, instead of using available publicly accessible repositories, may be considered inadequate".[^nsfbio] - **Citing the concept DOI for a result.** It is meant for the latest version, which may change; cite the version DOI.[^zcite] - **Keeping everything forever.** Storage, energy and risk grow; decide and record retention. ## Deciding and adopting ### When to use and when not to use **Use it** where data carries a sharing or retention promise (funder, journal, consent), underpins publications, comes from participants, or will outlive its team: cohorts, consortia, long-term observations. **Do not build it, or build little,** when: - a curated domain repository already does the work (deposit there; keep a register entry); - the data is intermediate, with no promise and no reuse value (delete it on schedule); - the need is clinical-study data management (database builds, randomisation), a different discipline; - the data is confidential R&D that will never be published (records management and [KnowledgeOps](../knowledgeops/)); - you are one small lab: use a generalist repository, the checklists and a calendar, not an institutional service. ### Decision framework ::: figure Decision tree: where to deposit and how open ```mermaid flowchart TB S["Dataset ready to share"] --> W{"Consent and licences allow sharing?"} W -- "No" --> MO["Metadata-only record or recorded decision to withhold"] W -- "Yes" --> P{"Personal data?"} P -- "Yes" --> AN{"Documented as anonymised?"} AN -- "No" --> CA["Controlled access, open metadata"] AN -- "Yes" --> D P -- "No" --> D{"Domain repository exists?"} D -- "Yes" --> DR["Use it"] D -- "No" --> Z{"Over generalist limits?"} Z -- "Yes" --> IR["Institutional or domain archive with DOI"] Z -- "No" --> E{"Embargo needed?"} E -- "Yes" --> EM["Embargoed record with release date"] E -- "No" --> G["Generalist repository, version DOI, licence"] classDef core fill:#e8f3f2,stroke:#16756f,color:#122029 classDef risk fill:#fbeceb,stroke:#b4332b,color:#122029 classDef decision fill:#fdf3e3,stroke:#c9800f,color:#122029 class S,DR,IR,EM,G core class W,P,AN,D,Z,E decision class CA,MO risk ``` Text equivalent: If consent, contracts or third-party licences do not allow sharing, publish a metadata-only record or record a decision to withhold. Personal data that cannot be documented as anonymised (no one identifiable by means reasonably likely to be used) goes to controlled access in a domain archive or secure environment, with open metadata. Otherwise use a designated or domain repository if one exists; else an institutional or domain archive with a DOI if the data exceeds generalist limits; else an embargoed record with a release date if an embargo is needed; else a generalist repository with a version DOI and a licence. ::: Permissions and personal data come first because no repository fixes a consent problem; the identifiability test follows GDPR Recital 26, and the controller decides on the DPO's advice, alongside the ethics committee's decision.[^gdpr] Domain fit comes next: NIH "expects the use of established repositories for preserving and sharing scientific data when they are available".[^nih2026] ### Maturity model **This Atlas's model**, not an official one. SciOps proposes a different five-level model (Initial, Managed, Defined, Scalable, Optimizing) and observes that "Most neuroscience teams currently operate between Levels 1 (Initial) and 2 (Managed)"; several of its authors declare equity in DataJoint Inc.[^sciops] | Level | Observable behaviour | Exit criteria | |---|---|---| | 1. Ad hoc | Data on personal devices; "available on request" | Every new project has a DMP and a storage location | | 2. Planned | DMPs written; some deposits at publication; no checks | A register of datasets with owners exists | | 3. Managed | Templates, pre-deposit checks, stewards, DOIs for most published data | ≥ 90% on-time sharing; DAS sampling in place | | 4. Measured | KPIs reviewed; controlled-access process with turnaround targets; fixity audits; retention reviews | Every dataset has a review date; access agreements reconciled | | 5. Adaptive | Machine-actionable DMPs feed checks; domain standards by default; reuse measured; tail costs budgeted | Sustained for two review cycles | Organisations slide back when stewards leave or rules change; treat each as a review trigger. ### Adoption roadmap: 30/60/90 days | Window | Actions | Milestone | |---|---|---| | Days 1–30 | List datasets due in the next 12 months; list active NIH awards and schedule each plan conversion before its FY2027 progress report; adopt a README template; pick default repositories; name stewards; start the register | Register covers every funded project | | Days 31–60 | Pre-deposit checks in use; intake form live; access-decision template agreed with ethics and DPO; DAS sampling baseline | First datasets pass checks before deposit | | Days 61–90 | First KPI report; retention review of the oldest holdings; runbook tabletop; preservation line in new budgets | KPIs reviewed by the research office | ## Toolkit ### Checklists #### Pre-deposit - [ ] Promises checked: plan, funder, journal, consent, agreements with data providers - [ ] README covers files, variables, methods, licence and contact; codebook attached - [ ] Open or well-documented formats; proprietary originals kept alongside if needed - [ ] Personal and third-party data screened by a person, not only by a script - [ ] DataCite mandatory and recommended properties; ORCID iDs for creators - [ ] Licence, or written access conditions for restricted data - [ ] Checksums created at the release point and verified - [ ] Version DOI ready for the data availability statement #### Controlled-access readiness - [ ] Consent mapped to permitted uses (for example DUO terms) - [ ] Access committee named, with a deputy and a turnaround target - [ ] Agreement template, training requirement and output checking defined - [ ] Public landing page with non-sensitive metadata #### Leaver or end of project - [ ] Data moved off personal accounts, devices and HPC scratch before the allocation ends - [ ] Register owner reassigned to a person who is staying - [ ] Embargo, retention and review dates recorded #### Annual review - [ ] Fixity audit passed or incidents recorded - [ ] DAS sample tested - [ ] Retention reviews due this year decided - [ ] Access approvals reconciled with agreements and outputs ### Templates #### Deposit request form ```text DEPOSIT REQUEST ref: DR-____ Dataset title: Version: v__ PI (ORCID): Steward: Funder and award ID: DMP link: Ethics approval ref: Consent allows: open / restricted / none Trigger: publication (DOI/accepted date) / award end (date) Contents: data types, size (GB), file formats Personal data: none / pseudonymised / identifiable -> DPO consulted? Y/N Third-party data or licences: Proposed access: open / embargoed until ____ / restricted (conditions) Proposed repository and reason: Licence: Retention review date: Checks: rdo_check READY? Y/N Human review by: ________ date: ______ ``` #### README skeleton ```text # (version ) ## Overview what, why, who; related paper DOI ## Files one line per file or folder; formats ## Variables name, type, unit, allowed values, missing-value codes ## Methods collection, processing, software and versions ## Access licence or access conditions; how to request ## Licence e.g. CC BY 4.0 ## Citation version DOI; concept DOI for all versions ## Contact role mailbox (not a personal address) and ORCID ## Changes what changed in this version and why ``` #### Data availability statements ```text Open: The data are available in at https://doi.org/ under . Restricted: The data are available under controlled access at https://doi.org/. Access requires approval by because ; requests are answered within working days. Embargoed: The data will be released at https://doi.org/ on ; metadata are available now. ``` #### Access and retention decision record ```text DECISION RECORD ref: AD-____ Dataset and version DOI: Decision: open / embargoed / restricted / withheld / dispose / keep until ____ Reasons (consent, law, contract, risk of re-identification, reuse value): Conditions for access (who, purpose, training, outputs): Consulted: PI / DPO / ethics / data provider Decided by: Date: Next review date: Register updated: Y/N ``` ### Repositories compared Compared only where each one's documentation was checked (October 2026); terms change. | Criterion | Zenodo | Dryad | Dataverse (software) | |---|---|---|---| | Operator | CERN[^zinfra] | Non-profit; curated service[^dryadcost] | Open-source software; each installation is its own repository (e.g. Harvard Dataverse Repository)[^dvabout] | | Curation | Accepts all formats; "no promises of usability"[^zpol] | Curators "check every submission"[^dryadcur] | Depends on the installation | | Size | 50 GB per record, more "on a case-by-case basis"; up to 100 files per record (files API)[^zpol][^zdev] | Files ≤ 10 GB recommended; 2 TB per publication through the web interface[^dryadfiles] | Depends on the installation | | Licence | Depositor chooses; required for public files[^zpol] | All data CC0[^dryadreuse] | Set per dataset; Terms of Access for restricted files[^dataverse] | | Restricted data | Restricted files shared on depositor approval[^zpol] | No PII accepted[^dryadfiles] | "cannot be downloaded unless permission has been granted"[^dataverse] | | Versions | Version and concept DOIs[^zver] | Not checked for this page | Major and minor versions[^dataverse] | ### FAQ **Should we deposit code as well as data?** Yes, when the data supports a published result. Give the code its own versioned record and identifier and link it to the data ([reproducibility](#reproducibility-active-storage-and-transfer)). **The PI is moving to another institution. Does the data go too?** Not automatically. Ownership follows the award terms and institutional policy, so agree in writing what moves, what stays and who now owns each register entry, then update the DMP and the repository contact. **Retention has ended, but the dataset has a DOI. Can we delete it?** Delete the files if the decision record allows, but not the DOI: a DataCite DOI cannot be deleted, so its landing page becomes a tombstone that says what was disposed of and when.[^dctomb] **A participant withdraws consent after publication. What now?** First ask which consent. Consent to take part is an ethics requirement; the EDPB, writing on clinical trials, says it "must not be confused" with consent as a GDPR lawful basis.[^edpbctr] If consent is the lawful basis, past processing stays lawful, but that processing stops, and data with no other lawful basis is deleted.[^gdpr][^edpbctr] Either way, stop further releases, record the withdrawal, and let the controller decide, on the DPO's advice, about shared copies. **What if the repository closes?** Read its succession policy before depositing (Zenodo promises "best efforts"[^zpol]), keep your own bag and register entry, and plan to migrate. A DOI keeps resolving only if its registrant points it to the new home. ## Going further ### Advanced topics and emerging direction - **Established:** FAIR, DataCite metadata, version DOIs, BagIt, OAIS, CoreTrustSeal, committee-based controlled access. - **Emerging:** machine-actionable DMPs (the RDA standard is "a metadata application profile" for exchanging them);[^madmp] structured funder forms (NIH, NSF, 2026), which allow automated plan-to-deposit cross-checks; open data-citation collections;[^mdc] identifiers that link data, software and projects (SWHID, RAiD).[^dcschema] - **Speculative or marketing:** fully automated "FAIRification". Tools check metadata; people judge understandability and ethics. ### Learning path | Stage | Do this | |---|---| | Prerequisites | Basic file handling and spreadsheets; [DataOps](../dataops/) helps for pipelines | | 1 day | Read the FAIR principles (Box 2 of the 2016 paper) and your funder's current policy; do Lab 1 | | 1 week | Lab 2 in a sandbox; write a README and codebook for a real dataset; read a repository's policies end to end | | 30 days | Start a register for one group; sample 20 DAS links; run the Lab 3 tabletop with your DPO | | Specialise | Preservation (OAIS, CoreTrustSeal), controlled access (Five Safes, DUO) or a domain standard (BIDS) | | Portfolio | A versioned published dataset; a checker adapted to your field; a draft retention policy | ## Wrap-up ### Executive recap and cheat sheet | Topic | What to remember | |---|---| | Definition | The operations of research data management: plan, describe, identify, deposit, control access, preserve, retire. | | Essential practices | Living DMP; README and codebook; version DOIs; code linked to data; checks at release points; written access decisions; retention register. | | Metrics | On-time sharing, DAS resolution, metadata completeness, fixity, access turnaround, retention coverage. | | Watch-outs | "On request"; FAIR = open; DOI = preservation; embargo = protection; concept DOI in papers; re-bagging; data only on scratch; unfunded retention; unenforced agreements. | | Next actions | Register the datasets due this year; adopt the README and deposit templates; run Lab 1 on one real dataset. | ### A judgement call: open release or controlled access? **Situation (illustrative).** A team has survey data and interview transcripts from 300 people with a rare condition. Consent allows "sharing for research"; the journal requires a data availability statement; the funder expects maximum appropriate sharing. | | Option A: de-identify and publish openly | Option B: controlled access only | |---|---|---| | Appeal | Fast, simple, most reuse, fits "open science" | Lowest disclosure risk | | Risk | Small population and free text make re-identification plausible; open copies cannot be recalled | Requests stall if nobody owns them; reusers pay friction; one platform can close ([Case 1](#case-1-uk-biobank-controlled-access-at-scale-sourced)) | **Why the attractive choice can be wrong.** Option A looks most "FAIR", but FAIR does not require openness,[^gofaira12] and a rare condition plus free text undermines de-identification. Option B is closed in practice if requests wait for months. **Recommendation (assumptions stated).** Tier it: publish metadata, the codebook, aggregate tables and the instrument openly with a DOI; put record-level survey data under controlled access with conditions tagged in DUO terms, a named committee and a published turnaround target; withhold transcripts unless consent clearly covers sharing them. Review in two years with access statistics. **Uncertainty:** the controller judges re-identification risk with the data in front of it, on the DPO's advice and with the ethics committee, and consent wording can override every row of this table. ### Learning objectives and mastery check After this page you should be able to **explain** why ResearchDataOps is an ambiguous label over an established practice; **distinguish** FAIR from open and an embargo from protection; **package and check** a dataset; **choose** a repository and access level; **measure** sharing; and **run** a sensitive-data incident. 1. A colleague says the data is FAIR because the CSV is on the lab website. What is missing? ::: answer Show answer for question 1 A persistent identifier, metadata in a searchable resource, a licence and a home that outlives the lab (FAIR F1, F4, R1.1).[^fair2016] ::: 2. A paper cites a dataset's concept DOI; a reviewer gets different numbers. Why, and what should be cited? ::: answer Show answer for question 2 The concept DOI represents all versions; Zenodo recommends it to those who want to "always cite the latest version", and the latest version changed after the analysis.[^zcite] Cite the version DOI of the data actually used. ::: 3. A fixity audit shows a mismatch, but the manifest was regenerated yesterday after an edit. What does that tell you? ::: answer Show answer for question 3 The re-bagging blessed yesterday's edit, so the mismatch is a newer change, and yesterday's edit may itself be undocumented. Compare with the repository copy and the last release-point manifest; make manifests only at release points. ::: 4. An NIH award ends in March. In June the team wants to pay a repository for ten years of preservation. Can award funds pay? ::: answer Show answer for question 4 No. NIH allows long-term preservation fees, but "funds for these activities must be spent during the performance period".[^nih2021] Plan and prepay before the award ends, or find institutional funding. ::: 5. A steward proposes a two-year embargo for a survey file with participants' email addresses, "to be safe". What is wrong? ::: answer Show answer for question 5 An embargo delays publication; it does not prevent it. Zenodo makes embargoed content public automatically when the embargo ends.[^zpol] Remove the identifiers and document the anonymisation, or use controlled access with a recorded decision. ::: ### Conclusion "ResearchDataOps" is a name without a literature, but the work it names is real, overdue and invisible until a reviewer, a funder or a participant asks for something nobody can produce. The useful version is unglamorous: current plans, READMEs written for strangers, identifiers for what gets cited, checksums at release points, written access decisions, and a register that remembers promises after the people who made them have moved on. Openness is a means, not the goal; the goal is data that can still be trusted and reused long after the project has ended. ## Sources and further reading Every source was opened on 2026-10-17 or 2026-10-18; dates are the sources' own. Archive copies are named where a site refused scripts. Claim-level support: `research/researchdataops-ledger.md`. [^fair2016]: [The FAIR Guiding Principles for scientific data management and stewardship](https://www.nature.com/articles/sdata201618) — Mark D. Wilkinson et al., *Scientific Data* 3:160018, 2016-03-15. Checked 2026-10-17. [^gofaira12]: [A1.2](https://www.gofair.foundation/a1-2) — GO FAIR Foundation, undated (the go-fair.org principle page redirects here). Checked 2026-10-17. [^jddcp]: [Joint Declaration of Data Citation Principles](https://force11.org/info/joint-declaration-of-data-citation-principles-final/) — Data Citation Synthesis Group, ed. M. Martone, FORCE11, 2014. Checked 2026-10-17. [^oais]: [Reference Model for an Open Archival Information System (OAIS), CCSDS 650.0-M-3](https://ccsds.org/Pubs/650x0m3.pdf) — CCSDS, Recommended Practice, Issue 3, December 2024. Checked 2026-10-17. [^oaismsi]: [Open Archival Information System (OAIS)](https://msi.dublincore.org/standards/oais/) — DCMI Metadata Standards Index, undated (states publication as both CCSDS 650.0-M-3 and ISO 14721:2025). Checked 2026-10-17. [^cts]: [CoreTrustSeal Trustworthy Data Repositories Requirements](https://www.coretrustseal.org/why-certification/requirements/) — CoreTrustSeal, 2026–2028 edition. Checked 2026-10-17. [^ctshome]: [CoreTrustSeal](https://www.coretrustseal.org/) — CoreTrustSeal home page ("16 Requirements"), undated. Checked 2026-10-17. [^care]: [The CARE Principles for Indigenous Data Governance](https://www.gida-global.org/careprinciples) — Global Indigenous Data Alliance (GIDA); drafted 2018-11-08. Checked 2026-10-17. [^bagit]: [RFC 8493: The BagIt File Packaging Format (V1.0)](https://www.rfc-editor.org/rfc/rfc8493.html) — J. Kunze et al., IETF (Informational), October 2018. Checked 2026-10-17. [^rocrate]: [RO-Crate 1.2: Introduction](https://www.researchobject.org/ro-crate/specification/1.2/introduction.html) — RO-Crate community, specification 1.2. Checked 2026-10-17. [^duo]: [Data Use Ontology (DUO)](https://www.ga4gh.org/product/data-use-ontology-duo/) — GA4GH, DUO v1.0 approved 2021-02-23. Checked 2026-10-17. [^fivesafes]: [What is the Five Safes framework?](https://ukdataservice.ac.uk/help/secure-lab/what-is-the-five-safes-framework/) — UK Data Service, undated. Checked 2026-10-17. [^madmp]: [RDA DMP Common Standard for machine-actionable Data Management Plans](https://github.com/RDA-DMP-Common/RDA-DMP-Common-Standard) — RDA DMP Common Standards Working Group, GitHub. Checked 2026-10-17. [^dcschema]: [DataCite Metadata Schema](https://schema.datacite.org/) — DataCite; version 4.7 released 2026-03-03. Checked 2026-10-17. [^dcprops]: [Overview: DataCite Metadata Schema 4.7 documentation](https://datacite-metadata-schema.readthedocs.io/en/4.7/properties/overview/) — DataCite Metadata Working Group, 2026. Checked 2026-10-18. [^dccreator]: [2. Creator: DataCite Metadata Schema 4.7 documentation](https://datacite-metadata-schema.readthedocs.io/en/4.7/properties/creator/) — DataCite, 2026. Checked 2026-10-17. [^dcstates]: [DOI States](https://support.datacite.org/docs/doi-states) — DataCite Support, undated. Checked 2026-10-17. [^dctomb]: [Best Practices for Tombstone Pages](https://support.datacite.org/docs/tombstone-pages) — DataCite Support, undated. Checked 2026-10-17. [^dcsens]: [Can I register a DOI for sensitive data?](https://support.datacite.org/docs/can-i-register-a-doi-for-sensitive-data) — DataCite Support, undated. Checked 2026-10-17. [^orcid]: [About ORCID](https://info.orcid.org/what-is-orcid/) — ORCID, undated. Checked 2026-10-17. [^ror]: [Research Organization Registry (ROR)](https://msi.dublincore.org/standards/ror/) — DCMI Metadata Standards Index, undated (ror.org rate-limits scripts). Checked 2026-10-17. [^zpol]: [Policies](https://about.zenodo.org/policies/) — Zenodo (CERN), undated. Checked 2026-10-17. [^zinfra]: [Infrastructure](https://about.zenodo.org/infrastructure/) — Zenodo (CERN), undated. Checked 2026-10-17. [^zver]: [What is DOI versioning?](https://support.zenodo.org/help/en-gb/1-upload-deposit/97-what-is-doi-versioning) — Zenodo FAQ, published 2026-04-11. Checked 2026-10-17. [^zcite]: [How to share or cite a Zenodo record?](https://support.zenodo.org/help/en-gb/25-citations/234-how-to-share-or-cite-a-zenodo-record) — Zenodo FAQ, published 2026-08-03. Checked 2026-10-17. [^zdev]: [Developers (REST API)](https://developers.zenodo.org/) — Zenodo, undated. Checked 2026-10-18. [^dryadcur]: [Dataset curation](https://datadryad.org/help/guides/curation) — Dryad, undated. Checked 2026-10-17. [^dryadcost]: [Submission costs](https://datadryad.org/help/requirements/costs) — Dryad, undated (fees change). Checked 2026-10-17. [^dryadfiles]: [File requirements](https://datadryad.org/help/requirements/files) — Dryad, undated. Checked 2026-10-17. [^dryadreuse]: [How to reuse Dryad data](https://datadryad.org/help/guides/reuse) — Dryad, undated. Checked 2026-10-18. [^ccfaq]: [Frequently Asked Questions](https://creativecommons.org/faq/) — Creative Commons, undated (section "Data"). Checked 2026-10-18. [^dcclicence]: [How to License Research Data](https://www.dcc.ac.uk/guidance/how-guides/license-research-data) — Alex Ball, Digital Curation Centre, 2011, revised 2014-07-17. Checked 2026-10-18. [^dataverse]: [Dataset + File Management](https://guides.dataverse.org/en/latest/user/dataset-management.html) — Dataverse Project User Guide (latest). Checked 2026-10-17. [^dvabout]: [About the Project](https://dataverse.org/about) — The Dataverse Project, undated; read via the [Internet Archive (2026-10-05)](https://web.archive.org/web/20261005185827/https://dataverse.org/about) (HTTP 403 to scripts). Checked 2026-10-18. [^re3data]: [About](https://www.re3data.org/about) — re3data.org, undated. Checked 2026-10-17. [^datalad]: [A brief overview of DataLad](https://handbook.datalad.org/en/latest/intro/philosophy.html) — The DataLad Handbook (latest). Checked 2026-10-17. [^globus]: [How To Log In and Transfer Files with Globus](https://docs.globus.org/guides/tutorials/manage-files/transfer-files/) — Globus documentation, undated. Checked 2026-10-17. [^esnetdtn]: [Science DMZ: Data Transfer Nodes](https://fasterdata.es.net/DTN/) — ESnet fasterdata, 2025-01-16. Checked 2026-10-17. [^nersc]: [Perlmutter Scratch](https://docs.nersc.gov/filesystems/perlmutter-scratch/) — NERSC documentation, undated. Checked 2026-10-17. [^cff]: [Citation File Format (CFF)](https://citation-file-format.github.io/) — Citation File Format project, undated. Checked 2026-10-17. [^swhid]: [SWHID.org](https://www.swhid.org/) — SWHID Working Group, undated (states adoption as ISO/IEC 18670:2025 on 2025-04-23). Checked 2026-10-17. [^wrroc]: [Workflow Run RO-Crate](https://www.researchobject.org/workflow-run-crate/) — RO-Crate community working group, undated. Checked 2026-10-17. [^fuji]: [F-UJI: Automated FAIR Data Assessment Tool](https://www.f-uji.net/) — PANGAEA / FAIRsFAIR, undated. Checked 2026-10-17. [^mdc]: [Data Citation](https://makedatacount.org/data-citation/) — Make Data Count, undated (mentions an October 2026 data file). Checked 2026-10-17. [^counter]: [Code of Practice](https://www.countermetrics.org/code-of-practice/) — COUNTER Metrics (links the Code of Practice for Research Data, 2018). Checked 2026-10-17. [^nih2021]: [NOT-OD-21-013: Final NIH Policy for Data Management and Sharing](https://grants.nih.gov/grants/guide/notice-files/NOT-OD-21-013.html) — NIH, 2020-10-29, effective 2023-01-25; read via the [Internet Archive (2026-10-12)](https://web.archive.org/web/20261012001532/https://grants.nih.gov/grants/guide/notice-files/NOT-OD-21-013.html). Checked 2026-10-17. [^nih2026]: [NOT-OD-26-046: Updated Elements of an NIH Data Management and Sharing Plan](https://grants.nih.gov/grants/guide/notice-files/NOT-OD-26-046.html) — NIH, 2026-02-25; read via the [Internet Archive (2026-09-24)](https://web.archive.org/web/20260924204136/https://grants.nih.gov/grants/guide/notice-files/NOT-OD-26-046.html). Checked 2026-10-17. [^nih2026b]: [NOT-OD-26-100: Implementation Update: NIH Data Management and Sharing Plan Requirements](https://grants.nih.gov/grants/guide/notice-files/NOT-OD-26-100.html) — NIH, 2026-07-29; read via the [Internet Archive (2026-10-05)](https://web.archive.org/web/20261005212059/https://grants.nih.gov/grants/guide/notice-files/NOT-OD-26-100.html). Checked 2026-10-17. [^nih2026c]: [NOT-OD-26-120: Accelerated Implementation of Data Management and Sharing Plan Changes for NIH Career Development (K) Awards](https://grants.nih.gov/grants/guide/notice-files/NOT-OD-26-120.html) — NIH, 2026-10-05; read via the [Internet Archive (2026-10-08)](https://web.archive.org/web/20261008121551/https://www.grants.nih.gov/grants/guide/notice-files/NOT-OD-26-120.html). Checked 2026-10-17. [^nsf2026]: [Policy Notice: Implementation of Policy Changes to PAPPG 24-1, Supplement 2 (NSF 26-202)](https://www.nsf.gov/policies/document/pappg24-1-supplement-2) — U.S. National Science Foundation, published 2026-01-22, updated 2026-01-29. Checked 2026-10-17. [^nsf2011]: [Grants.gov Application Guide: A Guide for Preparation and Submission of NSF Applications via Grants.gov](https://nsf-gov-resources.nsf.gov/pubs/policydocs/grantsgovguide0111.pdf) — U.S. National Science Foundation, 2011-01-18 (summary of significant changes). Checked 2026-10-17. [^nsfbio]: [NSF BIO Data Management and Sharing Plan Guidance](https://www.nsf.gov/bio/data-management-plans) — NSF Directorate for Biological Sciences, updated 2026-08-12. Checked 2026-10-17. [^ostp2022]: [Ensuring Free, Immediate, and Equitable Access to Federally Funded Research](https://bidenwhitehouse.archives.gov/wp-content/uploads/2022/08/08-2022-OSTP-Public-Access-Memo.pdf) — Alondra Nelson, White House OSTP (archived site), 2022-08-25. Checked 2026-10-17. [^hepg]: [Horizon Europe Programme Guide, version 5.1](https://ec.europa.eu/info/funding-tenders/opportunities/docs/2021-2027/horizon/guidance/programme-guide_horizon_en.pdf) — European Commission, 2025-09-15. Checked 2026-10-17. [^epsrc]: [Policy framework on research data: expectations](https://www.ukri.org/who-we-are/epsrc/our-policies-and-standards/policy-framework-on-research-data/expectations/) — UKRI / EPSRC, updated 2022-03-31. Checked 2026-10-17. [^gdpr]: [Regulation (EU) 2016/679 (General Data Protection Regulation)](https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32016R0679) — EUR-Lex, 2016 (Articles 4, 6, 7, 17, 28, 33, 34, 39, 44 and 89; Recitals 26 and 85). Checked 2026-10-18. [^edpbctr]: [Opinion 3/2019 concerning the Questions and Answers on the interplay between the Clinical Trials Regulation (CTR) and the General Data Protection regulation (GDPR)](https://www.edpb.europa.eu/sites/default/files/files/file1/edpb_opinionctrq_a_final_en.pdf) — European Data Protection Board, adopted 2019-01-23. Checked 2026-10-18. [^vines2014]: [The availability of research data declines rapidly with article age](https://pubmed.ncbi.nlm.nih.gov/24361065/) — Timothy H. Vines et al., *Current Biology* 24(1):94–97, 2014 (abstract read through NCBI E-utilities). Checked 2026-10-17. [^gabelica2022]: [Many researchers were not compliant with their published data sharing statement: a mixed-methods study](https://pubmed.ncbi.nlm.nih.gov/35654271/) — Mirko Gabelica, Ružica Bojčić, Livia Puljak, *Journal of Clinical Epidemiology* 150:33–41, 2022 (abstract read through NCBI E-utilities). Checked 2026-10-17. [^bids2016]: [The brain imaging data structure, a format for organizing and describing outputs of neuroimaging experiments](https://pubmed.ncbi.nlm.nih.gov/27326542/) — Krzysztof J. Gorgolewski et al., *Scientific Data* 3:160044, 2016 (abstract via NCBI E-utilities). Checked 2026-10-17. [^bidssite]: [BIDS: The Brain Imaging Data Structure](https://bids.neuroimaging.io/) — BIDS community, undated (dataset count as shown on 2026-10-17). Checked 2026-10-17. [^ukbrap]: [Research Analysis Platform](https://www.ukbiobank.ac.uk/use-our-data/research-analysis-platform/) — UK Biobank, last updated 2026-09-18; read via the [Internet Archive (2026-10-11)](https://web.archive.org/web/20261011153950/https://www.ukbiobank.ac.uk/use-our-data/research-analysis-platform/) (HTTP 403 to scripts). Checked 2026-10-17. [^ukbreopen]: [UKB-RAP: update on phased re-opening of access](https://community.ukbiobank.ac.uk/hc/en-gb/community/posts/39236359774749-UKB-RAP-update-on-phased-re-opening-of-access) — UK Biobank community forum, message signed by Rory Collins, 2026-09-18; read via the [Internet Archive (2026-09-24)](https://web.archive.org/web/20260924045942/https://community.ukbiobank.ac.uk/hc/en-gb/community/posts/39236359774749-UKB-RAP-update-on-phased-re-opening-of-access). Checked 2026-10-17. [^ukbclosure]: [Ongoing closure of the UKB-RAP](https://community.ukbiobank.ac.uk/hc/en-gb/community/posts/36090225990173-Ongoing-closure-of-the-UKB-RAP) — UK Biobank community forum, post of 2026-05-14; read via the [Internet Archive (2026-05-19)](https://web.archive.org/web/20260519115107/https://community.ukbiobank.ac.uk/hc/en-gb/community/posts/36090225990173-Ongoing-closure-of-the-UKB-RAP). Checked 2026-10-17. [^transmitter]: [Data access changes to UK Biobank stir unease in neuroscientists](https://www.thetransmitter.org/community/data-access-changes-to-uk-biobank-stir-unease-in-neuroscientists/) — Calli McMurray, *The Transmitter*, 2024-07-16. Checked 2026-10-17. [^phri]: [Research Data Operations](https://www.phri.ca/what-we-do/research-data-operations/) — Population Health Research Institute (PHRI), undated. Checked 2026-10-17. [^sciops]: [SciOps: Achieving Productivity and Reliability in Data-Intensive Research](https://arxiv.org/abs/2401.00077) — Erik C. Johnson et al., arXiv:2401.00077, v1 2023-12-29, v2 2024-11-06. Checked 2026-10-17. [^carcc]: [RCD Professionalization and Facings](https://carcc.org/rcd-professionalization/facings/) — Campus Research Computing Consortium (CaRCC), undated. Checked 2026-10-17. [^rdmkitds]: [Your role: Data Steward](https://rdmkit.elixir-europe.org/data_steward) — ELIXIR RDMkit, undated. Checked 2026-10-17. [^rdmkitlc]: [Data life cycle](https://rdmkit.elixir-europe.org/data_life_cycle) — ELIXIR RDMkit, undated. Checked 2026-10-17. Changelog: 2026-10-17, first version: researched, drafted and checked by the managing author (one agent in sequence); Labs 1 and 3 executed, Lab 2 not executed; evidence ledger in `research/researchdataops-ledger.md`. Changelog: 2026-10-17, r1 revision after the independent fact check: GDPR breach gates fixed; Lab 2 split so automation cannot publish (run against a `curl` stand-in); checker treats embargoed personal data as public and covers tag files; labs run in subshells; NIH FY2027 transition; reproducibility and storage section; CARE; figure fixes; UK Biobank re-checked; FAQ rebuilt. Changelog: 2026-10-18, post-score polish: DPO and controller roles; AI services; data licences; consent withdrawal; checker and script fixes; figure 2 redrawn; Dataverse re-sourced; repeats trimmed.