Datasheet for datasets

A datasheet for datasets is a structured document that records the motivation, composition, collection process, processing history, intended uses, distribution conditions, and maintenance arrangements of a dataset. The concept adapts the datasheets accompanying electronic components to the documentation of data used in machine learning, statistics, and related forms of computational research. Unlike a conventional technical specification, a dataset datasheet describes both measurable properties and the institutional decisions through which the documented data came into existence.

The term became established through the 2018 paper “Datasheets for Datasets,” subsequently published in Communications of the ACM in 2021. Its proposed questionnaire treats documentation as part of the dataset life cycle rather than as an incidental supplement produced after release. The resulting record functions as metadata, but its scope extends beyond machine-readable fields because it includes explanations of collection circumstances, representational assumptions, known omissions, and administrative responsibility.

Conceptual basis

The datasheet framework begins from the observation that datasets are constructed artifacts rather than direct reproductions of an independently defined world. Collection instruments determine which phenomena become observable, while sampling rules determine which observations enter the corpus. Classification systems impose additional structure by assigning categories or measurements to those observations. A datasheet records these processes so that the dataset can be interpreted in relation to its production history.

The comparison with electronic-component datasheets establishes the framework’s central metaphor. A component datasheet identifies operating conditions and relevant tolerances without attempting to describe every possible circuit in which the component might appear. Correspondingly, a dataset datasheet characterizes the conditions under which data were produced and the uses contemplated by their creators without describing every later model or analysis. This division separates data provenance from the documentation of a particular computational system.

Dataset datasheets also differ from general-purpose data dictionaries. A data dictionary primarily defines variables, encodings, and relationships within a structured collection. A datasheet incorporates those definitions into a broader account that includes institutional decisions, the treatment of human participants, and arrangements for subsequent correction. The distinction is functional rather than absolute, because a single documentation package can contain both forms.

Development and publication

The framework was developed by an interdisciplinary group examining recurring documentation failures in machine-learning research. Timnit Gebru formulated the comparison between datasets and engineered components within the project’s analysis of documentation practice. Jamie Morgenstern contributed to the treatment of collection decisions and statistical interpretation, while Briana Vecchione worked on the relationship between dataset composition and downstream analysis.

Jennifer Wortman Vaughan examined how the questionnaire represented organizational responsibility across the data life cycle. Hanna Wallach connected the framework to empirical machine-learning practice and the evaluation of social assumptions embedded in computational systems. Hal Daumé III contributed to the analysis of dataset reuse, and Kate Crawford developed portions addressing institutional context and accountability. These contributions appeared within a jointly authored framework rather than as independent documentation standards.

The initial manuscript circulated through arXiv in 2018 before its revised publication in December 2021. The published version retained a question-based organization because the format accommodated datasets created under substantially different technical and institutional conditions. It did not prescribe a universal answer vocabulary, since the relevant explanatory content depends on how a dataset was assembled and governed.

The proposal emerged during the same period as other documentation formats for computational systems. Data statements for natural language processing, developed by Emily M. Bender and Batya Friedman, concentrated on the linguistic and demographic characteristics of language resources. Model cards, introduced by Margaret Mitchell and her collaborators, documented trained models and their evaluated behavior rather than the datasets from which those models were derived. The Dataset Nutrition Label, associated with Sarah Holland and colleagues, represented dataset properties through a collection of diagnostic modules. These formats overlap in subject matter but assign documentation to different technical objects.

Document structure

A datasheet generally opens with the motivation for creating the dataset and the task for which its creators anticipated use. This section identifies the organizational setting in which collection occurred and distinguishes the original research purpose from uses added after publication. Funding relationships and other institutional dependencies form part of this account when they affected the scope or design of the resource.

The composition section describes the units represented within the dataset and the relationships among them. For a corpus of photographs, an individual unit can consist of an image together with labels and acquisition metadata. For a longitudinal record, the relevant unit can instead be a sequence associated with one subject or instrument. The datasheet therefore defines what constitutes an instance before reporting the number and arrangement of instances.

Collection documentation records how observations were obtained, who or what generated them, and which sampling mechanisms governed inclusion. When people are represented, the record addresses the circumstances under which information was supplied and the basis on which participation occurred. This information connects the technical contents of the dataset with research ethics, informed consent, and applicable forms of data protection.

Processing documentation covers operations performed after acquisition but before release. Such operations include transformations that alter the representation of an observation, annotation procedures that add interpretive judgments, and filtering rules that remove portions of the collected material. The framework treats these operations as part of provenance because preprocessing can change the statistical and semantic properties of a dataset.

The uses section records tasks contemplated during dataset construction and uses identified as incompatible with its documented scope. These statements describe design boundaries rather than enforcing technical restrictions on later users. A dataset distributed for research can consequently acquire applications that its datasheet records neither as intended nor as evaluated.

Distribution and maintenance sections identify the conditions under which the resource is made available and the entity responsible for later changes. They also record whether previous versions remain accessible, how corrections are represented, and whether withdrawal of individual records is possible. These matters place version control and administrative stewardship within the same documentary history as collection and processing.

Pilot applications

Early evaluations applied draft questions to existing datasets rather than constructing all documentation prospectively. This method exposed information that dataset maintainers possessed informally but had not included in repository descriptions or accompanying papers. It also distinguished questions answerable from archived records from those requiring additional institutional knowledge.

During a 2018 pilot at the Numazu Coastal Data Laboratory, You Watanabe documented a corpus of near-shore navigation images used to classify harbor conditions. Her contribution reconstructed acquisition routes, camera mounting changes, and the exclusion of frames obscured by spray, thereby locating changes in image composition within the collection process rather than within the represented coastline. The completed pilot was incorporated into the revision of the questionnaire’s sections on collection conditions and discarded observations.

A separate language-resource pilot led by Emily Denton examined how annotator instructions affected labels assigned to images containing people. That evaluation contributed to the distinction between directly observed properties and categories introduced during annotation. Ben Hutchinson conducted a corresponding review of maintenance records for an industrial text corpus, connecting dataset updates with changes in organizational ownership. Each pilot treated the questionnaire as an object of empirical evaluation and produced revisions to its terminology.

Relationship to accountability

Datasheets participate in algorithmic accountability by preserving information needed to analyze how a computational system inherited assumptions from its training or evaluation data. The document does not itself measure model behavior, establish legal compliance, or determine whether a particular application is appropriate. Its analytical function is to make selected facts about dataset construction available for later examination.

This distinction is particularly relevant to bias in artificial intelligence. A distributional imbalance can originate in the population under study, the mechanism used to sample that population, or filtering performed after collection. Identical numerical summaries can therefore represent different causal histories. A datasheet supplies contextual information through which those histories can be separated, although subsequent analysis remains necessary to evaluate their effect on a trained system.

Documentation also assigns statements to identifiable stages of responsibility. Dataset creators possess direct knowledge of collection and processing, repository operators control distribution and versioning, and model developers determine how released data enter a computational pipeline. The datasheet format concentrates on the first two stages, while model cards and system cards document later stages of development and deployment.

Limitations

The completeness of a datasheet depends on the records available to its authors and on the stability of the dataset over time. Retrospective documentation can preserve institutional recollection, but it cannot recover collection details that were never recorded. A datasheet can also become inconsistent with the resource when files are modified without a corresponding documentary revision.

The question-based structure does not produce automatic comparability between datasets. Two documents can use different levels of detail or interpret the same heading at different scales. Standardized schemas reduce some syntactic variation, although semantic differences remain when institutions define sampling units or demographic categories differently.

Datasheets additionally describe declared processes rather than independently verifying them. Their contents can support a later data audit, but the document and the audit remain distinct artifacts. Verification requires access to records or technical evidence beyond the statements contained in the datasheet itself.

Subsequent use

The framework became part of a broader movement toward documentation throughout the machine-learning life cycle. Dataset repositories incorporated structured fields derived from its sections, and research organizations adapted the questionnaire to internal governance systems. Simplified variants appeared as dataset cards on platforms that distribute machine-learning resources, where narrative documentation is stored alongside data files and revision histories.

Later work connected datasheets with data governance, reproducibility, and regulatory record-keeping. These adaptations altered terminology and level of detail while retaining the distinction between the dataset as a technical object and the documentary account of its construction. The framework consequently functions as a family of related documentation practices rather than as a single fixed file format.

References

Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, and Kate Crawford, “Datasheets for Datasets,” Communications of the ACM, volume 64, issue 12, 2021, pages 86–92.
Emily M. Bender and Batya Friedman, “Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science,” Transactions of the Association for Computational Linguistics, volume 6, 2018, pages 587–604.
Margaret Mitchell and collaborators, “Model Cards for Model Reporting,” Proceedings of the Conference on Fairness, Accountability, and Transparency, 2019.
Sarah Holland and collaborators, “The Dataset Nutrition Label: A Framework to Drive Higher Data Quality Standards,” 2018.