startthat.work home

Working document types

Working documents are what techniques need as input and make as output. The schema calls them artefacts. If a technique needs a document that nothing has made yet, such as a coded corpus, step 4 adds a preparation technique that makes it, and the order of those techniques follows from the documents. Each type below has a definition and the states it can be in, such as draft or checked. A type marked single use is spent by the first technique that reads it. The types are grouped by kind: the brief, material that has been prepared, findings, and reports.

Brief

Charter or question

The question, scope and success criteria for the work.

States: draft, agreed.

Prepared material

Data source inventory

List of the sources to be used, with owner, access and licence.

States: draft, checked.

Raw dataset or corpus

Documents or records exactly as gathered.

States: raw.

Data dictionary

Definitions of fields, units and codes in a dataset.

States: draft, checked.

Codebook

Categories and rules for coding text or records, fixed before testing.

States: draft, fixed.

Data quality report

Findings of the quality check on the material, including problems found.

States: complete.

Screening log

Record of items included and excluded, with reasons.

States: complete.

Cleaned dataset

Dataset after cleaning, deduplication and standardising.

States: cleaned.

Feature set

Variables or representations built from the cleaned data for analysis.

States: built.

Training sample

Part of the data used to develop codes, features or models.

States: drawn.

Held-out test sample

Part of the data kept unseen until final testing. Single use: once opened for a test it cannot be reused after the technique is changed.

States: sealed, opened. Single use.

Coded corpus

Documents or records with codes or labels applied.

States: coded, validated.

Entity table

Table of the people, organisations, places or other entities found, with one row per entity.

States: draft, resolved.

Evidence base

The set of sourced items with a quality or credibility grade.

States: collected, appraised.

Claims register

List of the claims made, each with its evidence, claim level and confidence label.

States: draft, checked.

Finding

Assumptions register

List of working assumptions, with what changes if each is wrong.

States: draft, reviewed.

Hypotheses matrix

Hypotheses scored against evidence.

States: draft, scored.

Comparison matrix

Cases, periods or options side by side on the same dimensions.

States: draft, reviewed.

Result

The output of an analysis technique: a model, estimate, ranking, map or synthesis.

States: draft, checked, validated.

Validation report

Findings of the checks on a result, including checks that failed.

States: complete.

Report

Final report

The communicated conclusion with limits and assumptions.

States: draft, reviewed.