startthat.work home

Steps, kinds of technique, stakes and relations

Short definitions of the eight steps, the kinds of technique, the stakes levels and the relations between techniques. The steps and kinds are explained in full on the process pages. The stakes level you give at intake decides how many checks run, and the relations are used when techniques are chosen and ordered.

Process steps

StepWhat happens
1. IntakeThe user's input becomes a brief. Gaps become questions to the user.
2. Rule out techniquesEach technique is tested against the brief. Techniques that cannot run on the material are removed. The rest are eligible.
3. Choose analysis techniquesThe model reads a list of the eligible techniques and chooses at most 10, with a reason for each.
4. Plan analysis sequenceThe chosen techniques are placed in four named groups in running order, each technique with a description of what it does and a note of whether it needs an additional source. Preparation techniques and checks are added.
5. Run the techniques5.1 Examine the question. 5.2 Prepare the material, including any additional sources. 5.3 Analyse the material.
6. Check the findingsVerification and challenge techniques run on the findings. The library checks are applied to each result.
7. Combine the findingsThe findings become conclusions. Each conclusion gets a confidence label and a one-line reason.
8. AnswerThe answer is given with a list of what it is based on. The user can ask for a follow-up round.

Kinds of technique

Each technique has a phase tag. The tag gives its kind and the step it runs in.

Phase tagKind and stepDefinition of the tag
frameQuestion technique, 5.1Settle the question, scope, terms and assumptions before evidence is examined.
gatherPreparation technique, 5.2Find, collect and inventory the material.
verify and preparePreparation technique, 5.2Check quality and fitness of the material, then clean, extract and structure it. Ends when the material is ready for analysis. Material that cannot be read is reported in the answer basis and the techniques that need it are dropped.
analysePrimary technique, 5.3Explore, measure, model, compare or trace relationships in the prepared material.
validateVerification or challenge technique, step 6Check the result before concluding: held-out validation, agreement with a standard, sensitivity and alternative techniques.
concludeNot a technique: step 7State what the findings support, with a confidence label and a one-line reason for each conclusion.
reportNot a technique: step 8Communicate the conclusion, its limits and the assumptions behind it.

Five steps in the AIGroundwork framework: frame, gather and verify, analyse, conclude, report. Verify and prepare and gather both map to gather and verify. Validate maps to conclude because the five steps have no separate step for checking a result.

Stakes

LevelDefinition
exploratoryThe output decides what to look at next. A wrong result costs little and is corrected before anyone relies on it.
decision-informingThe output is one input to a decision that can be reversed or revisited at moderate cost.
high-stakesThe output supports a decision that is costly or hard to reverse, affects people's rights, safety or finances materially, or will be published as authoritative.

Relations between techniques

RelationStrengthMeaning
requireshardThe technique must not start until the named technique has run or the named artefact exists in the named state. Every artefact in consumes is also required. Order between techniques follows from artefacts, so direct technique to technique edges are used only when no artefact carries the dependency.
should followsoftThe technique is better run after the named technique or family, for the stated reason, but can run without it. The skill warns when it is skipped.
substitutes forconditionalThe technique gives the same artefact as the named technique with different assumptions, and can replace it when the stated facet condition is met. It is not a symmetric relation unless both techniques record it.
pairs withsoftRunning the two techniques together covers a weakness of one of them, named in the reason. Pairing does not by itself make the second technique corroboration; the corroboration rule applies.
conflicts withhard when both would act on the same artefact instanceThe two techniques cannot both be applied to the same artefact instance, or one consumes what the other needs, as when a single use held-out sample has already been opened.
loop targetoutcome ruleThe outcome when the technique's fit test fails: proceed with the limit stated, return to a named phase, or stop because the data cannot answer the question.
parallel safederivedTwo techniques are parallel safe when they share no produced or consumed artefact instance. It is computed from consumes and produces and is never stored.