startthat.work home

Step 4: Plan analysis sequence

Code places the chosen techniques in one sequence. Each technique's kind decides its group and position. The groups do no work themselves, so the plan describes what each technique does.

The plan is shown as a table of techniques under four group headings, in running order and without numbers. The columns are the technique, what the technique does (the catalogue's one-sentence purpose), and whether it needs an additional source: none, or a comparator, a reference structure or an earlier period.

Position Group name What runs
1 Clarify the question Techniques that examine the question
2 Prepare the content Techniques that prepare the material. De-identification, when it is needed, is the first of these.
3 Analyse the content Primary techniques, which analyse the material. They are independent of each other, so a finding that several reach counts for more.
4 Test the findings Techniques that check the findings, then the library checks

The plan is work (section 7), so the user may never see it. It does not carry decision points. The one decision that follows from the order, whether to reword the question after the first group, is asked in 5.1.

A position with no chosen technique is left out. Steps 5 and 6 run this sequence: the first three positions are sub-steps 5.1 to 5.3 and the last is step 6.

The plan also adds two things, each with its reason:

  • A preparation technique that produces an input a chosen technique needs. The reason names the technique that needs it.
  • De-identification, when the material has personal information.

The plan keeps the reason for each chosen technique from step 3 and the reason for each addition. The plan is work (section 7). The reasons are kept so that the answer can describe the process followed.

Additional sources. 41 of the 236 techniques need a comparator (14), a reference structure (25) or an earlier period (2). Step 4 only records the need in the third column of the table. Nothing is searched for or gathered in step 4, because no technique has run. Step 5.2 resolves the need.

Library checks come from two lists that are combined. Each technique lists its own minimum checks for each stakes level, and each higher level includes the checks of the level below. The strength of statement adds further checks, for example a held-out validation for prediction or an alternative explanations check for the effect of an action.