HE1-01 Admiralty system (source reliability and information credibility grading)
Assigns each report a two-character grade for source reliability (A to F) and information credibility (1 to 6) and produces an appraised evidence base.
What it does
Gives each report a two-character grade, rating the source (A to F) and the information (1 to 6) separately. Artefact: a code such as B2; read the letter as the source's history and the number as how well the item is corroborated.
When to use it
Use when you need to say how far to trust each report from a source with a known track record, and whether other independent reports back it.
When it does not fit
The source is new, anonymous or has no comparators (default F and 6). The grades are ordinal, and reliability from past performance does not predict a new claim.
Result format
The result is a table with one row per report, giving a two-character grade such as B2, where the letter (A to F) rates the source and the number (1 to 6) rates the information. It states what each grade rests on, the source's history for the letter and the corroboration for the number.
Requirements on the material
| Property | Requirement type | Requirement |
|---|---|---|
| Content type | Always applies | requires=text |
| Unit of analysis | Always applies | requires=source; requires=claim |
| Producer interest | Changes the order | prefers=sponsored; prefers=self-interested seller; prefers=unknown |
| Source dependence | Changes the order | prefers=independent; prefers=unknown |
| Origin | Changes the order | prefers=secondary; prefers=tertiary; prefers=unknown |
| Appraises source credibility | Applies in some cases | yes |
Not restricted on: Structure, Variable types, Time structure, How the sample was chosen, Level of detail available, Number of sources, Whether sources can be joined, Amount of material, Known data problems, Document genre, Language, Personal information, Security classification, Licence.
| Minimum material | none |
|---|---|
| Additional sources needed | none |
| Not for (rules) | none |
| Not for (text) | The source is new or anonymous with no comparators, so the default grade F6 carries no information. Do not read a grade from past performance as a prediction about a new claim. |
Choosing and ordering
| Kind of technique | Preparation technique, runs in step 5.2 |
|---|---|
| Phase tag | verify and prepare |
| Other phase tags | gather |
| Kind of finding | gap; contradiction |
| Question answered | what; who |
| Strongest statement it supports | describe |
| Needs as input |
|
| Makes as output |
|
| Must exist first | None |
| Loop target | return to gather |
Library checks by stakes
Each stakes level includes the checks of the level below it. Checks that need a person apply at high stakes only, so the process leaves them out at lower stakes (decision 38). Repeatability: judgement dependent.
| Stakes | Check | Person involved | Counts as a different method |
|---|---|---|---|
| exploratory | spot check | optional | no |
| decision-informing | spot check | optional | no |
| decision-informing | human review sample | needed; left out below high stakes | no |
| high-stakes | spot check | optional | no |
| high-stakes | human review sample | needed | no |
| high-stakes | independent review | needed | no |
| high-stakes | second technique with different assumptions | none | conditional |
Failure modes and limits
| Model suitability | human judgement required |
|---|---|
| Typical model failure | Assigns a letter grade from the tone of the report instead of the source's record, or scores information credibility without checking whether the corroborating reports are independent. |
| Error sources | input data; coder or rater judgement; source selection |
| Certainty limits | Certainty drops when the track record is thin, when corroborating reports copy each other (inconsistency and risk of bias) and when grades are read as more than ordinal. |
Related techniques
- pairs with HE1-06 Triangulation (source, method, investigator and theory triangulation; corroboration): the information grade depends on corroboration, which triangulation tests
- pairs with IS-06 Source dependence and copy detection (source independence analysis): copy detection stops copied reports from raising the information grade
Evidence and tag basis
Documented. Read on a secondary page citing UK and US military publications; the primary document was not opened. (Wikipedia, Admiralty code)
Tag basis: input profile inference; selection inference; sequencing inference; robustness inference. All tag values are inference unless a source is named.