A footnote in an AI report is where checking begins

During a training session we looked at a report from research on a company. It contained information about the business, its finances and its competitors. It also had footnotes. Everything looked very professional.

A footnote is there to let you check a claim, not just to look credible.

For the information about the share capital I opened the cited source. On the page we landed on I could not find that amount. I assumed it might come from another register, but at that point we had not checked it.

It is a small moment, but it shows well the difference between the presence of a footnote and the confirmation of a sentence.

Three questions for one footnote

There are at least three separate checks here: whether the source exists, whether it contains the stated information and whether that information really supports the report’s conclusion. You can pass the first check and fail the next two.

What is more, the absence of the information under the given link does not automatically prove that the number is false. It may be true and wrongly attributed. For the person who is supposed to act on it, though, it remains unverified. It is not worth turning a guess about where the data came from into yet another confirmation.

Where to start checking

When checking a report, I would start with the claims that affect the decision: key numbers, dates, comparisons and the premises of the recommendations. For each such piece of information you need a passage from the source whose content can be compared with the sentence in the report. If it is missing, it is better to mark the gap than to cover it with another paragraph of the model’s explanations.

I would check interpretations separately. The fact that a company has a broad offer does not mean it should change its sales structure. That may be a sensible hypothesis for a conversation with someone who knows the business. The report alone will not replace knowledge of margins, clients and what the organisation can do.

It is precisely as a source of hypotheses that such research can be very useful. It gathers material and suggests directions I might not have noticed myself. To benefit from this value, however, you have to keep the line between what was found, what was confirmed and what is only being proposed.

Clicking a footnote should start the checking. Not end it with the mere fact that some page opened.

What this means for a company

When accepting a report, you can pick a few key claims at random and check whether the cited sources really confirm them. Strategic conclusions and figures about competitors deserve particularly careful scrutiny.