02 — AI product
Designing AI around evidence, not magic
Reconstructed example · real method
AIProduct designHealthcareThe problem with AI in patient research
My background is health literacy: making health information usable by anyone. Much of that is asking what makes information trustworthy. AI in patient research raises the same question.
Picture a researcher with fifty patient interviews. An AI tool reads them and writes up the main themes, with quotes. But how do you know the findings are real? AI can make things up, flatten nuance, and lose track of who said what.
So my question was: what design decision makes it impossible for a made-up finding to reach a report?
Who it's for
A researcher with dozens of patient interviews, and a team that will make decisions from the findings.
The one rule that can't break
Every AI finding stays linked to the words it came from.
How it gets built, from need to approved finding
Each step removes one way the output could be wrong. The human check is the final gate, not an optional extra.
01
The need
The real question the research has to answer.
02
Smallest useful version
(MVP)
The fewest features that make the answer trustworthy.
03
The rule it must follow
(Requirement)
Every AI finding keeps a link to its source.
04
Evidence
The AI drafts. Every claim carries its source. No source, no entry.
05
Human check
(Human in the loop)
A person clicks from finding, to quote, to full interview. Only then approves.
From what people said to what we can test
I researched what the people who'd use the tool needed, then turned each need into a requirement you can pass or fail.
“I need to trust where a quote came from.”
Testable requirementEvery finding links to at least one source. Zero unlinked findings reach a report.
“I don’t have time to reread every interview.”
Testable requirementA reviewer gets from any finding to its source quote in one click.
“One loud voice shouldn’t become a trend.”
Testable requirementEach finding shows how many people back it. Single-source findings are flagged.
Try it: the rule in action
Pick a finding. The quotes behind it light up. A finding with no source can't be approved.
Finding review · prototype
Made-up data
AI findings
Evidence for I1
S1 · Participant 02 · interview
“I keep the leaflet but I never know which bit is the bit I actually need.”
S2 · Participant 05 · interview
“My daughter reads it for me. She skips straight to the dosing.”
S3 · Participant 09 · diary
“Looked for what to do if I miss a dose. Gave up and phoned the pharmacy.”
S4 · Participant 05 · interview
“The side-effects list is the first thing I see and it puts me off.”
Every claim links to a source.
How AI goes wrong, and the safeguard for each
These are predictable for any AI working with interviews or open-text answers. Each one needed a named safeguard before the design was finished.
- Looks like · also called hallucination
- A confident finding that no participant actually said.
- Safeguard
- No source, no approval.
Human + AI
AI does
- find
- sort
- summarise
- spot patterns
Human does
- check
- interpret
- settle doubts
- approve
- stay accountable
The design makes the human check easy, not optional. The tool shows the evidence. The person decides what it means.
When AI stops and a person steps in
I added clear hand-off points to guidance for using AI in regulated work, so everyone knows exactly when a human takes over.
No source
Blocked. Nothing without evidence reaches a report.
Only one source
A reviewer decides if it stands.
Possible safety issue
Escalated to the right expert straight away.
Could identify someone
Removed, and checked again before anything is shared.