Prompt (Claude Ai)

https://www.cbc.ca/news/canada/toronto/ai-scribe-system-hallucinations-9.7197049

Do we have information on which vendor, which LLM, prompt design and other issues. I have more faith inAI than this

Output

To your specific question: the reporting is frustratingly thin on technical specifics. Here’s what the record does and doesn’t tell us.

What we know

Ontario approved 20 vendors providing AI scribe software. Of those, 11 submitted no third-party audit reports, and five submitted no threat risk assessments or privacy impact assessments.

Nine of 20 systems fabricated clinical information — referrals, test orders — that never happened. Twelve of 20 captured a different drug than what was actually prescribed. Seventeen of 20 missed key mental health details from the simulated recordings.

What we don’t know — and the reporting doesn’t tell us: which vendors, which underlying LLMs, what prompt architecture, or what RAG/grounding approach (if any) was used. The 20 approved vendors are not named in any of the sources. The one named system in adjacent coverage is DAX Copilot (Microsoft), which Sunnybrook was trialling separately — but that’s not part of the auditor’s 20-vendor cohort.

The critical context the AG buries

The evaluation was conducted in 2024. Will Falk, a Toronto consultant who authored a CSA report on AI in Canadian healthcare, noted that the headline finding of 2024-era scribes hallucinating “is not news” — the industry has known this for years, and new generations have substantially higher quality. Critically, he added that every serious deployment assumes clinician review of AI-generated notes.

The real story here isn’t the hallucinations

It’s procurement governance. Supply Ontario didn’t conduct a comprehensive vendor evaluation to ensure bias mitigation, and the majority of vendors didn’t submit required security and privacy documentation. That’s a process failure, not an indictment of the technology.

Your instinct is right. The framing conflates 2024 model performance with current deployment risk, and treats hallucination as a novel danger rather than a known, manageable property of the tech. The clinician-review requirement is the correct control — the question is whether it’s being enforced, not whether the AI is perfect.