Medical Attention

In-context: July 20, 2025

Here’s a quick wrap of the three papers we found interesting over the last few weeks with some take home points.
1:00 - Clinical knowledge in LLMs does not translate to human interactions
06:45 - From Tool to Teammate: A Randomized Controlled Trial of Clinician-AI Collaborative Workflows for Diagnosis
11:55 - Advancing Real-time Pandemic Forecasting Using Large Language Models: A COVID-19 Case Study
More details in the show notes on our website.
Episodes | Bluesky | info@medicalattention.ai

Ep.10 Are benchmarks broken?

In this episode, we’re lucky to be joined by Alexandre Sallinen and Tony O’Halloran from the Laboratory for Intelligent Global Health & Humanitarian Response Technologies to discuss how large language models are assessed, including their Massive Open Online Validation & Evaluation (MOOVE) initiative.
0:25 - Technical wrap: what are agents?
13:20 - What are benchmarks?
18:20 - Automated evaluation
20:10 - Benchmarks
37:45 - Human feedback
44:50 - LLM as judge
Read more about the projects we discuss here:
Meditron
Learn about the MOOVE or contact our team if you'd like to be involved
Listen to the LiGHTCAST including their recent excellent outline of the HealthBench paper
More details in the show notes on our website.
Episodes | Bluesky | info@medicalattention.ai

In-context: June 9, 2025

Here’s a quick wrap of the three papers we found interesting over the last few weeks with some take home points.
0:35 - Superhuman performance of a large language model on the reasoning tasks of a physician
06:20 - MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks
11:45 - Identifying and mitigating algorithmic bias in the safety net
More details in the show notes on our website.
Episodes | Bluesky | info@medicalattention.ai

In-context: May 2025

We’re trying out a new episode format! We’ll be doing a quick wrap of the top three papers we found interesting over the last few weeks with some take home points.
01:30 - Patient Reactions to Artificial Intelligence-Clinician Discrepancies
06:50 - AI-based volumetric six-tissue body composition quantification from CT cardiac attenuation scans for mortality prediction 
11:10 - Large-scale Local Deployment of DeepSeek-R1 in Pilot Hospitals in China
More details in the show notes on our website.
Episodes | Bluesky | info@medicalattention.ai

Ep.9 AI Mythbusting

In this episode, we tackle some common myths about how generative AI works, why this is the case, implications for healthcare and some quick fixes. These myths include 1) that LLMs can explain their reasoning 2) that LLMs can express uncertainty, 3) that LLMs can a) do maths, b) manage temporal data c) apply guidelines d) handle negation and finally that 4) that AI will replace clinicians.
02:00 Technical update - DeepSeek, other new models
10:00 - AI mybusting
15:50 - LLMs can explain their reasoning
21:50 - LLMs can express uncertainty
26:40 - LLM blindspots
41:50 - AI will replace clinicians
Episodes | Bluesky | info@medicalattention.ai

Ep.8 Algorithmic Bias

In this episode, we discuss algorithmic bias and fairness in healthcare. We explain what this is, the different definitions of “fairness”, explore the ways in which bias can enter the machine learning pipeline and some ways to combat it.
01:00 Technical update - NeurIPS
06:50 Technical update - ChatGPT-o3
14:00 - Algorithmic bias
Episodes | Bluesky | info@medicalattention.ai

Ep.7 Informatics Year in Review

We’ve returned after an accidental hiatus, just in time for the end of the year. In this episode, we’re joined by the team behind the American Medical Informatics Association (AMIA) Year in Review - Professor James Cimino, Pushkala Jayaraman and Dr Humayera Islam to talk about the main themes in healthcare AI for 2024.
01:00 Technical update (ChatGPT-o1, agentic AI)10:25 AMIA Year in Review
Episodes | Bluesky | info@medicalattention.ai

Ep.6 Human-computer interaction in healthcare

We’re back! This is the start of our regular discussions about healthcare AI topics and recent literature. On today’s episode -  the 10 commandments of decision support, the checkered history of EMRs, clinicians as “moral crumple zone” for AI models and much more.
01:18 Technical update (Llama 3.1, Phi releases)
06:25 Main discussion
41:20 Article round-up
Episodes | Twitter | info@medicalattention.ai