Google tests AMIE for clinical video consultations

Google’s research medical AI system, AMIE (Video), conducted synchronous video consultations with professional patient actors and received clinical evaluator ratings on par with primary care physicians across several core measures.

Fifteen trained actors portrayed conditions across cardiopulmonary, abdominal, HEENT, neurological or psychiatric, and musculoskeletal presentations. Google says studies involving real patients and their own health conditions must follow before anyone can draw conclusions about clinical use.

AMIE divides a video consultation among three agents

AMIE uses an asynchronous multi-agent architecture rather than assigning dialogue, clinical reasoning, and perception to one model process. Google says a single agent cannot currently sustain natural conversational response times while also conducting detailed reasoning and continuously processing audio-visual input.

The talker agent handles the spoken interaction with the patient. It aims to maintain conversational flow, drawing on information from the other agents. The planner agent runs in...