Miles Jamison
RTX BBN Technologies, an advanced technology research and development provider, has secured an award under an other transaction agreement to develop technologies that enhance medical chatbots for more reliable and accurate healthcare responses.
The Cambridge, Massachusetts-based company said Tuesday the funding received from the Advanced Research Projects Agency for Health, or ARPA-H, will be used to advance artificial intelligence-powered medical chatbots in support of the Chatbot Accuracy and Reliability Evaluation, or CARE, Exploration Topic.
Advancing Medical Chatbots
Existing medical chatbots and AI systems may provide inaccurate or misleading information, jeopardizing the patients. ARPA-H’s CARE intends to develop tools and technologies to assess medical chatbots used directly by patients to ensure users obtain reliable health information critical to patient care.
BBN will leverage its understanding of machine learning, language-based information processing and large language models to develop a system that will serve as a technological framework for evaluating the chatbots. The new system, called the Monitoring, Evaluation and Diagnosing of Intelligent Chatbots, or MEDIC, will incorporate feedback from healthcare providers and patients to enhance chatbot interactions. It will also cross-reference chatbot responses with medical sources to ensure accuracy and utilize advanced engineering to enable realistic interactions. MEDIC will also identify inaccuracies and omissions with the chatbot responses.
“Evaluating medical chatbots requires more than simply checking for correct answers; it demands a deep understanding of how these systems address the complex needs of diverse users,” Damianos Karakos, BBN principal investigator, said.
“Our goal is to develop an adaptable framework that rigorously assesses chatbot performance in real-world scenarios, focusing on key aspects like bias, fairness and the risk of generating misleading information,” Karakos added.
