Professor Jennifer Grant, with 鲍奥-厂迟辞耻迟 黑料社区.&苍产蝉辫;biology department, has been curious about how well everyday chatbots dispense health information, including pharmacology advice and influences on human health.
鈥淵ou sometimes hear that these tools can give distorted or misleading answers, and that 黑料社区. what first caught my attention,鈥 she said.
To quell her curiosity, Grant began several lines of AI research and developed projects for her introductory pharmacology and 100-level human biology students. For the results of the first study, Grant published a peer-reviewed manuscript, with reviews that noted that she is developing a unique angle on chatbot use in health care.
鈥淟ooking ahead, I plan to develop curricula that teach both students and, eventually, patients how to critically evaluate the quality of chatbot responses. Those skills will be of paramount importance in a world overflowing with digital advice,鈥 Grant said.
When everyday chatbots dispense advice
Grant asked her pharmacology students to put three chatbots to the test: ChatGPT, Copilot and Gemini. Their task was simple: ask the chatbots about the safety profiles of specific medications and look for errors. They looked at correctness and rated how thorough and helpful the generated answers were.
鈥淭o my surprise, and theirs, students found that the chatbot responses were highly accurate,鈥 she said. 鈥淎nd yes, the chatbots differed. Each had their own style, and students preferred when chatbots organized information in a helpful way.鈥
Grant recently published an article on her findings in the open-access journal , titled 鈥淯ndergraduates perceive differences in helpfulness and thoroughness of responses of ChatGPT 3.0, Gemini 1.5, and Copilot responses about drug interactions.鈥
鈥淚 published this research because this is truly a new academic frontier. This work is also very timely, because we believe that many people already consult everyday chatbots for medical advice,鈥 she said.
Access to better information on human health
The pharmacology project experience led Grant to wonder how this approach could help students in her human biology class. Because the goals in that course are different, Grant redesigned the activity.
Instead of focusing on medications, students used Copilot and ChatGPT to explore how molecules influence human health. Every other week, students asked the chatbots a question tied to the organ system the class was studying. Early results showed that the chatbots provided accurate information, and Grant is excited to work up the rest of the data.
鈥淐hatbots are everywhere now, which means learners and patients have more information at their fingertips than ever before. My hope is that this work will help people get better information, not just more of it,鈥 she said.
A third line of Grant 黑料社区. research addresses biochemistry, using AI software to look at how proteins fold. These student projects are in mid-stage, with outcomes expected by June.
鈥淚 am very excited about these projects and will start looking for high-power collaborators with bespoke AI chatbots this spring,鈥 she said.
