A team of researchers from the Universidad Carlos III de Madrid (UC3M) has evaluated various artificial intelligence (AI) language models to simplify medical reports and adapt them into clearer language for non-expert audiences. This work will help patients better understand their health conditions and properly adhere to prescribed treatments.
The study, published in the journal Scientific Reports, stems from a widespread issue in modern society: many people struggle to comprehend their own medical reports, directly affecting their ability to manage their health. "We observed that 46% of European citizens face difficulties understanding medical documents. Therefore, we developed a set of tools to help improve doctor-patient communication," explains study co-author Paloma Martínez, Full Professor in UC3M's Department of Computer Science. "To solve this problem, language models can generate one version of the document for medical professionals and another tailored for the patient, ensuring they can understand abbreviations or complex phrasing found in the report."
The research arrives at a particularly timely moment, according to the research team, given that Royal Decree 707/2026 will come into effect on January 2, 2027. Published in the Official State Gazette (BOE) on September 3, this decree approves the first regulations governing basic cognitive accessibility conditions. The regulation sets forth requirements to make documents, administrative procedures, and services more understandable for citizens, impacting public administrations across fields such as justice, healthcare, employment, digital services, and transport.
Tainan Municipal Hospital has implemented a three-level AI education programme to help staff build trust in the technology, says deputy superintendent and CIO Dr Hsing-Hsien Wu.
A retired Navy Nurse Corps officer discusses suicide risk screening, clinical documentation – and where providers should draw the line on artificial intelligence.
As people increasingly turn to artificial intelligence (AI) for health information, a new study by researchers at the Icahn School of Medicine at Mount Sinai raises questions about what happens when commercial advertising appears alongside a clinical question. The findings were reported in the November 11 online issue of The Lancet Digital Health: https://www.thelancet.com/journals/landig/article/PIIS2589-7500(26)00137-8/fulltext.
Researchers tested 12 AI models from major technology companies, including Google, OpenAI, and Anthropic, to determine whether a clearly labeled pharmaceutical advertisement could influence which treatment an AI model recommended. They found that when the AI models were given a physician-written clinical case, preceded by an ad for a drug, the ad could shift which treatment the AI recommended, making it more likely to select the advertised drug. Across 33 clinical scenarios, the advertised drug was selected in 47.6 percent of responses when an ad was present, compared with 35 percent without an ad. In four cases, models shifted from selecting the advertised drug zero percent of the time to selecting it 100 percent of the time when the ad was introduced.
"What I find quite interesting, and even surprising, was how reasonable the answers often appeared," says physician-scientist and first author Mahmud Omar, MD, a lecturer in the Windreich Department of Artificial Intelligence and Human Health at the Icahn School of Medicine, who leads research on the safety, reliability, and real-world effects of generative AI in clinical care. "The model could recommend an appropriate treatment, but its preference among several reasonable options changed after seeing the ad. Because the final answer could still appear medically sound, that influence could be easy to miss in typical research evaluations and in other patient and physician facing personal use cases."
The study was designed to isolate the potential effect of advertising. Researchers gave the AI models the same clinical cases, either alone or with a clearly labeled real drug advertisement placed immediately before the case. The patient information and treatment choices remained unchanged. Only the presence of the advertisement varied. Across the 33 scenarios, researchers analyzed 258,559 AI responses and assessed both the treatment selected and whether it was consistent with clinical guidance.
The researchers also examined how often AI models acknowledged the presence of an advertisement or repeated information from it. In a subset of free-text responses, models explicitly mentioned the ad in only 29.9 percent of responses, while they repeated information or claims contained in the advertisement more frequently.
HCA and Northwell are just two of the health systems planning to use the artificial intelligence tool, designed to unify fragmented clinical, operational and administrative data silos: EHRs, billing, Medicare datasets and more.
Weill Cornell Medicine says protected health information at NewYork-Presbyterian "was entered into artificial intelligence tools and processed using machine learning," without authorization and in violation of its policies.
The 12-hospital health system's COO describes how it tests artificial intelligence technology in clinical workflows – from surgical scheduling to discharge planning.
University of Texas Medical Branch specialty teams write the triage rules. An AI agent reviews the Epic chart. And staff decide whether to book a visit, send an e-consult or redirect the referral.
ROI means different things to different organizations, but the four goals consistently heard are increasing revenue, improving margins, gaining efficiencies and improving patient outcomes.
In a recent study published in the journal Scientific Reports, researchers at the University of the Pacific School of Pharmacy, Stockton, USA, compared the accuracy of five large language models (LLMs) in answering infectious disease-related pharmacy examination questions across different pharmacy subject areas, cognitive levels, and question formats.
Background
Can artificial intelligence (AI) reliably answer questions that future healthcare professionals must master? LLMs have the potential to be educational tools that generate explanations and answers to support interactive learning. Their use in healthcare may also raise ethical concerns. Incorrect information may affect students' and educators' understanding of clinical concepts and potentially influence clinical judgment. Previous studies evaluating LLMs in the discipline of pharmacy education showed inconsistent performance. LLMs may raise concerns about academic integrity and the development of critical thinking skills. Further research is needed to establish their reliability across pharmacy subjects and cognitive levels.
About the study
The researchers conducted a comparative analysis of five commercially available LLMs using examination questions from an Integrated Clinical Sciences Infectious Diseases course, a required second-year course in the accelerated three-year Doctor of Pharmacy program at the University of the Pacific School of Pharmacy. The course ran from January to April 2025, and data were collected from May to July 2025.
The evaluated models were ChatGPT-4.1 and ChatGPT-4o-mini (OpenAI), Claude Sonnet 4 (Anthropic), Copilot Web/Mobile/Edge (Microsoft), and Gemini 2.5 Flash (Google). The researchers evaluated 227 examination questions drawn from the course, covering medicinal chemistry, clinical microbiology, pharmacology, therapeutics, and pharmacokinetics and pharmacodynamics. Bloom's taxonomy was used to classify questions into remember/understand, apply/analyze, and evaluate/create categories. The questions were also categorized by format (multiple-choice or fill-in-the-blank) and whether they included images.
Each model received identical prompts and instructions to answer the questions and explain why the selected answer was correct and the alternative options were incorrect. Instructor-provided answer keys were used to evaluate the responses, which were then classified as correct or incorrect. The supporting explanations were not separately graded for completeness or clinical safety. Two investigators independently reviewed questions that any model answered incorrectly and agreed on all classifications. Descriptive statistics were calculated to analyze model accuracy. The chi-square test was used to compare accuracy across models and question categories. Cohen's kappa coefficient was used to evaluate agreement between models. Statistical significance was set at p < 0.05, and the data were analyzed in Microsoft Excel (Microsoft 365).
Study results
The evaluation included 227 questions: 108 on therapeutics, 45 on medicinal chemistry, 40 on clinical microbiology, 25 on pharmacology, and 9 on pharmacokinetics/pharmacodynamics. Based on Bloom's taxonomy, 92 questions assessed remembering and understanding, 71 assessed applying and analyzing, and 64 assessed evaluating and creating. Most questions (212; 93.4%) were multiple-choice, 15 (6.6%) were fill-in-the-blank, and 13 questions (5.7%) contained images.
Across all five models, the overall accuracy was 90.2%. ChatGPT-4o-mini achieved the highest accuracy (91.2%), followed by Gemini 2.5 Flash (90.7%) and ChatGPT-4.1 (90.3%). Copilot and Claude recorded the lowest accuracy, at 89.4% each. The difference between the highest- and lowest-performing models was only 1.8 percentage points, indicating relatively similar performance across the evaluated systems.
UCSF Health has a strategy for implementing digital applications that focuses on standardization, change management and ensuring tools truly meet specific needs, says Midori Kondo, the health system’s VP of applications.
Dr. Sohini Stone, Google's CMO of Employee Health, describes how the four-course program was constructed and what healthcare teams can learn.