A study led by NYU Langone Health researchers shows how an artificial intelligence tool helps leaders predict which patients will decline in the hospital or return soon after being discharged.
Credit: Getty Images/andreswd
A new artificial intelligence tool could help doctors and hospital administrators predict patients’ near-future health with enough accuracy to improve countless daily decisions that shape their care.
The tool is designed to analyze patient case details to help answer important questions that healthcare providers everywhere ask themselves each day, such as: Which patients are at risk of declining during a hospital stay. How long will that patient need a bed? Will insurance cover their procedures?
Led by NYU Langone Health researchers, a new study describes the design, testing, and deployment in real world settings of Lang1, an AI prediction engine trained on records from more than 10 million patients over 20 years of care.
Published online in Nature Medicine, the work shows that the AI tool can “read” clinical notes written about each patient by their doctors and is up to 24 percent better at predicting that person’s healthcare outcomes,” depending on what it is being asked, than the largest, most expensive, general AI models in the world. These include ChatGPT-4o, Meta’s Llama, and DeepSeek R1, a model 671 times Lang1’s size (based on how many parameters it considers as it predicts).
“With the advent and deployment of this model, we are talking about health systems going from guessing what each patient needs to knowing,” said corresponding study author Eric K. Oermann, MD, associate professor in the Departments of Neurosurgery and Radiology and director of the Health AI Research Lab at NYU Langone. “Further, and unlike with giant models, our smaller, better-trained tool is inexpensive enough to be widely adopted by hospitals.”
When Lang1 accurately identifies, for instance, which patients are at highest risk of dying in the hospital, care teams can escalate monitoring and initiate palliative care consultations. By predicting which patients will return to the hospital within 30 days of discharge, the tool signals early that a patient needs a longer stay or better discharge planning to avoid a sometimes dangerous and expensive return trip. While the tool will collect new intelligence, it is designed only to augment decisions made by human care providers.
Lesser-Known Foundation of Medicine
Physicians today spend just 26 percent of their time directly caring for patients, say the authors. The rest goes to dealing with documentation, insurance, scheduling, and administration. The researchers argue that this systemic problem translates into rushed doctor visits, long waits in emergency rooms, and denial of insurance claims. Operational functions are so difficult, they said, because hospital staff must digest huge volumes of unstructured doctors’ notes and predict patients’ near-future health to inform last-minute care decisions.
Each uninformed choice can trigger a wave of inefficiencies, said Dr. Oermann. Solutions require the pattern recognition that AI excels at, but existing AIs trained on generic internet text fail to understand the constraints faced by hospitals in the real world.
In the new study, Lang1 was nearly 24 percent better than models like DeepSeek R1 at predicting which patients would face an insurance denial. Lang1 was 9 percent better at predicting 30-day readmissions, but only 1.6 percent better at predicting who was at highest risk for in-hospital mortality. The latter makes sense, said the authors, because, unlike medical records related to insurance denial risk, mortality-related language appears frequently on the internet, enabling generalist models to better compete.
The specific study measure of predictive performance was a statistical concept called “area under the receiver operating curve,” the probability that Lang1, or any of the other tools, would effectively separate any two groups – for example, patients who will be readmitted versus those who will not be readmitted."
Extrapolating from the literature, the authors noted that the average cost to a hospital per readmission is about $15,000 per 30-day readmission and $14,000 per insurance denial. With the average hospital experiencing about 10,000 of each in a year, the authors predicted that, even if Lang1-guided decisions achieve only modest improvements, annual savings per hospital could approach $4 million.
The new tool, actually a family of AI models together called Lang1, builds on the research team’s prior model, NYUTron, which showed for the first time that an AI tool trained on clinical notes could make accurate predictions. Lang1 is nearly 70 times larger than NYUTron, is trained on a body of evidence more than 10 times larger and includes several other advances.
Specifically, the team trained Lang1 on 80 billion tokens (bits of words used by AI models) from notes in electronic health records—mixed with 627 billion tokens of general internet text from Wikipedia, books, academic papers, etc. Adding general internet training text did not hurt clinical task prediction based on electronic health records, but it did help the tool better “understand” the messy language in doctors’ notes.
A doctor’s note might describe a patient’s symptoms and treatments but will never conclude that “this patient will return to the hospital in 18 days,” said Dr. Oermann. To predict patient outcomes, Lang1 had to be shown millions of notes, comparing its own predictions with how patients actually fared in a process called supervised fine-tuning.
In addition, the research team built into the new tool called ReMedE, which measures more effectively than current approaches how well AI tools make predictions in real-life hospital conditions, said the authors. ReMedE examines any tool’s prediction accuracy in performing five critical “prediction tasks” on actual, operational outcomes like readmission, exposing potentially harmful gaps in prediction accuracy before any AI model is deployed.
NYU Langone is set to offer Lang1 and ReMedE to other hospitals as a confidential service, generating independent proof that the tool can be applied nationally. As part of the study, the team already validated their results at Boston’s Beth Israel.
“Our effort is not a demo but is instead a fully built system benchmarked against the best models to show how well it transfers across tasks and institutions,” said first study author Lavender Jiang, a doctoral student at the NYU Center for Data Science. “Specialized models represent the future of healthcare AI.”
Along with Dr. Oermann, authors from NYU Langone’s Department of Neurosurgery were Radhika Dua, Xu Han, Xujin Chris Liu, Frederick Wolff, Robert Steele, Anton Alyakin, Qingkai Pan, Yanbing Chen, Karl Sangwon, Daniel Alber, Jaden Stryker, and Jin Vivian Lee. Also, a first study author was Angelica Chen from the Courant Institute School of Mathematics, Computing, and Data Science at New York University, along with Kyunghyun Cho. Other NYU Langone authors were Kevin P. Eaton, MD, at NYU Grossman School of Medicine, Yindalon Aphinyanaphongs, MD, PhD, in the Departments of Medicine and Population Health, and Jeff Zhang in the Division of Applied AI Technologies.
Dr. Oermann is a consultant for Sofinnova Partners, a venture capital firm that specializes in the development of biotechnologies, sustainability, and pharmaceuticals. He also has equity in Artisight Inc., a company that produces a software for healthcare organization operations, and his spouse is employed by Eikon Therapeutics, which develops cancer therapies. The terms and conditions of these arrangements are being managed in accordance with the policies and practices of NYU Langone Health.
Funding for the study was provided by the National Cancer Institute’s Early Surgeon Scientist Program grant 3P30CA016087, National Science Foundation award 1922658, the W.M. Keck Foundation, an Apple AIML PhD fellowship, and by grants RS-2019-II190075 and RS-2024-00509279 to the Institute for Information & Communications Technology Promotion from the Korean government.
About NYU Langone Health
NYU Langone Health is a fully integrated health system that consistently achieves the best patient outcomes through a rigorous focus on quality that has resulted in some of the lowest mortality rates in the nation. Vizient Inc. has ranked NYU Langone No. 1 out of 118 comprehensive academic medical centers across the nation four years in a row, and it continues to have the most No. 1– and top 10–ranked specialties among medical centers in the United States, according to U.S. News & World Report. NYU Langone offers a comprehensive range of medical services with one high standard of care across seven inpatient locations, its Perlmutter Cancer Center, and more than 330 outpatient locations in the New York City area and Florida. The system also includes two tuition-free medical schools, in Manhattan and on Long Island, and a vast research enterprise.
