MGH Researchers Use Artificial Intelligence for COVID-19 Prognosis Prediction

Researchers at Massachusetts General Hospital, an affiliate of Harvard Medical School, have created a new artificial intelligence-based tool for predicting the prognosis of potential COVID-19 patients.

Led by Harvard Medical School professors Michael B. Westover, Gregory K. Robbins, and Shibani S. Mukerji, the team designed and implemented a machine learning model based on outpatient data from the respiratory illness clinics and emergency department at Mass General. Their findings were published in The Journal of Infectious Diseases, a peer-reviewed publication, on Oct. 24.

The model, COVID-19 Acuity Score, outputs a figure which care providers can use to determine whether an individual patient has a high chance of developing complications that would require hospitalization or time in an intensive care unit.

The data used to train the model came from 9,381 adult outpatients seen between March 7 and May 2, and the researchers used data from another 2,205 patients seen between May 3 and May 14 as a prospective validation group.

The validation group, which consists of patients in each of the model’s four output categories — hospitalization, critical illness, death, and no hospitalization — is key to evaluating the model’s performance, according to Haoqi Sun, a Medical School neurology instructor and the study’s first author.

“We really want to look into the future to make sure that model is robust for future data and future patients,” Sun said.

This large patient sample for training and validation also distinguishes the tool, according to Mukerji. She said many other models use data from as few as 200 patients.

“It really matters because you have to have a wide variety of individuals coming through because COVID itself is heterogeneous,” Mukerji said. “People's background histories are heterogeneous and so if you don't have that modeled within some sort of learning, then you're just learning on small datasets.”

The project began near the very start of the pandemic, when clinicians across the world, including those at Massachusetts General Hospital, faced an entirely unfamiliar respiratory virus with unknown risk factors.

“The basic problem was very simple,” Mukerji said. “We were starting to see patients in an outpatient setting, where the clinical providers did not know whether or not that person was going to be at risk, in a couple of days time, to go to the hospital.”

As the pandemic progressed and Boston suffered from a first wave in late April and early May, non-primary care and infectious disease physicians began staffing the emergency clinics due to the influx of potential COVID-19 patients, Mukerji said.

“They're different providers, like neurologists, who may not really see a lot of respiratory viral patients and never know when to pull the trigger to go to the hospital,” Mukerji said. “You have a problem on the back end, providers providing patients with their results, and trying to figure out, do I send them to the hospital — and our hospitals are getting full — who's the right person to send it?”

“Wouldn't it be nice if there were something that we could use to help stratify individuals? Can we use some other tool to help our clinical guidance, you know, help our clinical gut to say this person is at risk,” she added.

By focusing on outpatients, Sun said the model does not require lab test values, making it applicable in settings with walk-in patients like urgent care or emergency departments.

“There are many inpatient models which utilize these lab values,” Sun said. “They have very good predictions, very accurate predictions, but this is not usually available.”

Mukerji also said using data from hospitalized patients only fails to accurately reflect the situation in clinics and emergency departments, where many patients may not need to be hospitalized.

“This is now taking a step back and saying, ‘You know what, as an outpatient provider, I don't know if the person in front of me should be hospitalized. Should I hospitalize them? Am I scared enough?’” Mukerji said. “ There's so much unknown and so, you don't want to be sending everybody to the emergency room.

—Staff writer Virginia L. Ma can be reached at virginia.ma@thecrimson.com.