How a Different AI Approach Can Cut the Queue

Instead of treating patients on a first-come, first-served basis, the U.S. healthcare system typically relies on a triage process whereby clinicians assign patients a preliminary diagnosis to decide where to place them in the queue.    

Doctors want to make sure they treat the most-urgent cases first; a collapsed lung needs to be addressed more quickly than, say, bronchitis. At the same time, hospitals also want to reduce overall wait times and costs whenever possible to keep waiting rooms clear and patients and staff satisfied.   

In recent years, hospitals have increasingly turned to AI to assist them with this process. They’ve sought out algorithms that can take over triage, ensuring urgent cases get seen first while having patients spend as little time as possible waiting.   

That’s the goal that Itai Gurvich and Jan Van Mieghem, both professors of operations at the Kellogg School, pursued in research along with Simrita Singh, a former Kellogg PhD student and now an assistant professor at Santa Clara University. Through theoretical modeling and a computational study, they compared two ways to use AI to triage patients: an algorithm that first diagnoses a patient’s “type” from the x-ray (using image recognition) and then uses that diagnosis to assign the patient a priority rank, versus an algorithm that directly decides a patient’s priority rank based only on the raw features of a patient’s x-ray.   

They found that the second, “direct model” was more effective at reducing wait times.   

“If our ultimate goal is to prioritize patients, then our objective is not necessarily accuracy [of diagnosis] but rather the cost of making the wrong patient wait too long,” Van Mieghem says.    

When tested on a dataset of chest x-rays, this process of bypassing an initial diagnosis when triaging patients significantly reduced the total cost of waiting (where the waiting cost of urgent cases is much higher than that of nonurgent cases).   

“There’s a benefit to thinking differently about the noisy information you get from the x-ray,” Gurvich says. “It reduces the waiting time for the most-urgent cases.”   

A clear winner   

The researchers first conducted a theoretical analysis of the optimal way to train a machine-learning algorithm to read patient chest x-rays and sort patients into queues based on how urgently they need to be treated. The team posited that there was a better way to get AI to effectively queue patients than the typical way of having AI triage patients based on a likely diagnosis. It meant solving the problem a different way than the typical process a human healthcare worker might use.   

If you train the model to focus on the cost of wait time instead of the accuracy of the diagnostic type, for example, “it’s going to spit out not the type of illness but the priority rank that should be given to that patient,” Van Mieghem says. “The question is, which approach is better?”   

To answer that question, the researchers created a mathematical model that puts patients directly into the queue based on features of their x-ray. This “direct-to-queue” approach was built to prioritize reducing the overall cost of waiting, and the approach factors in the idea that urgent cases should be seen first as well as the fluctuations in how busy an ER can be.    

They put this approach head-to-head with the more traditional, diagnosis-first (or “type-first”) approach and found that they performed equally well in only one specific scenario: when AI was able to accurately diagnose a patient’s disease type 100 percent of the time. In all other scenarios, the direct-to-queue approach was more effective at reducing the total cost of wait times.  

Given that it’s highly unlikely in the real world to get preliminary diagnoses correct every single time, even for AI, the direct-to-queue model generally performed better. 

Reducing waiting costs   

Then the researchers tested their theory on a dataset of 112,120 anonymized patient chest x-rays.   

They consulted with medical clinicians to rank the urgency of 13 different lung conditions, as well as the urgency of different features—the shading and shapes—that could appear on the x-rays. Then they used an AI image-classification system called MobileNet to analyze and sort the x-rays into queues.    

They trained it to do this using either a diagnosis-first model or a direct-to-queue model. In both cases, MobileNet sorted patient cases into four different queues based on urgency of treatment: critical (most urgent), urgent, important, and routine (least urgent).   

The researchers found that the direct-to-queue approach outperformed the diagnosis-first model in their simulated radiology setting, reducing total waiting costs across all patients in the dataset by more than 30 percent.     

“The difference between the approaches is that type-first has an intermediate step that says, ‘let’s cluster these images into a set of types [or diagnoses], and then we assign each type into a specific priority queue,’” Gurvich says. “We’re saying that, unless you have a perfect prediction of types, then you should go directly from the image to deciding which queue to put it in. We remove the intermediate step of predicting the type.”   

Everyday queues   

Hopefully, people don’t find themselves having to queue up to receive treatment after a chest x-ray. But there are lots of other kinds of queues into which people might get sorted where the same logic applies.  

Help centers at many organizations put customers into priority-based queues all the time, Van Mieghem says, and these findings could help them decide how best to sort those customers. Bank call centers, for instance, often decide how urgent a specific person’s needs are based on their history with the bank. But if the person has no past interactions with the bank for it to call on, then machine learning could make inferences from that person’s “features”—like their zip code—just as MobileNet made decisions based on the x-rays’ visual features.   

For AI engineers and developers, the findings emphasize the importance of focusing on an algorithm’s core objective when training the AI model. In the case of chest x-rays, it was to reduce the total cost of waiting and not necessarily to get an accurate preliminary diagnosis.   

“The operational context should inform the way you do machine learning,” Gurvich says.