MIT Researchers Develop AI System That Defers to a Human Collaborator

Researchers at MIT have developed an artificial intelligence system that is able to understand when to defer a task to an expert, adapting to the collaborator’s availability and level of expertise.

A lot of AI systems use this collaborative approach, in which an automated service works in most cases, while a human is brought in for edge problems.

SEE ALSO: Challenges of Adopting AI Medical Imaging

Facebook’s content moderation platform runs like this, using image and language recognition systems to automatically filter inappropriate content, while a large team of human moderators deal with more challenging content.

According to the team, the human-AI hybrid model performed eight percent better than either the human or AI could on their own. It is also able to reduce the computational cost and train the AI platform with fewer data samples, saving businesses time and money.

“Our algorithms allow you to optimize for whatever choice you want, whether that’s the specific prediction accuracy or the cost of the expert’s time and effort,” said David Sontag, associate professor at MIT. “Moreover, by interpreting the learned rejector, the system provides insights into how experts make decisions, and in which settings AI may be more appropriate, or vice-versa.”

The system can be integrated into all sorts of hybrid models, for instance, radiologists can use it to analyze the majority of scans, reducing the examination time to a few minutes. “In medical environments where doctors don’t have many extra cycles, it’s not the best use of their time to have them look at every single data point from a given patient’s file,” said Sontag.

MIT has only tested the system with algorithms so far, but intends to start a trial with humans and AI collaborating in the near future.