AI systems, through training, have become highly adept at recognizing photos and their contexts. However, videos, with their moving images and frames, have been beyond the reach of AI-based analysis.
Researchers at Scripps Research Institute have come up with an approach they may enable AI to effectively conduct real-time moving images from video. The institute announced development of an AI model “that can watch and understand moving images with the subtlety of a human brain.”
The project, called MovieNet, processes videos “much like how our brains interpret real-life scenes as they unfold over time,” they explained in a recent article in the Proceedings of the National Academy of Sciences, covered in Medical and Life Sciences News.
The MovieNet model “can perceive moving scenes by simulating brain cells make real-time sense of the world,” the report states. The human brain, they note, does not process still scenes — it infers from a flowing series of images. “Conventional AI excels at recognizing still images, but MovieNet introduces a method for machine-learning models to recognize complex, changing scenes.”
The applications may serve a range of use cases, from medical diagnostics to autonomous driving, “where discerning subtle changes over time is crucial.”
See also: How Image Annotation is Leading the Way in ML and AI
To mimic the brain’s abilities, the researchers studied tadpoles, of all things, which have superior visual capabilities. They “identified neurons that respond to movie-like features — such as shifts in brightness and image rotation — and can recognize objects as they move and change. Located in the brain’s visual processing region known as the optic tectum, these neurons assemble parts of a moving image into a coherent sequence.”
MovieNet scored higher than conventional AI models in recognizing changing scenes, the researchers claimed. “By breaking down visual information into essential sequences, MovieNet effectively compresses data like a zipped file that retains critical details.”
This represents another step in the ability to deploy AI in applications or systems requiring situational awareness.
