The system is similar to how human vision works by discarding irrelevant or redundant information, allowing the ONN to quickly sort out important information.
Cornell University researchers have developed an optical neural network (ONN) that can significantly reduce the size and processing time of image sensors. By filtering out irrelevant information before a camera detects the visual image, the ONN pre-processor can achieve compression ratios of up to 800-to-1, equivalent to compressing a 1,600-pixel input to just four pixels. This is one step closer to replicating the efficiency of human sight.
How the new optical neural network works
The ONN works by processing light through a series of matrix-vector multiplications to compress data to the minimum size needed. The system is similar to how human vision works by discarding irrelevant or redundant information, allowing the ONN to quickly sort out important information, yielding a compressed representation of the original data. The ONN also offers potential energy savings over traditional digital systems, which save images and then send them to a digital electronic processor that extracts information.
The researchers tested the optical neural network image sensor with machine-vision benchmarks, used it to classify cell images in flow cytometers, and demonstrated its ability to measure and identify objects in 3D scenes. They also tested reconstructing the original image using the data generated by ONN encoders that were trained only to classify the image. Although not perfect, this was an exciting result, as it suggests that with better training and improved models, the ONN could yield more accurate results.
Their work was presented in a paper titled “Image Sensing with Multilayer, Nonlinear Optical Neural Networks,” published in Nature Photonics.
See also: Using Photonic Neurons to Improve Neural Networks
What this could mean for future applications
ONNs have potential in situations where low-power sensing or computing is needed, such as in image sensing on satellites, where devices that use very little power are required. In such scenarios, the ability of ONNs to compress spatial information can be combined with the ability of event cameras to compress temporal information, as the latter is only triggered when the input signal changes.
