Scaling Edge AI: A Call for Hardware, Data and Talent

Artificial intelligence (AI) holds a lot of promise for edge devices, bringing real-time analysis power to the places it is needed, be it factory floor sensors, vehicles, or smartphones. And it neatly addresses the current issues associated with running AI algorithms and data centralized data centers or clouds, which consume exorbitant amounts of energy, incur latency issues, and may be vulnerable to security breaches. But edge devices, with their small footprints and low power, are too constrained to support AI at this time.

There are ways to address these concerns and get AI out to the edges, and these were explored by the CTOs from 22 leading microelectronics companies at SEMI’s Chief Technology Officer Forum. In a recent report, Dr. Pushkar Apte and Tom Salmon provide a summary of the meeting, along with recommendations.

Participants at the forum included representatives of Accenture, Advanced Micro Devices, Advantest, Applied Materials, Brewer Science, Dell Technologies, Dow/Dupont, EMD Materials, Entegris, Galaxy Semiconductor, GridMatrix, HPE, Imec, Lam Research, Microchip, Qualcomm, Quantum Semiconductor, Resilinc, Soitec, Teradyne, Tokyo Electron and Veeco. Guidance was provided by Dr. Evgeni Gousev from Qualcomm and Professor Boris Murmann from Stanford University.

Much effort is still required across the industry for practical implementation to realize its full potential, they caution. “It is a huge challenge to pack significant computing and analytical capability in a tiny, inexpensive device with a severely limited power budget. The following pieces of advice came out of the conference:

Develop system-level hardware: Attendees agreed that “edge AI development requires holistic thinking, co-optimizing the entire hardware-software-system stack,” Apte and Salmon relate. “The hardware must be specifically designed for energy efficiency and to operate with milliwatt power budgets. This contrasts with the current approach of using hardware that is just good enough.” In addition, “since these edge devices will be ubiquitous, it is also extremely important to design security in the devices and eliminate ethical biases in algorithms upfront.”

Introduce smart data management: Efficient data management is necessary “since both the compute capability and energy budgets of edge devices are severely limited,” conference leaders stated. “Edge AI algorithms and architectures require sparse or compressed data sets.” Then there is the matter of the quality of data used for model training, which also needs to be improved.  

Develop an ecosystem: Edge artificial intelligence requires a robust ecosystem. An example of progress in this space is tinyML, a not-for-profit organization that promotes small-footprint AI technology that encompasses “machine learning architectures, techniques, tools, and approaches capable of performing on-device analytics for a variety of sensing modalities (vision, audio, motion, chemical, etc.) at the milliwatt (or below) power range, targeting predominately battery-operated devices.”

Enable supply chains: Then there is the matter of getting the right raw materials, and later, manufactured components to where they are needed. Edge AI technologies “require full-stack optimization, including supply-chain elements like packaging and materials,” Apte and Salmon relate. “AI system costs can be mitigated by using innovative packaging techniques like heterogeneous integration and 2.5D or 3D – in place of expensive system-on-chip (SoC) solutions.”

Inspire and develop talent: The final and most important piece in developing an edge AI presence is attracting talented people to design and build such components. “For the short term, the industry must invest in training the existing workforce to become savvy in AI data techniques,” Apte and Salmon explain. For the long term, “the industry must inspire students by focusing on applications for social good and sustainability. The industry must expand its search for talent beyond the usual suspects already in the technology world – it must make a special effort to reach out to underserved communities in high schools and middle schools.”