Amazed by Real-time Applications? Thank the Computer Gaming Industry

There are many serious applications in the real-time realm, including financial transactions, system maintenance, public safety, and robotics. However, many of today’s advanced applications owe their capabilities to a fun pastime: Gaming.

“In gaming, real-time capability is essential; it forms the basis of everything we create,” related Teiji Yutaka, distinguished fellow at Sony Interactive Entertainment and creator of the PlayStation. Technology originally developed for gaming is being transferred to other fields, he explained in a recent profile published by Sony. “For example, in virtual production, the camera must be synchronized with the displayed background in real time, which directly leverages real-time computer gaming technology horned in gaming,” he said. “Applying realistic effects like real-time ray tracing to other industries can lead to even greater advancements.”

Lately, the combination of simulation and artificial intelligence has been advancing within gaming environments — which also has great promise for business applications. “Simulation relies on computational power to derive answers through deduction, while AI uses induction, applying knowledge and data from the past to predict outcomes,” said Yutaka. “By combining these approaches, we can take a new path: progressing calculations partially and using AI’s inductive reasoning to reach final conclusions.”

See also: Is Real-Time Even Enough Anymore for Consumers? Welcome to the Age of ‘Smartcuts’

Gaming and real time go hand-in-hand

Computer gaming as we know it today moved into the real-time realm as Yutaka and his team embedded more graphics processing into chips. Now, these gaming systems employ real-time graphics, real-time ray tracing, and user interface technology. “Games are essentially content that generates scenes and behaviors in real-time,” Yutaka explained. “With the evolution of computational power, from the original PlayStation through to PlayStation 5 and beyond, the increasing processing capability allows for more extensive simulations, bringing us closer to true realism.”

In real-time ray tracing, a “ray” refers to a light beam, he continued. “By processing the light source and object properties, we render 3D graphics on a display. Traditionally, textures on polygons were processed individually to determine colors. In ray tracing, countless rays are simulated by tracing them from the light source, calculating how light bounces, and determining how it reaches the viewer’s eye.”

Moving forward, “if computational power increases by tens of thousands of times from current levels, real-time 3D graphics could achieve visual representations indistinguishable from reality to the human eye,” he predicted.

AI also will play a role in this real-time realism. Image degradation is a challenge, and AI may address this, Yutaka said. “For instance, when the number of rays is limited, images can exhibit noise similar to photographs taken in low light. AI-driven denoising techniques have been developed to address this issue.”

Of course, with all types of technology, replicating human characteristics remains a challenge. “Accurately rendering individual strands of hair, subtle movements, and nuanced facial expressions still require improvement,” he said. “Since humans play a crucial role in games, even slight unnaturalness can trigger the uncanny valley phenomenon, leading to discomfort and detracting from the experience. Achieving realistic human representation and enabling natural control are among the primary challenges we face today.”