Complex real-time data integration needed for smart motorways is now a reality thanks to data frameworks like Apache Spark and can be delivered in a cost-effective manner.
In April 2023, the UK decided to scrap building new smart motorways due to safety issues and high costs.
Smart motorways are a type of highway that uses technology to improve the flow of traffic and reduce congestion (RAC). Their purpose is mainly to allow a smooth flow of traffic and regulate congestion through the use of variable speed limits and lane control, and often using the hard shoulder as an additional lane. They are the most advanced “managed” roads in the country. Unfortunately, these motorways are highly unpopular with the drivers and were linked to a number of tragic accidents. Instead of being easy to use, they are confusing, stressful, and downright misleading places at times. And peppered with vicious speed cameras.
Misleading (or total lack of information) is one major annoyance. I have personally witnessed several times an overhead sign suddenly flashing 40Mph, followed by an empty stretch of the road with no further speed restrictions or hazards in sight. Drivers abruptly hit the brakes, which catches other drivers behind unawares, leading to some hairy moments. The worst thing is there is no explanation whatsoever from the system operator.
Another one I see often is the display that says, “debris on road” or “crew on road” with the speed/lane restrictions but with no sight of either the work crew or debris anywhere. It’s just not good enough for our information-rich day and age.
The biggest challenge with smart motorways is that they fall well short of their full potential, perhaps hampered by the vision, technology, and finances. This a classic example of a great concept poorly executed. And now we got to the point where the entire notion of a smart motorway is being tossed to the side.
See also: Artificial Intelligence to Drive Safer Highways Program
Rethinking smart motorways
Could smart motorways ever live up to their potential? Or were they always doomed to failure? What if we made them “smarter”? Let’s consider what a “smart” motorway should really be.
A smart motorway is truly dynamic and “aware” of what is happening at every stretch of its length. It gives the users real-time information about what is happening ahead and around them, clearly warns of impending hazards, and dynamically manages speed limits (and lane restrictions) based on solid, live data.
As a minimum, the system should combine data feeds from satellites, road sensors (radar, laser, thermal, motion, etc.), cameras, road works activities, emergency services, weather, and roadside services in real time, run it through AI algorithms and provide relevant inputs to the operations control (OC). The OC dynamically manages the traffic flows using real-time insights and AI-assisted interpretation of the data patterns. Plus, they pass the information to the drivers well in advance using, initially, mainly the existing infrastructure.
That’s just today’s basic functionality enriched with real-time analytical capabilities that massively improves the quality of road information and decision-making by the OC and the motorway users. At this basic stage, the control center will be able to use the AI to interpret live data and update the vehicles with live traffic conditions in advance, greatly enhancing the quality of information available from just the satnav and driver’s eyes.
For example, if there is a localized severe weather event about to take place ten miles ahead, the AI could warn the drivers of the impending hazard before it manifests. The OC could introduce speed restrictions, lane closures, or diversions dynamically while keeping the drivers informed. This approach enables the so-called dynamic decisioning to take place, where the motorway control center and road users gain the ability to act upon the information in real time.
See also: The Ultimate Mobile Devices: Cars Connected at the Edge
Real time is key to smart motorways
The benefits of dynamic decisioning in operational settings are considerable. Just think about how much time and money can be saved by forecasting the hazards and congestion more accurately, giving the drivers real-time updates on causes, duration, and whether a diversion is a better alternative to slow progress. Speed limits can now be more accurately controlled by using real-time traffic information.
The next step in smartening up our motorways will make the roads and vehicles “talk” to each other automatically, in real-time. Think about it, we have “smart” motorways, and we have “smart” cars, but they are only connected to each other through the tires and driver. There is no automatic information exchange between the two (or car-to-car for that reason). The driver only sees what’s in their immediate vicinity, plus the occasional traffic updates from the satnav/radio, often after you are already stuck in a jam. The OC can only assess what the traffic does through cameras and the occasional sensor. There is no vehicle-to-vehicle or vehicle-to-infrastructure data exchange taking place at all (exceptions are the ride-sharing companies communicating within their fleets).
The motor industry is compensating for poor infrastructure by heading in the direction of ever smarter vehicles (semi-autonomous driving, driver assists, alertness monitoring, satnavs, over-the-air software updates, monitoring of componentry, office functions, etc.), packing them with sophisticated tech. My last car used thermal optics and AI to warn me of people and animals on the road at night – I loved it, but talk about overkill!
As clever as modern cars are, they are still some ways off from being driverless. Fully autonomous driving is infinitely more complex, requiring an array of advanced onboard sensors, over-the-air data feeds, and huge computing power to make instantaneous decisions. If we continue on the current path of tech evolution, the cars will need to become several degrees “cleverer” before they can be safely let loose on public roads.
Combining streaming data with real-time data
But what if the motorways “met” the automotive industry halfway by smartening up the infrastructure? Vehicles could easily stream telemetry data to the road receivers on a localized basis, in real-time. It does not have to be the full gamut of data, just core (anonymized) parameters of the vehicle type, speed, and direction of travel. And because the streaming data is localized, the transmitters and receivers can be low-powered, making them cost effective to install and operate.
This streaming data will then be combined in real-time with the other road users’ data in the vicinity and enriched with the external sources mentioned above to establish the exact vehicle position on the road and in relation to the others. If, perhaps, not full autonomy right away, it would accelerate the implementation by reducing the number of onboard sensors and computations that each car must perform. We could start with safe autonomous driving on the motorways.
To handle that level of data exchange, we will need to upgrade the OC to allow it to handle dynamic information in an automated fashion. There would be multiple satellites and automated OCs that focus on certain stretches of the motorways to keep them cost effective. A central OC would coordinate the traffic on a system-wide level.
With such a level of automation built in, we can take this concept even further. If we enrich the data exchange with the travel destination feeds, the traffic management system can automatically determine what drivers are better off diverting (and why!), thus reducing the number of vehicles in the congested area. Motorway congestion can be more precisely controlled by allowing vehicles access during peak times based on dynamically shared satnav itineraries, the severity of live traffic, and even paid “peak time” access tokens. If the system is aware of the road conditions in real time, it can make optimization decisions dynamically for all traffic in the area and keep the drivers updated with the latest information. In driverless cars, the system could issue automatic divert commands to direct them away. Such advanced traffic control measures will alleviate both traffic congestion and local pollution.
In 2022, drivers wasted 80 hours in traffic jams across the UK), costing the individuals $926 in lost time (not even taking lost fuel into account). Scaled up to roughly 23.5m who use roads to commute (RAC), the total comes to $24bn. That’s per year. If we could reduce this number by 10-20%, it would go a long way to helping the economy, let alone massively optimizing the use of our road network.
Smart motorways in action
Next step. Now that our motorways are evolving into much smarter ones, even more functionality can be added. Since the OC processes traffic and related information in real-time and is aware of the live location of all vehicles in its area of control, we could start to reduce the number of accidents by applying predictive analytics on the live data. The system could identify erratic and dangerous driving patterns and alert the culprits and other users in the immediate vicinity. Thermal and motion sensors could identify animal (people) incursions, sending alerts to all the road users in the area.
When we have breakdowns, accidents, or other hazards occurring, the system will issue immediate alerts to all drivers in the vicinity and automatically dispatch emergency services and recovery trucks as required. The control center can clear the emergency lanes by notifying specific vehicles on the road ahead of time so the services can get there faster.
See how much more useful the smart motorways could be? But let’s take this concept further still. Because the information is in real time, the system can help law enforcement services track stolen vehicles faster, more precisely, and safely. It can determine if truck loads are above permitted limits by using AI to analyze the axle loads and vehicle behavior while in motion. It would check if vehicles lacked MoT and insurance, drivers were using mobile phones, or even littering – a big scourge of our roadside!
The road surfaces can be closely monitored and repaired faster, reducing the roadworks disruption to a minimum. Motorway toll charging can be implemented based on exact distances traveled and perhaps even replace some of the duties we pay today – likely wishful thinking on my part. I am sure there can be more functionality added over time (inc. revenue-generating services). And the technology can be scaled out to cover the A-roads and city centers in the future.
The smart motorway infrastructure has hardly evolved over the past half a century. Sure, we have got control centers, better cameras, and number plate recognition systems, but generally, it’s the same as it used to be for quite a long while now – slow to react, expensive to run, and poor value to the road users. It’s baffling how little has been done to make our motorways and roads truly clever over the decades.
From an infrastructure point of view, smart motorways should be top of the government agenda, especially considering the state of our aging rail network. As long as we have road vehicles that travel on terra firma, we will be using motorways. And as cars, trucks, buses, etc., are becoming more sophisticated, so must the roads.
It is always viewed as incredibly expensive to build and upgrade motorway infrastructure. And it is if it’s done as big bang projects covering every little aspect of the upgrades. Instead, we should reuse a lot of the existing point systems and augment them with the latest in sensor, communications, cloud, and AI analytics technologies, all underpinned by advanced real-time data integration solutions.
Once the initial setup is done and tested, the system can be enhanced, and the legacy tech phased out. The overall architecture needs to be properly designed (the federated approach fits very well here), but the implementation should be on a modular, agile basis.
Real-time data challenges
Viewing the challenge from the real-time data integration perspective, we must build dynamic, low latency, unifying data capabilities that would work across the federated point solutions space and provide real-time dynamic decisioning. To achieve that, the data needs to be production-grade and interoperable from the get-go. Since the data flows in real time, we need to be able to trust it from source to destination. Decisions are made live – there is no questioning the quality and reliability of the data.
This means multiple quality-ensuring processes must be applied to the streaming data also in real time, such as data lineage at the record level, error checking, schema drift management, and automatic processing of late-arriving data. The integration must be able to address transmission gaps, late-arriving data, and data drifts on-the-fly. On top of that, we need to perform complex transformations and data enrichment from other data sources (streaming and batch) also in-flight.
There will be thousands of data pipelines in this project, connecting various point systems, sensors, vehicle data, data lakes, data warehouses, and apps. All must conform to production-grade standards.
Today we have powerful data frameworks like Apache Spark that allow the creation and management of efficient dataflows in real-time. Complex real-time data integration is now a reality and can be delivered in a cost-effective manner. That’s one big win for the smart motorways upgrade project. Hardware will evolve, new apps will be developed, and new digital capabilities will come to the fore, but a cleverly architected data integration solution will seamlessly accommodate such advances for years to come.
The UK government says it is prioritizing technology development. Smart motorways provide a perfect setting for bringing together an array of innovative ideas and companies to create something really special. The initiative will provide many years of skilled employment and allow our technology sector to develop even faster. Once we have successfully deployed the smart motorways, we can export the technology and know-how to other nations. It could be a massive win for the nation. All it needs is a will to do it!
