Mixed Reality Testing Underpins Waabi’s Commercial Driverless Launch

A Waabi truck testing self-driving freight services on the open road.

Waabi

Self-driving truck developer Waabi has provided new insights into their AI-centered process to develop a competent robot driver, developed with very little exposure to real traffic compared to legacy competitors. According to the company, this approach allows for development of a robo-driver at much less cost within much less time. For Waabi, this means briskly getting from idea to product and to revenue – a Holy Grail for any startup.

Here are key points based on a recent wide-ranging dialogue with Raquel Urtasun, Waabi Co-Founder and CEO.

Waabi World

Early on, the company created Waabi World, According to Waabi, this is “a high-fidelity, closed-loop simulator which automatically generates all the situations a self-driving vehicle might encounter in the real world, and tests the entire autonomy system with near-zero domain gap. This means that the AI system under test should drive exactly the same way in simulation as it would in the real world when presented with the same situation. Furthermore, Waabi World is a super-intelligent engine that can efficiently create scenarios that, with high probability, make the system under test fail, as well as cover all the possible failures it might have.” The aim is to provide a radically new level of efficiency in the test and development phase.

“Waabi World also solves the data problem that plagues the broader AI industry by intelligently creating richer, more informative examples than real-world data, massively reducing training and testing times as well as minimizing costs,” said Ms. Urtasun. “It is a revolutionary system that has extraordinary positive consequences in the development and deployment of fully autonomous systems — well beyond trucking — dramatically reducing development risk and providing the first solution to perform a rigorous scientific safety case.”

While there are skeptics, Ms. Urtasun makes the bold claim that their approach is “provably safe.” “Before deployment in the physical world, fully autonomous systems need to be validated and verified, with robust scientific evidence forming the safety case. In the self-driving industry, in particular, there’s a need to go far beyond the current practice which is built on accumulating miles of real-world driving as a measure of deployment readiness. While the volume of these miles is in the millions, it is nowhere near what would be required to provide the rigorous evidence necessary for a comprehensive safety case.”

Waabi has developed a new generation of foundation models, which, as Ms. Urtasun puts it, “are uniquely capable of perceiving the world, creating interpretable abstractions, and then using these abstractions to reason in the same way humans do, in order to find the best possible action to perform. These models are the backbone of the company’s virtual driver — the Waabi Driver. Further, the system is capable of human-like reasoning, with abilities to generalize across the unknown, resulting in safer behavior on the road.”

Because Waabi World is capable of running in real time, Waabi engineers can create scenarios and run them on a truck which is moving at speed on a test track, i.e. Mixed Reality Testing. This takes the testing process to yet another stage in preparing for on-road driverless operations.

Mixed Reality Testing

Closed-course testing has been around since the beginning of the automotive age as a way of understanding and certifying performance of cars and trucks. AV developers have used test track testing extensively to validate capability.

Track testing is a useful tool to evaluate autonomy, but it has its limits. Creating specific traffic scenarios in a multi-lane track setting requires precise choreography across other vehicles and even mock pedestrians. To get sufficient data, testing requires that the same scenario be run multiple times, but all the moving parts make repeatability challenging. Useful and important data is gained, within these types of limits.

In the development process, the findings from track testing with an early software version needs to be re-validated when a new software package is released. This iterative process eats up time and money.

Most fundamentally, though, is lack of diversity. Scenarios encountered by drivers include other vehicles, pavement quality, debris on the road, pedestrians (alert or looking at their phone), traffic signals, signage, animal incursions, and weather variability which affects both perception and tire traction. It is impractical to physically test all permutations on a typical test track.

Early on, Ms. Urtasun concluded that the legacy track testing modality was inadequate for autonomous driving. “Despite the billions invested across the industry in these facilities, this testing modality cannot deliver the volume, diversity, and complexity of scenarios in the realistic and repeatable manner that comprehensive AV safety testing demands.”

Instead, “Waabi’s Mixed Reality Testing (MRT) offers a revolutionary alternative that completely transforms what’s possible on a closed-course track. MRT enables the Waabi Driver to drive autonomously down a physical test track while simultaneously experiencing numerous intelligent, simulated actors that coexist in this hybrid reality and react to each other and to the physical world in naturalistic ways,” she said.

All this is possible by leveraging Onboard Waabi World, a version of Waabi’s neural simulator. As Onboard Waabi World generates new scenarios, the real physical sensor readings are modified instantaneously so the Waabi Driver can react to the blend of real and virtual elements while driving in the physical world.

“This fusion creates a first-of-its-kind reality that unlocks unlimited testing possibilities previously impossible to achieve safely or practically,” Ms. Urtasun asserts.

A Dynamic, Intelligent Alternate Reality

The alternate reality is enabled by Waabi World’s sensor simulation capabilities, which modify the multimodal sensor data from the physical sensors mounted on the truck (LiDAR, cameras, etc.) in real-time. The modified sensor data flows directly into the full onboard software stack, causing the Waabi Driver to respond to virtual scenarios as if they were physically present on the track. The Waabi Driver thinks the mixed world is reality, triggering the same decision-making processes and responses as if they existed in the first place.

The entire process happens in a few milliseconds on the onboard compute.

With MRT, all the previous constraints of closed-course testing are eliminated. Waabi is able to conduct exponentially more tests than was previously possible.

Achieving Unprecedented Safety Testing at Scale

Waabi has been utilizing MRT as a central testing approach for more than two years. Ms. Urtasun said that MRT “has been a key innovation that has allowed Waabi to advance faster, safer and with unparalleled capital efficiency, enabling us to achieve feature-complete autonomous driving capabilities at the beginning of this year and paving the way to our imminent driverless launch,” adding that “this breakthrough allows us to build an AV we can truly trust and accelerates the path to safer roads for everyone.”

Are regulators and early freight customers convinced? Ms. Urtasun emphasized that the Waabi team has shared their approach and results in-depth with regulators and fleet customers. This is typical of what other autonomy developers are doing as well – while the detailed information is proprietary, it can be shared under non-disclosure agreements with key stakeholders.

But what about extreme weather? Can this be created with high fidelity within simulation? “We have not yet handled snow and hail as this is not currently in our Operational Design Domain. It is on our roadmap,” Ms. Urtasun said.

Just The Beginning

Wabbi has been conducting on-road testing with safety drivers for three years, in preparation for a commercial truck driverless launch later this year. Ms. Urtasun is “confident” about this timeline.

But this is just the start, she says. “For Waabi to expand to other use cases, such as warehouse vehicles and robotaxis, very little change will be required.” The long term commercial play is in selling software. As to the company’s revenue model, Ms. Urtasun said “we will be software providers to vehicle manufacturers, beyond our current partnership with Volvo Autonomous Solutions, enabling manufacturers to offer Driver-as-a-Service products. The Waabi driver can adapt to various sensor configurations according to an OEM’s needs.” She added that Waabi has no plans to conduct fleet operations to stimulate the market and earn early revenue, in contrast to the path being taken by some others in the truck autonomy space.

The Spark

Wrapping up our interview, I asked Ms. Urtasun to think back in time: when did the idea of autonomous vehicles first enter your mind? “Fifteen years ago, I was working at the Toyota Technology Institute at the University of Chicago,” she said. “They had projects aimed at reducing traffic crashes using intelligent systems. I was inspired by the opportunity to address traffic safety, in part because people close to me had been affected by road incidents. This struck me as a great way to apply technology.”

All the splashy awesomeness of autonomous driving can be dazzling. But, as we all know, the purpose of this tech is to improve our travels while avoiding the road tragedies that are still all too common.