Skip to Content

An Ambitious Plan to Build a Self-Driving Borg

Mobileye wants competing carmakers to contribute on-the-road data to help teach automated cars how to drive safely.
October 10, 2016

Self-driving cars might fill the roads a lot sooner if carmakers can put aside their rivalries and share the data that would teach computers how to drive safely.

Mobileye, an Israeli company that supplies advanced computer hardware and software to many carmakers to enable cars to spot objects on the road, is now developing ways to train cars to drive themselves. The effort involves feeding computers huge quantities of driving behavior into a vast, highly realistic simulation, so that they can learn how to drive for themselves. And Mobileye aims to have different customers contribute the data that their vehicles collect.

“If you want to leverage many, many cars, you need to leverage as many carmakers as possible,” says Amnon Shashua, cofounder and CTO of Mobileye. Shashua says his company has devised a solution that will let carmakers contribute while retaining control over their data.

Mobileye’s technology is at the heart of many systems currently being developed by carmakers, so its plans will affect how self-driving systems emerge. It is also a critical time for the technology, as carmakers struggle to move from experimental systems to highly reliable commercial ones. Many experimental self-driving vehicles follow rules that have been programmed manually, and it can be difficult to account for every possible eventuality. Mobileye’s approach represents a new direction. Shashua says Mobileye will announce several deals before the end of the year, the first with Volkswagen.

Mobileye has been at the center of a controversy over the limits of vehicle automation in recent months. Its vision technology is used in Tesla’s Autopilot system, which was involved in a fatal accident in Florida currently being investigated by the National Highway Transportation Safety Administration (see “Tesla Crash Will Shape the Future of Automated Cars”). Tesla apparently programmed automated driving behavior into its vehicles using a picture of the road captured from the Mobileye system. Tesla and Mobileye parted ways after the carmaker implied that the vision system was at fault when its vehicle, operating in Autopilot mode, crashed into a truck turning across the highway. Mobileye fought back, saying that it had raised concerns about using the vision system to enable semi-automated driving.

Many of Mobileye’s existing products make use of deep learning networks trained to recognize visual information accurately. These networks are fed images that have been annotated by hand, and have been used to build vehicle systems capable of recognizing road signs or tracking the vehicle in front in order to maintain a safe distance. To enable full automation, the company plans to train networks using driving behavior with an approach called reinforcement learning, which involves network experimentation and reinforcing positive results (in this case, driving safely). Reinforcement learning can be used to train a computer to do something that would be difficult to program, and it promises to make it easier to account for all the different scenarios a car might encounter on the road.

The biggest issue will be persuading car companies to work together. “It makes a ton of sense for car companies to share data, particularly for a problem like this where a vast amount of diverse data is required.” says Karl Iagnaema, CEO of a startup called nuTonomy, which is testing automated taxis in Singapore. “Typically, however, leaders are unwilling to pool resources, for fear of diluting their advantage. It makes sharing of resources difficult.”

Since it isn’t practical to have cars learn on the real road, Mobileye has developed realistic simulations using real-world data it has collected. Inside these simulations, computer algorithms can experiment with different ways to navigate traffic situations. The plan is for different carmakers to feed driving behavior data from sensors into this shared learning network.

The idea of using computer simulations for vehicle training is certainly gaining popularity among researchers (see “Self-Driving Cars Can Learn a Lot by Playing Grand Theft Auto”). Shashua says he hopes that the simulation platform his company is developing could serve as the gold standard for testing and verifying self-driving algorithms.

Keep Reading

Most Popular

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

OpenAI teases an amazing new generative video model called Sora

The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.

Google’s Gemini is now in everything. Here’s how you can try it out.

Gmail, Docs, and more will now come with Gemini baked in. But Europeans will have to wait before they can download the app.

This baby with a head camera helped teach an AI how kids learn language

A neural network trained on the experiences of a single young child managed to learn one of the core components of language: how to match words to the objects they represent.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.