Self-Driving Cars: How Close Are We Really? (without the hype)

For the last couple of years we have heard many companies and CEOs promising us fully autonomous cars in 2018 – 2020 [1, 2, 3, 4]. We are halfway through 2018 and, although they are definitely making progress, I do not see fully autonomous cars operating on a large scale in that time period. Of course they can argue that they have never promised it on a large scale, but they definitely made it sound like it was coming towards everyone’s driveway in the near future.

And so it is only normal that the single question I get asked the most as an autonomous car engineer/researcher is: “When are the autonomous cars going to be ready?”. Now to disappoint you immediately, I am not going to answer that question in this blog. However, I would like to give a brief overview of the trends we can see happening in this field. It is based on my experience in industry/academia and what I read and hear around me. Although I use references and examples, this is still a subjective blog describing my view for the interested reader. If you are looking for a more in-depth, technical discussion of the different techniques used in this field, I highly recommend reading this excellent blog post by Markus Wulfmeier.

Anyone following the news even slightly must have seen a crazy amount of companies all trying to nail the self-driving car problem. Although they claim to genuinely care about making the world a better place, they all aim for a share of the big pile of gold on offer autonomous cars become mainstream. I divide them in three categories, which I will describe in more detail below: 1) the established car manufacturers, 2) the tech giants, and 3) the new kids on the block, usually smaller start-ups

The established car manufacturers

The first group are the established car brands (BMW, Audi, Daimler, Toyota etc.) and their suppliers (Bosch, Continental etc.). These brands have a long-standing history of building reliable, luxurious cars, which are not only ways of transportation but also status symbols. Consequently, I assume that 9/10 people, if not more, prefer to own a BMW instead of a Google pod (can you even call that a car?).

However, these manufacturers are not used to managing enormous software projects where lots of the required techniques are still not mature and not even within their traditional field of expertise. They simply do not have the human and material resources necessary, in contrast to the major tech companies [5].

The brands realise this and because they are afraid to lose their core business they have started betting on multiple horses. They all have their internal research department (some have multiple in different places around the world) close to the actual development tracks, they establish labs / campuses / start-ups [6, 7, 8, 9] away from the bureaucracy and traditional thinking of the company to speed up long-term development, and they team up with smaller start-ups or the bigger tech companies (more on that later).

It should be noted that their (short-term) vision of car autonomy for customer cars is substantially different to that of the tech companies [10]. Their customers like to drive the cars themselves, they aren’t interested in building a robot car:

“We don’t want a car to be a device to carry you from A to B; we want you to love it, so we need to do things to make people love it”  — Thomas Müller, Head of Development Brake/Steering/Driver Assistance Systems at AUDI AG

Therefore, for now, they focus mainly on functions that can assist the driver in dangerous or “boring” situations like traffic jams. A good example of that is the traffic jam pilot in the new Audi A8:

However, they have started to admit that a future with fully autonomous robot taxis in the big cities is unpreventable. Consequently, they have started some collaborations to provide / design vehicles for these kind of services [11, 12, 13]. In fact, almost all collaborations mentioned in the next paragraph are pilot test of tech giants who want to set up such a taxi service and use traditional car manufacturers to provide the vehicles.

The tech companies

This group includes almost all major tech companies nowadays (Alphabet/Waymo, Apple, Baidu, NVIDIA, etc.). However, it started with a few enthusiastic people at Google, who had just won the DARPA Grand Challenge and were genuinely interested in making driving safer and improving our living environments.

Soon other tech giants realised that there is a lot of overlap between their expertise and the autonomous car problem. For instance, machine/deep learning techniques developed for image classification such as Google image search can be used for urban perception. Add the potential profit of this emerging market and “the self-driving car race” was born.

In contrast to the car manufacturers, their vision is not so much an autonomous car but more an autonomous pod. A capsule without steering wheel and gas pedal, so that the human error is completely taken out of the system for maximal safety. However, this seems to be a big leap for the general public and, more importantly, the tech giants lack the required knowledge to build high-end automotive production lines. This is not something trivial as shown by the problems Tesla is experiencing with the Model 3 [14].

Because of these two reasons, we currently see a lot of collaborations between the traditional car brands and tech companies [15, 16, 17, 18, 19, 20, 21, 22]. This is natural; both companies do what they do best! I foresee this also for the future of customer cars, in which you would buy, for example, an Audi with Waymo autonomous software. Maybe customers might even get the choice out of various autonomous driving packets at different price points from different (tech) suppliers with more functions as the price increases.

These collaborations will also lead to the end of the taxi driver. In bigger cities, we will see fleets of autonomous taxis ready to pick you up when you call for one with your smartphone. Besides the tech giants we also see taxi services like Uber [23] and Lyft [24, 25] currently investing a lot of resources to get this up and running in some US cities. They have the advantage that they already have the user base and just need the fleet.

The start-ups

It almost feels like there is a new contender in this category every day. Most of these start as a small group of (academic) people who are leading in their respective field. Their expertise is a specific part of the autonomous car pipeline, for instance, mapping, localisation, machine/deep learning, or hardware like LiDAR scanners. Some of them have grown considerably and expanded their capabilities to the full pipeline.

In the end, I don’t see these growing to a size at which they are able to compete with the previous groups. The best performing ones will probably be bought by bigger players for their specific knowledge, as has already happened to some [26, 27, 28, 29], or become a part supplier. Because the bigger companies in most cases already have an autonomous pipeline in place, one could question whether these start-ups aren’t better off focussing their (research) efforts on one particular aspect of autonomous driving instead of the full pipeline.

So, how close are we really?

Before I try to answer this question, I want to stress that high ways and urban environments are two completely different worlds seen through the eyes of a self-driving car.

The highway setting is a simplified case of urban driving. All vehicles move in the same direction at approximately the same speed, there are no crossings only entries and exist, and we don’t have to account for vulnerable traffic participants. Because the environment is constrained, this is the easiest place to start the autonomous revolution. And so did Tesla… its Autopilot is developed solely for this scenario. In contrast to what the general audience believes, the system is not really clever. In essence it’s not much more than a smart cruise control: a combination of lane and distance keeping. Something that the German car manufacturers are also capable of. However, their nature is more conservative and so they haven’t brought it to the public yet, probably for good reasons…[30] It needs to be mentioned that Tesla has just promised an update with fully autonomous features for their cars to come out next month, but it is still not clear which functions are going to be included [31].

Urban environments are a different story. Currently, there are no commercial systems on sale that provide autonomous driving capabilities for these environments. However, lots of companies, including Waymo, are running tests in suburbs and cities, sometimes open to the public [32]. Their cars are more advanced and function autonomously around vulnerable traffic participants. Does this mean that Waymo is the closest to releasing self-driving cars anytime soon? It is hard to say… All companies have their own test programs in different environments (high-way, suburbs, busy inner city), which makes it hard to compare. The best comparison data we have for now are the disengagement reports [33] and a comprehensive study by Navigant [34]. In those, Waymo and GM seem to crank up the most miles and perform the best.