Autonomous Cars Having A Rough Time Doing Off-Roading

Dr. Lance Eliot, AI Insider

Image for post
Image for post

Green laning. Mudding. Rock crawling. Dune bashing.

These are all various kinds of off-road driving experiences.

For those of you that happen to own or use an Off-Road Vehicle (ORV), I’m sure that you are smiling right now as I mention such aspects such as green laning, which involves driving an ORV on forest trails in a greenery environment, or perhaps you are more akin to doing dune bashing, which involves rolling up and down sand dunes at the beach.

Rock crawling can be particularly scary as you inch your ORV over the top of rocks and sometimes try to scale large-scale rock formations, doing so at steep and rather precarious angles. I

n contrast, mudding usually involves splashing and slipping and sliding your way through gooey mud. The mud wants to cling to every pore of the ORV and can even blind the driver by coating the windshield with a nearly impenetrably thick layer of mishmash dank mud.

Type Of Vehicle For Off-Roading

Do you have to use a specially equipped vehicle to do off-roading?

No, you don’t, and it all depends on what kind of off-road circumstance you are facing.

If you want to get or craft an ORV, there are lots of auto makers that provide consumer-oriented trucks, jeeps, and SUV’s that are built for doing off-roading.

Some ORV’s are classified as RTV’s (Road Taxed Vehicles).

An RTV is considered street-legal and able to drive on our everyday public roads, along with then shifting into an off-roading mode once the situation presents itself. Or, it could be that you opt to get an ORV that is not legally able to traverse on conventional roads, and if so you are likely to have a CCV (Cross Country Vehicle). The beauty for some off-roaders of having a CCV is that you can add or subtract automotive components and do so without having to be worried that you’ll get a ticket for driving it on normal streets. These kinds of off-roaders will trick out their ORV with an anything-goes mindset.

Formally Defining Off-Roading

What does constitute going off-road?

For some people, if their GPS cannot map an area, they would say they are off-road. Have you ever driven to your local mall and gotten into the middle of the mall parking lot, and then asked your GPS to give you directions to get out of the mall? At times, the GPS won’t have any map to refer to and will simply tell you to first find your own way to an identifiable road. The mall itself is considered “off-road” in that the GPS has no idea about the nature of the smaller streets and weaving paths within the confines of the mall.

There are various trail ratings that are used to gauge how difficult an off-road situation might be. This helps for those that are purposely going off-road to designated areas and they then can anticipate how easy or hard the off-road location might be. This also is handy for doing competitions because you can then say that you succeeded in a really easy off-road or a really hard off-road situation.

I’ll offer this scale for purposes of trying to ascertain an off-road difficulty factor:

0: Not off-road (an on-road situation and without any off-road aspects)

1: Borderline off-road (mainly on-road with some easily navigated off-road)

2: Mild off-road (off-road but likely ok for a conventional car)

3: Arduous off-road (off-road and unlikely for a conventional car)

4: Grueling off-road (requires an ORV and somewhat taxing)

5: Severe off-road (ORV stretched to its limits)

6: Unnavigable off-road (not navigable by a ground-based vehicle)

AI Autonomous Cars And Off-Roading Aspects

What does this have to do with AI self-driving driverless autonomous cars?

At the Cybernetic AI Self-Driving Car Institute, we are developing AI software for self-driving cars. One crucial aspect relates to off-roading.

According to the official Levels of self-driving cars, a Level 5 self-driving car does not necessarily need to be able to handle off-road driving.

As per the SAE standard, a Level 5 self-driving car is supposed to be able to drive the car for all driver-manageable on-road conditions.

Notice that there are then two important caveats.

One caveat is that the driving circumstances must be driver-manageable.

The other caveat is that the driving situations apply only to on-road and not off-road occasions.

We can argue about what is a driver-manageable circumstance, and likewise we can argue about what is off-road versus on-road.

Let’s consider separately at first the driver-manageable aspects. We’ll then consider the off-road versus on-road aspects. And, I’ll then intertwine the two elements since they can indeed intersect with each other.

A driver-manageable situation involves a driving task that presumably a human could drive. If a human could not drive the car, it might be “unfair” to expect that the AI can somehow magically drive the car. For example, my car is pinned in, sandwiched among other cars in a parking lot. As a human, no matter what I do, I cannot drive my car out of the situation. I’m stuck. We could not reasonably expect that the AI can drive the car out of the situation either. It is not a driver-manageable circumstance, whether by human or by automation.

The grey area of being driver-manageable though involves the nature of the driver. You might not be able to drive a car out of some mud, while someone else more experienced at driving in muddy situations might be able to do so.

Next, let’s consider the notion of Level 5 not being required to deal with off-road situations.

What is an on-road versus an off-road circumstance?

I don’t think you can say that off-road is only when you are driving an ORV. As my earlier examples illustrate, you can get yourself into an off-road driving situation and not be in a vehicle that was intended for off-roading per se. A conventional car can just as readily be involved in off-roading. Therefore, you cannot dictate that the type of car ergo means you are off-roading (plus, of course, you might be using an ORV while driving on normal highways, therefore you are using it for both on-road driving and off-road driving).

You cannot claim that off-roading is solely when you are in a remote location.

Sometimes, off-roading can come to you, such as if a hurricane strikes your neighborhood.

For some pundits, off-roading involves going outside the bounds of your GPS. To them, when being even in a mall parking lot, and if your GPS cannot provide a map of your location, it is off-roading because the GPS can no longer provide navigation instructions about the car for you.

This is partially why there is a huge effort afoot to try and digitally map every square inch of populated areas. For an AI self-driving car, if it has no map to showcase where to go, in some respects that AI becomes unable to navigate the car. Sure, the AI can still drive the car, but where is it intended to drive to? There is a kind of push toward the Holy Grail of having detailed maps of everywhere that you might want to go, which then will make life easier for the AI of the self-driving car.

Having AI system such as robots navigate an unmapped area and gradually figure out as much as it can about the area is a well-known and much studied problem in computer science, and it is more commonly referred to as SLAM, which is an acronym for Simultaneous Localization And Mapping. We will gradually see this same kind of technique used by AI self-driving cars that find themselves in unmapped areas.

Basis For the Carve-Out Of Off-Roading

I think you can guess why the Level 5 includes the carve-out that the topmost level only involves on-road situations.

For on-road driving, presumably you would have a map that informs the AI about where you are driving and it would be coupled to a GPS. For off-road driving, there is less likely to be any map available of where to go. It is a harder driving situation.

Also, off-roading tends to involve tricky driving.

When I refer to driving as being tricky, this dovetails us into the other aspect of the Level 5 about being driver-manageable. Someone that drives their car each day to work on paved roads might be quite taken aback if you asked them to try and drive across rocks and boulders in the desert. There is a skill involved in handling off-road driving.

We are faced then with situations that are off-road and challenging, and also circumstances where there is a question if the driving is driver-manageable or not. Often, an off-road situation is also one that tends to be driver-manageable challenging. Obviously, you can have on-road situations that are also driver-manageable challenging, but I’d say it is fair to suggest that more-often-than-not an off-road situation will likely be also driver-manageable challenging, more so than an on-road driving task.

Qualms About The Official Off-Roading Exclusion

I am doubtful though that consumers are going to be happy to discover that their vaunted AI self-driving car does not necessarily know how to cope with off-road driving situations.

It might be reasonable as an auto maker to say that your AI self-driving car cannot handle driving in some obscure unexplored outback location, but if it won’t get you out of a muddy parking lot at a movie theatre, I’d bet that people will howl about such a limitation.

Similarly, suppose you have your AI self-driving car parked at home, and a hurricane causes wanton destruction and damage to the streets of your neighborhood. If the AI of the self-driving car considers such a road to now be off-road, you are going to be stuck. Furthermore, since there might not be any driving controls provided in the Level 5 self-driving car, you cannot drive yourself out of the situation, though you might want to do so.

Another somewhat disturbing concern too about the levels of self-driving cars is that a carve-out for off-road situations even exists without some other level to include it. What are we to call an AI self-driving car that can-do off-road driving? You could say it is Level 5, but now we’ll have some Level 5 self-driving cars that can handle off-road driving and others that cannot. It would seem prudent to include another level, perhaps Level 6, which would include off-road driving.

As an aside, if you want to know about another argument on this topic of driver-manageable, I’ve so far suggested that the definition of driver-manageable encompasses only if a human could have driven the car in the matter at-hand. I also raised the point that human drivers vary in their driving skills, such as me versus the highway patrol officer that was able to get me out of a snow jam. Let’s pretend we decide that driver-manageable includes driving to the capability of the greatest possible driving that any human could drive. This then deals with the potential for having sub-par drivers, such as me and snow driving.

There are some pundits that suggest that the AI might be able to driver better than even the best of any human driver. Therefore, if we limit the definition of driver-manageable to only what the best human can drive, we are not allowing for the possibility that an AI driving system might eclipse human driving capabilities. As such, in theory, we need to posit that the driver-manageable definition encompasses not only the best human driver, but also includes whatever an AI driving system might be able to achieve that presumably could surpass the capabilities of humans.

There will also likely be AI systems developed for ORV’s. You might assume that if there is AI that can cope with an ORV, it would seem a simple matter to then download the AI into a non-off-road self-driving car and it could too handle going off-road. The problem there is that the ORV is bound to be designed and built for off-road purposes. It might have the high-clearance, the special tires, and other equipment that allows it to handle off-road situations that a normal non-off-road self-driving car could not.

Can Today’s On-Road Sensors Handle Off-Roading

This also brings up the aspects of the sensors on a self-driving car.

Today’s sensors that are being used on self-driving cars are quite limited in their capacity to detect the kinds of objects and ground surface novelties that you would find when going truly off-roading. We are likely to see advances in sensor technology that will make sensors more capable for that kind of detection. Will we see those advanced sensors be included in everyday self-driving cars?

Maybe yes, maybe not.

The cost might be high so instead those sensors might get included into a pricier ORV.

Ethical And Societal Implications of Autonomous Car Off-Roading

One question to consider is whether or not the AI self-driving car might be electronically prohibited from going into certain off-road locations.

You might have a human that wants to go into an off-road location for the spirit of doing so, but it could be an area deemed as not legally allowed for off-roading. In that case, it would be conceivable that the AI might have been notified that it is not to drive the car in such a locale. Today, if a human decides to sneakily drive in a closed-off or banned off-road area, they can pretty much try to do so and maybe not get caught. In the case of an AI self-driving car, presumably the AI itself might be notified that it is not to take the vehicle into such areas.

We’ll need to see how the ethics of this plays out, and whether there are regulatory restrictions that might arise.

Speaking of human occupants, there’s another angle to this off-roading topic that is worthy of attention, namely the role of human passengers when an AI self-driving car is off-roading.

If there are human occupants inside an AI self-driving car when it goes off-roading, should the AI be conversing with those human occupants? For example, the AI self-driving car comes up to a water fording predicament. The AI is unsure whether the vehicle can make it across the water. Maybe it can, maybe it cannot. There is uncertainty involved.

This raises all sorts of complications. Imagine that the only occupants are small children. They were put into the AI self-driving car by their parents and sent along, while the parents had perhaps no idea beforehand that a situation might arise involving the water fording possibility.

Another variant involves having an AI self-driving car that has no human occupants in it, meaning that at the time of going off-road, there is a chance that the self-driving car is merely trying to get to point B from some earlier point A. Or, it might have been commanded by a human that told the AI self-driving car to go ahead and do the dune bashing. Should an AI self-driving car be able to drive in off-road situations and not have any human occupants?

Machine Learning And Off-Roading

Another means to gradually figure out off-road situations would involve the use of Machine Learning (ML), and particularly deep learning and the use of artificial neural networks.

The results of the deep learning could then be fed back down into the AI self-driving cars of an entire fleet. This would allow other AI self-driving cars that had not yet experienced the particulars of the off-road locale to then potentially be able to do so.

Some might argue that going truly off-roading is something that humans would want to do as human drivers.

In other words, even if the AI can handle going off-road, the excitement of going off-roading can only be gleaned if you are sitting in the driver’s seat. As such, there might still be cars that allow for humans to takeover the controls, doing so in limited situations such as off-road competitions and the like. Others might insist that no humans should be able to drive a Level 5 self-driving car, regardless of the reason.

Can you get as much joy from being a passenger as you can being a driver?

Suppose the AI allows you to direct it while you are off-roading — would that be sufficient as a kind of “back-seat” driver rather than being directly at the driving controls?

Time will tell.


I’m reminded of the famous line by off-road driver Ivan Stewart that there needs to be a sense of style in how you off-road and he coined the phrase CAR (Comfortable, Accurate, Relaxed).

He had said that if you have a cup of coffee in a cup holder and it is spilling over or in danger of falling out, you are not doing off-road driving to its pinnacle. Merely navigating a muddy road or climbing a rock is not sufficient, you also need to do it with style and grace.

I suppose we need to first get the AI to be able to cope with off-roading, and once we’ve gotten that far, we can then aim for the stylishness elements.

I assure you though that we are going to need AI that can cope with going off-road. It might be the little kinds of off-roads of life like my movie theatre muddy parking lot, or it might be seeking gold at an abandoned mine in the middle of a desert.

Off-road, it’s a thing to be dealt with. Let’s step-up to the rocky challenge.

For free podcast of this story, visit:

The podcasts are also available on Spotify, iTunes, iHeartRadio, etc.

More info about AI self-driving cars, see:

To follow Lance Eliot on Twitter: @LanceEliot

For his blog, see:

For his Medium blog, see:

For Dr. Eliot’s books, see:

Copyright © 2019 Dr. Lance B. Eliot

Written by

Dr. Lance B. Eliot is a renowned global expert on AI, Stanford Fellow at Stanford University, was a professor at USC, headed an AI Lab, top exec at a major VC.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store