AI Self-Driving Cars Have To Cope With Our Crumbling Roadways And Not Assume A Perfect Infrastructure

Dr. Lance Eliot, AI Insider

Image for post
Image for post

[Ed. Note: For reader’s interested in Dr. Eliot’s ongoing business analyses about the advent of self-driving cars, see his online Forbes column:]

I recently busted one of my tires when I hit a pothole on a major highway.

Potholes like that beast tend to get worse over time.

More and more cars fall into it or ram it or roll over it, all of which causes the hole to widen and deepen.

It was fortunate that I didn’t lose control of my car, and I wonder if other drivers would be so lucky.

You might be thinking that I must be a “bad driver” since I failed to avoid hitting the pothole, but I assure that I essentially had no choice. There were other cars next to me, the pothole was not readily seen until the last minute, and so on.

Generally, I would say that the “safest” approach was to go ahead and bite-the-bullet and hit the pothole, hoping that I’d be able to retain control of the vehicle.

Of course, there are a lot more potholes than the one that shredded my tire.

One concern expressed by those that study our crumbling roadway infrastructure is that we seem to be mired in a continual mode of quick repair.

This fix-and-forget kind of approach is belied by the fact that often times a repair is made that lasts only a short time.

The American Society of Civil Engineers recently published a report that says there are around 57% of the roads in Los Angeles that can be rated at a poor condition. By poor condition, they are asserting that those roads are in significant deterioration, are well-below roadway standards, and have a strong risk of overall failure.

Those of you that aren’t here in California are probably not especially sympathetic to our roadway plight in that you likely have something going on where you live that has a similar gloomy roadway dilemma, perhaps even worse than our roads.

Here’s a big number for you: $4.6 trillion dollars.

That’s how much the American Society of Civil Engineers estimates is the cumulative price needed to make our U.S. transportation infrastructure into something of an above average grade (right now, they say that the U.S. is maybe a D+).

My story about the pothole is really a microcosm of our overall roadway infrastructure.

Our economy depends upon our ability to drive on the roads. One could say that our society depends on our ability to drive on the roads. Our elaborate and crisscrossing roadway infrastructure is the essence of how we live.

It is easy to take it for granted.

Autonomous Cars And Foul Infrastructure

What does this have to do with AI self-driving driverless autonomous cars?

At the Cybernetic AI Self-Driving Car Institute, we are developing AI software for self-driving cars. One aspect involves making sure that the AI can handle driving on rough roads and contend with our deteriorating roadway infrastructure.

We’ll start with the assumption that the roads will continue for the foreseeable future to deteriorate and it will be a sad and unavoidable fact of life.

As such, what should AI developers be doing in terms of the AI for self-driving cars?

Some AI developers tell me that there’s nothing special they need to do. The road is the road. Good or bad, there’s presumably no need to care. Just focus on having the AI be able to drive a car and that’s sufficient, in their book.

I tend to disagree with their head-in-the-sand approach.

We believe that the AI ought to be specially prepared for a likely lousy infrastructure that contains roadway potholes, pits, cracks, debris, and for which the painted lines on the roads will be faded or disappear, and that street signs might be obscured or missing, etc. These are all the potential and inevitable consequences if there is not something Herculean done to improve the infrastructure.

One aspect that catches the attention of the AI developers that don’t seem to believe in caring about the untoward infrastructure involves my mentioning the faded or disappearance of lane markers and lane lines.

The AI system uses the camera sensors to try and detect where those lane markers and lane lines are. Then, once so detected, the AI guides the controls of the self-driving car to stay within those lines when traveling in a lane, and also for purposes of changing lanes. It is essential that this AI approach must have available relatively obvious and clear-cut lane markings. Without the lane markings, the AI system is pretty much unable to discern where a lane is and where to keep the self-driving car while moving along on the roads.

Human drivers of course also depend upon the lane markers and lane lines, but they are also able to handle a great deal of ambiguity when the lane indications are slim or intermittent.

So, the point is that the traditional AI technique of relying on apparent lane markings and lane lines is likely to get undermined as the roadways worsen. It is crucial to bump-up the AI to be more sophisticated in ascertaining lane positioning. If we don’t boost the AI for this, the vaunted hope of having less fatalities due to the advent of AI self-driving cars will be called more so into question.

Another aspect about the AI system is that it needs to be using all of its capabilities to try and detect roadway issues and obstacles, which will be even more so crucial as the crumbling infrastructure continues to degrade.

Example About Potholes

Let’s use my pothole example.

As mentioned, I was unable to detect the pothole prior to making the right turn at the corner of the downtown street.

Could the AI have done a better job?

I’m not so sure it could have in this circumstance.

There are other questions then that arise.

Would the AI be able to quickly enough counter the physics of the lurch caused by the hitting of the pothole?

Would it be able to correct for the shove that the self-driving car got by rolling into and over the pothole?

Even if it was able to detect the pothole in-advance of hitting it, would the AI be able to appropriately identify the alternatives such as swerving over or trying to come to a halt and assess the risks associated with those alternatives, thus making a “reasoned” selection of what to do?

These are serious questions regarding the driving capability of the AI.

AI Development Mindset

I suppose some AI developers would assert that the AI has to be ready for potholes all the time anyway, and there isn’t a special case involved in dealing with these roadway evils. Though this is partially true, it also belies the idea that with a crumbling infrastructure the pothole is going to no longer be a rare event of an edge case nature and will instead be a probable and frequent encounter.

The AI might need to cope with having to drive down any given street and be dodging a large crack in the street there, and a pothole over here, and then another pothole a few feet to the left, and maybe debris chopped out of a pothole by a prior car that hit the hole.

I tend to refer to this as the AI dodgeball mode.

The AI needs to be able to play a kind of dodgeball game of maneuvering in and around the various obstacles and roadway problems.

I doubt that most AI developers have considered ensuring that the AI can handle this somewhat repeated and continual effort of lots of dodges to be strung together, doing so while keeping the self-driving car safely on the road and not hit any other cars or nearby pedestrians.

In essence, the usual assumption is that the self-driving car will encounter one anomaly, the AI will be able to deal with it distinctly, and then if another anomaly appears it will be completely later in time, considered a separate occurrence and fully independent of the first encounter.

The reality is that a lot of the roads are likely to be a morass of deterioration on a given road, often due to the heavy traffic on that particular road.

One aspect that might help AI self-driving cars to contend with banged-up roads is the use of V2V (vehicle-to-vehicle) electronic communications.

With the use of V2V, one AI self-driving car could tell another AI self-driving car to do those kinds of things.

Presumably, in an orderly fashion, one AI helps another AI. Each self-driving car that follows the next would be forewarned about the pothole. This would also allow those AI self-driving cars to act in concert with each other, often referred to as a swarm, allowing each to avoid the pothole by making timed and coordinated maneuvers.

There is also the likely advent of V2I (vehicle-to-infrastructure) electronic communication.

Part of the reason that the roadway infrastructure might hasten to deteriorate could partially be due to the advent of AI self-driving cars.

You might be shocked to think that the AI self-driving car emergence could somehow worsen the roadway infrastructure, since the AI is supposed to be a polite driver that obeys the laws and tries to drive as cleanly and legally as possible (I’ve debunked those assumptions, by the way!).

The reason that the advent of AI self-driving cars will likely exasperate the crumbling infrastructure is due to the belief that we’ll want to use the AI self-driving cars non-stop.

It is anticipated that AI self-driving cars will be used extensively for ridesharing purposes. You are at work during the day and allow your AI self-driving car to be making money for you while you are at the office.

The odds are that we are going to see a beehive of activity of self-driving cars cruising around night and day, waiting to pick-up and drop-off passengers.

This continual driving is going to put more miles onto our already destitute roadways. More miles on falling apart roads means those roads will continue to fall apart.

We can predict it will make those roads a lot worse. The constant pounding of self-driving car after self-driving car is a punishment that a crumbling infrastructure will not be able to readily withstand.

I suppose one potential good news is that the AI self-driving cars will hopefully make use of Machine Learning (ML) and be able to therefore increasingly get better at detecting lousy roads and sufficient driving on lousy roads. I had mentioned earlier the V2V of AI self-driving cars sharing with each other.

Another form of sharing will be via OTA (Over-The-Air) electronic communication.

OTA consists of the AI self-driving car providing to the cloud of the automaker or tech firm the data that the AI self-driving car is collecting while driving on the roads. This would include the camera data, video data, radar data, and so on. At the cloud level, the auto maker or tech firm can do analyses and try to use ML and deep learning to improve how the AI self-driving cars operate. These improvements can be pushed back down into the AI self-driving car, providing updates or patches for when something amiss in the software needs to be upgraded or fixed.

Thus, the AI self-driving cars in the fleet would not only be aware of the existence of the pothole, but also have some driving tactics and strategies to contend with it.

Doing Something About The Infrastructure

Depending upon the status of AI self-driving cars at the juncture of moving forward on improving the infrastructure, we could use the data from the AI self-driving cars to better understand where the crumbling infrastructure is most occurring. Keep in mind that the AI self-driving cars will have their myriad of sensors and will be crisscrossing the roads and continually capturing visual images, radar, LIDAR, etc.

This is a huge amount of data that can be used to mine when trying to prioritize where to put our energies and money on infrastructure improvements. This data can reveal which roads are most traveled and which are least traveled. It can reveal the roughness of the roads. There are a slew of handy analyses and metrics that can be discerned from this vast collection of data.

Another factor involves whether or not to merely fix the infrastructure as though we will continue to have only conventional cars, or whether to consider doing other kinds of improvements or upgrades to the infrastructure that tie into the advent of AI self-driving cars.

For example, I had mentioned herein the use of edge computing, which will be a boon to AI self-driving cars. Perhaps the crumbling infrastructure can be enhanced by the adoption of edge computing.

We will need a provision for dealing with AI self-driving cars that breakdown.

I realize that some pundits claim that AI self-driving cars will never breakdown, but this is crazy talk. A car is a car. There will be lots of reasons for an AI self-driving car to breakdown, including as previously pointed out that they will be trying to run non-stop 24×7. We’ll need to contend with the towing of broken-down AI self-driving cars, another topic that I’ve covered in my presentations and writings, and for which the infrastructure can be shaped to aid toward appropriately handling these situations.


With the existing roadway infrastructure that is falling apart at the seams, we need to be ready for the advent of AI self-driving cars. It would be a shame to have AI self-driving cars that cannot readily use what might be impassable roads by the time that the AI is ready to hit the roads. Think of the irony that we might have in-hand self-driving cars, but they cannot go anywhere because of the marred roads. Or, we might put AI self-driving cars onto the roads, and their working for us non-stop causes the roads to hasten in crumbling.

One aspect involves making sure that the AI is savvy enough to be able to deal with the lousy infrastructure. There is though only so much that the AI can do in this regard. It would be like having all human drivers have to learn to drive gingerly so as to not unduly upset the roads. Better still would be to fix the infrastructure.

Fixing it means not just making what already exists passable, it also means that we would want to perform upgrades and improvements that dovetail with the emergence of AI self-driving cars. The motto often heard of “fix the darned roads” should be augmented by the clamor to “tech-up the roads” so that we’ll have a synergistic effect of good tech-savvy roads that coincide with the prevalence of AI self-driving cars.

Come to think of it, I’m going to have some signs made-up that say this and stand at the pothole tomorrow to alert my fellow mankind of what we need to do next.

Wave at me and honk your horn in support, would you please?

For free podcast of this story, visit:

The podcasts are also available on Spotify, iTunes, iHeartRadio, etc.

More info about AI self-driving cars, see:

To follow Lance Eliot on Twitter:

For his blog, see:

For his AI Trends blog, see:

For his Medium blog, see:

For Dr. Eliot’s books, see:

Copyright © 2019 Dr. Lance B. Eliot

Written by

Dr. Lance B. Eliot is a renowned global expert on AI, Stanford Fellow at Stanford University, was a professor at USC, headed an AI Lab, top exec at a major VC.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store