Your Driverless Car Is A Snitch, Junkyard Tells All

Dr. Lance Eliot, AI Insider

Image for post
Image for post
Few realize that a self-driving car has a lot of private info, revealed in junkyards

My latest rental car that I picked-up at the Chicago O’Hare airport was a treasure trove of information about prior renters.

I could readily see data that had been imported into the car’s infotainment system that had come from at least six different smartphones. Lots of favored playlists of people that I didn’t know, but I now knew their taste in music.

Even scarier for those prior renters was that many of their contacts had also gotten transferred into the on-board systems of the car. Plus, via the built-in GPS tracking, I could see the specific locations and dates/times of where many of these prior travelers had gone while using the rental car. If I had been a nefarious person, it would have been possible to use all of this info in rather untoward ways. In my case, I was just curious to see what others had opted to leave behind.

Digital Artifacts Leftover in Cars

In a modern world, we leave behind not just physical artifacts but also digital artifacts.

It is easy to pair your smartphone to the infotainment and GPS systems of a rental car. Apparently, a lot of people don’t add together two-plus-two and realize that what goes in won’t necessarily come out. When you pair your smartphone, depending upon the nature of the settings, you might be allowing the car’s on-board systems to slurp-up all kinds of info out of your handy cellphone.

I suppose that there are some people that are unaware of the transfer of data that occurs. They live in their own non-digital world or are just part of the unwashed of the digital realm. For some of them, the smartphone and Bluetooth are already somewhat magical, I suppose.

There are likely other people that know that it happens, yet perhaps assume that it will miraculously be erased for them. Perhaps it’s like the Mission Impossible movies and the inputted data will sizzle and disappear after it is done with your travel journey. This transferred data will self-destruct in ten seconds, good luck, Jim.

Certainly, the rental car agency could include as part of their rental car clean-up checklist the step of them resetting and blanking out the on-board systems that might have collected your private info. This added step would be nice for the renting public. You would never need to worry about it again, assuming that the rental agency actually did the erasure properly, and consistently, and without making any errors or omissions.

FCC Provides a Warning About Digital Artifacts Leftovers

The Federal Communications Commission (FCC) has tried to forewarn people about the Bluetooth pairing dangers, including saying this: “If you connect your mobile phone to a rental car, the phone’s data may get shared with the car. Be sure to unpair your phone from the car and clear any personal data from the car before you return it. Take the same steps when selling a car that has Bluetooth” (this is stated at the FCC’s web site, https://www.fcc.gov/consumers/guides/how-protect-yourself-online).

Note that the FCC warning also mentions the notion of erasing your data when you sell your own personal car.

I’m betting that many people neglect to do so.

Here’s an added twist for you, what about when your car gets wrecked and it is hauled off to a salvage yard.

Would you be thinking about the data that you’ve left in your wrecked car?

Probably not.

What Happens to a Wrecked Car

If you are car has gotten so wrecked that it cannot be repaired, the odds are that you are mentally and physically done with that car.

You might idly wonder what happens to the carcass. Usually the totaled car is hauled off to a salvage yard. Some people call them junkyards, others refer to them as scrapyards. Any usable parts would be potentially resold onto the used parts market, or in some cases the scrapyard hangs onto the parts or to the carcass and allows prospective used-parts buyers to come and pick over the skeletons.

Some scrapyards remove the reusable parts and place them into a salvage warehouse, nearly arranged. More often, the scrapyards just pile up the “deceased” cars and allow car part seekers to roam around and find whatever they think they need.

The unusable elements could be turned into scrap that can be sold at bulk prices, especially scrap metal.

It is likely that about 75% of the average wrecked car can be put to some other use.

Have you ever had a car that was scrapped?

According to statistics by the federal government, there are about 15 million cars per year in the United States that end-up in a scrapyard. There are an estimated 250 million cars in the United States. Thus, as you can see, about 6% of the cars annually seem to go to scrapyards each year.

The Case of AI Self-Driving Cars

What does this have to do with AI self-driving driverless autonomous cars?

At the Cybernetic AI Self-Driving Car Institute, we are developing AI software for self-driving cars. One aspect that few of the auto makers or tech firms are considering is what will happen to the data that’s on-board an AI self-driving car once the self-driving car ends-up in a salvage yard.

I’d like to first clarify and introduce the notion that there are varying levels of AI self-driving cars. The topmost level is considered Level 5. A Level 5 self-driving car is one that is being driven by the AI and there is no human driver involved.

For self-driving cars less than a Level 5, there must be a human driver present in the car. The human driver is currently considered the responsible party for the acts of the car. The AI and the human driver are co-sharing the driving task.

Another key aspect of AI self-driving cars is that they will be driving on our roadways in the midst of human driven cars too.

Wrecked AI Self-Driving Cars Are a Data Treasure Trove

Returning to the topic of AI self-driving cars that end-up in a salvage yard, let’s consider why this might happen and what makes it different from a conventional car that is hauled into such a resting place.

First, the big reason that an AI self-driving car differs from a conventional car in terms of the salvage yard is that an AI self-driving car is chock full of sensors and computer processors.

A conventional car is likely to have a limited set of sensors, often not nearly as powerful and full-bodied as those that would be used on a true AI self-driving car. And, the computer processors in a true self-driving car need to be top-of-the-line, superfast to handle the AI running aspects, more so than the processors on a conventional car.

I am not saying that today’s modern conventional cars don’t have some semblance of sensors and processors. Instead, I am pointing out that on a Level 4 or Level 5 self-driving car, the odds are they are a step-up in terms of capabilities, along with often higher costs too, at least when purchased new.

Furthermore, the amount of on-board system memory is likely a lot more than you would have on a conventional car.

This is where the concern really focuses about having your wrecked AI self-driving car towed into a salvage yard. Remember my earlier story about car rentals that are turned-in and the renter has left personal data in the on-board systems? Magnify that kind of leftover info a thousand-fold, and you have the situation we are facing with AI self-driving cars.

An AI self-driving car is likely to have captured video streams that are left intact in the wrecked AI self-driving car. There is a treasure trove of telematic data about the activity of the self-driving car. There could be data that was transmitted back-and-forth via the OTA (Over-The-Air) electronic communications that might have taken place between your self-driving car and the cloud of the auto maker. There could be V2V (vehicle-to-vehicle) electronic communications stored in the on-board systems, involving your self-driving car communicating with other self-driving cars.

All of this then is in addition to whatever you might have placed into the self-driving car via your connected smartphone.

Things get even worse.

If your AI self-driving car has a voice activated Natural Language Processing (NLP) system that allows you to give verbal commands to the self-driving car, those might also be stored in the on-board systems. If the self-driving car is a Level 2 or Level 3, in which you co-shared the driving task, the odds are that there might be captured info about your driving and the driving aspects of the AI system.

Tesla Examples Found by Researchers

Let’s consider the Tesla cars.

According to Tesla’s owner manual, here’s the kind of Telematics info that could be kept on-board the car:

“To improve our vehicles and services for you, we may collect certain telematics data regarding the performance, usage, operation, and condition of your Tesla vehicle, including: vehicle identification number; speed information; odometer readings; battery use management information; battery charging history; electrical system functions; software version information; infotainment system data; safety-related data and camera images (including information regarding the vehicle’s SRS systems, braking and acceleration, security, e-brake, and accidents); short video clips of accidents; information regarding the use and operation of Autopilot, Summon, and other features; and other data to assist in identifying issues and analyzing the performance of the vehicle.” (source: https://www.tesla.com/about/legal).

Plus, this kind of data too:

“Data about accidents involving your Tesla vehicle (e.g., air bag deployment and other recent sensor data); data about remote services (e.g., remote lock/unlock, start/stop charge, and honk-the-horn commands); a data report to confirm that your vehicle is online together with information about the current software version and certain telematics data; vehicle connectivity information; data about any issues that could materially impair operation of your vehicle; data about any safety-critical issues; and data about each software and firmware update.”

In case you are thinking that this is merely an abstract problem and would not occur in the real-world, there is a fascinating study that was recently released about a computer security company that bought some wrecked Tesla cars at a salvage yard and examined those cars to see what they could find (for an article and a video of what they found, see: https://www.cnbc.com/2019/03/29/tesla-model-3-keeps-data-like-crash-videos-location-phone-contacts.html).

The researchers pored into four cars that they obtained, specifically a Tesla Model X, a Tesla Model S, and two of the Tesla Model 3 cars. Of course, they found paired data from smartphones. I’d say that’s pretty much to be expected of any modern-day car, and not especially surprising or unusual. This included nearly a dozen phonebooks of contact info, and various GPS navigation locations.

What’s more interesting is the aspect that for one of the Model 3’s, the researchers extracted the video of the Model 3 of when it had crashed. The car had veered off the road and crashed, which the front cameras recorded. Tying this to the GPS data, the researchers could ascertain the location, Orleans, Massachusetts, occurring on Manequoit Road, and the day and time of the crash. The airbags also deployed. They also tied the crash to the smartphone that was plugged into the car at the time, being able to figure out presumably the person driving the car.

They also looked at the log of the phone use and could see that a phone call from a family member (a contact in the database) had called the driver of the car, moments before the crash occurred.

I think we would all be rather shocked to find out that our private details could be so easily gleaned from our wrecked car. You would normally likely assume that those kinds of details would need to be gotten by a court order or a subpoena of some kind.

Also, you would likely assume that the data would be secured in some manner, making it hard for just anyone to retrieve. According to the researchers, by-and-large the data collected was unencrypted. There was no need to try and crack any difficult ciphers or codes.

I don’t want to seemingly be picking on Tesla, and it should be pointed out that the Tesla licensing does have some warnings about a salvaged Tesla, including this:

“An unsupported or salvaged vehicle is a vehicle that has been declared a total loss, commonly after extensive damage caused by a crash, flooding, fire, or similar hazard, and has been (or qualifies to be) registered and/or titled by its owner as a salvaged vehicle or its equivalent pursuant to local jurisdiction or industry practice. Salvage registration/titling typically can never be removed from the vehicle so that all future persons understand the condition and value of the vehicle. Tesla does not warrant the safety or operability of salvaged vehicles. Repairs performed to bring a salvaged vehicle back into service may not meet Tesla standards or specifications and that is why the vehicle is unsupported. Consequently, any failures, damages, or injuries occurring as a result of such repairs or continued operation of an unsupported vehicle are solely the responsibility of the vehicle owner” (source: https://www.tesla.com/about/legal).

In a manner of speaking, presumably it is the duty of the car owner to cope with the matter of having their own car salvaged and taking any needed steps.

According to the researchers, Tesla apparently reported to them that:

“Tesla already offers options that customers can use to protect personal data stored on their car, including a factory reset option for deleting personal data and restoring customized settings to factory defaults, and a Valet Mode for hiding personal data (among other functions) when giving their keys to a valet. That said, we are committed to finding and improving upon the right balance between technical vehicle needs and the privacy of our customers” (as stated in: https://www.cnbc.com/2019/03/29/tesla-model-3-keeps-data-like-crash-videos-location-phone-contacts.html).always

You can interpret the response by Tesla as befits your own views about what responsibility the car maker has versus the car owner.

Coping With AI Self-Driving Cars Once Wrecked

As the advent of AI self-driving cars continues to increase, there will be more and more circumstances involving wrecked AI self-driving cars.

Right now, the Teslas, which are considered pretty much a Level 2, those are the most prevalent of any semblance of an AI self-driving car and so it is logical that those would be getting wrecked, in the normal course of being on the roads, and end-up in salvage yards.

With the emergence of Level 3’s, once those become relatively popular, they will ultimately get into wrecks, sorry to say, but it’s a fact, because they are cars, and that’s what happens with cars, and so those too will eventually get piled into scrapyards.

The Level 4 and Level 5 self-driving cars are right now working in experimental modes and prove-of-concept (POC) modes, and are not owned by individuals per se. Instead, they are being crafted by auto makers and tech firms. This means those self-driving cars are lovingly tended by a slew of expert mechanics and AI professionals. If those self-driving cars get into a wreck, it isn’t as though they will just tow the self-driving cars to the nearest salvage yard and junk them there.

Nope. Those babies lead a pampered life, right now.

My point being that the auto makers and tech firms have not had to deal with the end-of-life aspects as yet of self-driving cars.

You might be tempted to suggest that at least the on-board data should always be encrypted. By doing so, it would mean that even if the wrecked self-driving car was given to a salvage yard, it would be arduous or perhaps infeasible for anyone to readily pluck the data out of the car in terms of knowing what the data actually contained (they might be able to grab it, but it would appear to be undecipherable).

Though this is a good idea, it also offers the downside of having to be continually encrypting and potentially decrypting data to make use of it to drive the self-driving car by the AI system.

This means that the on-board computer systems are going to do a lot of added computational work. The data being collected by the sensors would need to be turned from plaintext or plain-data into encrypted data. Would this happen only once the data is stored? That’s data in-rest or in-place. Would it also occur when the data is flowing throughout the on-board system, which is data-in-motion?

There are lots of questions to be considered. Would the added computational effort dilute the on-board computational processors and distract those processors from the “real work” of running the AI to drive the car? Would the time it takes to encrypt and decrypt create a potential delay in having the AI be able to readily make driving decisions, which are real-time and life-or-death kinds of matters?

It could be that some might argue that the salvage yards have an obligation to not allow the data from the towed-in self-driving cars to be handed out. Perhaps their should be legislation that requires salvage yards to protect your data and inform you about it. I doubt that many salvage yards will welcome such an added burden onto their shoulders.

You might say that it should be on the shoulders of the auto maker and tech firms that make the AI self-driving cars. That’s again something that has yet to be ascertained in terms of what the range and nature of their duties are. Much of this is still an open market approach and there is little yet in the way of regulatory rules about it.

I would guess that we’ll likely see lawsuits that will also arise due to these matters. Someone that has had a wrecked AI self-driving car that reveals private aspects will launch a lawsuit against the auto maker or tech firm, perhaps at the insurance firm, perhaps at the salvage yard, and maybe at anyone or anything in the life cycle steps after a self-driving car has gotten wrecked.

Things Will Get Worse In Terms of What’s On-Board

I’ll add more fuel to the fire.

It seems likely that true Level 5 AI self-driving cars will have cameras pointing inward and be recording the audio and video of whatever happens inside of the self-driving car. Why this kind of intrusion? It can be to help the AI figure out what the human passengers are doing and what they want the AI to do for them.

There’s another equally practical reason, namely for ridesharing purposes. Most would agree that the AI self-driving car of a Level 5 will be used for ridesharing purposes. Even if you own your own Level 5 self-driving car, you will likely let it roam and be a ridesharing vehicle while you are at work or asleep, allowing your self-driving car to make money for you.

By having the cameras that point inward, you can keep track of those pesky ridesharing passengers that might decide to trash the inside of your shiny AI self-driving car.

The overall point is that this kind of private data would also be presumably kept on-board the self-driving car. Once again, it might be accessed once the self-driving car is relegated to a salvage yard, if not otherwise protected or erased.

I’ll scare you about the outward facing cameras too.

As your AI self-driving car goes down the street in your neighborhood, it is capturing video, along with possibility audio, and radar, and LIDAR, and ultrasonic waves, which could be kept on-board the self-driving car. It might be sitting in there, a view of all of your neighbors, their dogs and cats, their comings and goings.

When you park your AI self-driving car in your garage, it might still be recording. This could occur in that the AI self-driving car might be setup to wait for you to ask it to do something, so it is sitting there in a semi-alert fashion. It is akin to Alexa or Siri, listening for a prompting word. Though in theory the listening mode is not recording, you never know how it might really have been established.

Some believe that the AI of the self-driving car will be a kind of therapist, allowing you or your children to interact with it on your daily commute. The AI might try to help you with that problem at work, or difficulties with your spouse. Or, your children might confide that they are failing in their classes and want to run away from home. All of this potentially could be recorded by the AI system.

This AI would use a mixture of NLP, socio-behavioral techniques, and possibly Machine Learning and Deep Learning. Whatever methods or technologies used, it all depends upon having data, including collecting it and keeping it around, in some manner, whether in whole or in a compressed or selected manner.

We really haven’t as yet established what the boundaries are going to be about the recording of such data.

I know some pundits claim that the voluminous data is so voluminous that it would not make any sense for the self-driving car to keep it on-board. The amount of on-board computer memory would be overly costly, use up too much power, and be large and heavy, weighing down the AI self-driving car. They say that this data either won’t be kept, or it will be shunted up to the cloud via the OTA.

We’ll have to wait and see how this plays out.

Conclusion

The next time you drive past a scrapyard, look at the pile of cars, and think to yourself about the hidden secrets that will someday be there, embedded into the computer memory of those AI self-driving cars that were unceremoniously dumped there.

Perhaps we’ll prevent that data dumpster treasure trove from happening, if we take heed now in the design and development of AI self-driving cars.

For free podcast of this story, visit: http://ai-selfdriving-cars.libsyn.com/website

The podcasts are also available on Spotify, iTunes, iHeartRadio, etc.

More info about AI self-driving cars, see: www.ai-selfdriving-cars.guru

To follow Lance Eliot on Twitter: @LanceEliot

For my Forbes.com blog, see: https://forbes.com/sites/lanceeliot/

Copyright © 2019 Dr. Lance B. Eliot

Written by

Dr. Lance B. Eliot is a renowned global expert on AI, Stanford Fellow at Stanford University, was a professor at USC, headed an AI Lab, top exec at a major VC.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store