What Evil Lurks In That AI Algorithm; Worse Too For Autonomous Cars

Dr. Lance Eliot, AI Insider

There can be hidden biases inside an AI algorithm

I was online the other day and trying to figure out if I could afford an expensive item that I was eyeing to buy. The affordability partially depended on aspects such as credit worthiness and other financial factors. There was a system capability that claimed it could aid me in determining whether I would be able to purchase the vaunted item. After a few seconds of the system clicking and whirling, it came back and essentially said no.

I pondered this aspect since it seemed counter-intuitive to what I had thought the system would say (I was expecting a yes).

Turns out that there was nothing available to get an explanation of why it had said no. You would think it might at least offer some facets about the fact that I recently had bought a yacht and a Learjet (well, not really, but you get my drift), and so maybe I was over-extended on my credit. Nope. There was no provision for providing any kind of explanation. For all I know, the system had used a random number generator.

When I called the toll-free number of the company, they explained to me that they take into account at least a zillion factors. I said, OK, tell me how those zillion factors played out in my case. The operator told me they couldn’t possibly go through all zillion factors with me, it was too voluminous. I said, OK, tell me about just one of the factors, any single factor that the operator might pluck out of the zillion used. The operator then said they weren’t able to tell me about any of the factors because the method used is proprietary and they can’t reveal their secret formulas. Seems like the conversation should have started there, but I suppose the script they are trained on tries to avoid using the “we can’t tell you” response.

I have written many times about the lack of algorithmic transparency that we are increasingly witnessing throughout society.

There are secret algorithms that underlie the kinds of decisions such as my aforementioned story about wanting to buy a new item. Some software developers in a backroom put together an algorithm, possibly based on specifications derived from analysts in their firm, and the algorithm lo and behold becomes the ultimate decision maker.