AI & Law: About OpenAI’s GPT-3

Consider the AI & Law aspects of the AI-powered auto-generative tool known as GPT-3

by Dr. Lance B. Eliot

For a free podcast of this article, visit this link https://ai-law.libsyn.com/website or find our AI & Law podcast series on Spotify, iTunes, iHeartRadio, plus on other audio services. For the latest trends about AI & Law, visit our website www.ai-law.legal

Key briefing points about this article:

  • GPT-3 is an AI-based software tool proffered by the for-profit firm OpenAI

Introduction

There is an abundance of attention going toward a recently released software tool called GPT-3 that embraces AI techniques and, in some ways, showcases the surprisingly adept capabilities of automatically generating natural language text including producing lengthy narratives of an impressive degree.

Unfortunately, existing limitations and the questionable manner of the input data utilized to train this particular AI tool bodes for hidden dangers and indubitably reveals the severe limitations in this latest state-of-the-art technology.

For the legal profession, GPT-3 and its ilk can provide some handy assistance in the practice of law. For example, using a modicum of some initial text, you can potentially have the AI tool produce an entire contract, albeit not based on any semblance of legal logic per se and instead merely due to regurgitating text and potentially fabricating dubious new text.

Another avenue for legal interest involves the possibility that the use of GPT-3 in certain domains might raise thorny issues of liability. Consider the use case of such an AI tool for dispensing medical advice on an auto-generated basis and the specter of offering misleading or outright harmful medical guidance looms large.

Digging Into GT-3

What does GPT-3 do?

In essence, it is an autocomplete capability akin to when you compose an email or write correspondence and the computer tries to fill in what you might want to say next. We’ve all experienced the anticipatory facets of starting a sentence with something like “Haven’t seen” and the next thing you suddenly see appear on the screen as potential next words are “you in a while.” Thus, the system examined your first few words, looked-up what typically follows those words, and then presents for your ease of use the templated words that might fit your needs.

All well and good. If you don’t like the words presented by the computer, you merely overwrite them with whatever you prefer to indicate instead. On the other hand, if you like the proffered words, you can readily accept them and proceed, thereby reducing the amount of effort on your part to compose your message. The autocomplete can be a handy timesaver and reduce the amount of work required to write messages.

The twist involving GPT-3 is that it is akin to the autocomplete feature but stoked on steroids, going far beyond the everyday autocomplete that you are used to leveraging.

Let’s dig into GPT-3 and examine how it has eclipsed the usual formulation of being yet another vanilla auto-complete function.

Standing for Generative Pre-Trained Transformer (GPT), this particular version is the third major instance of the capability, coined therefore as GPT-3. Released in beta use just a month ago by the company OpenAI, a for-profit AI firm located in the Bay Area of California, GPT-3 is a significant advancement over its prior siblings. One of the key differences entails the vastness of the training data used to give the autocomplete its facility to do the fill-in of anticipatory text.

The AI program is considered pre-trained, having examined millions upon millions of prior passages of online text, and can generate anticipatory text that is not just a word-for-word of what it has seen but then also transform the wording to try and better fit the circumstances involved. That’s why it has been given a designated naming that includes the keywords to generate, pre-trained, and transformer.

For the training of GPT-3, the system was allowed to crawl across the enormity of the Internet, seeking out any kind of text imaginable. Besides sucking up all of Wikipedia (the English version), the algorithm scanned online digitized books, tons of articles, slews of poetry, and so on. It has been formulated upon a slurp of whatever text and at times images that it happened upon.

Where this potentially can payoff is that rather than merely providing a few words of anticipatory text, the GPT-3 can provide a boatload of text for you. If you start a sentence with “Four score and seven years ago” then the GPT-3 could respond with the rest of the entirety of the Gettysburg Address.

At first, this seems somewhat simplistic in that it would appear as though it is just grabbing up text that previously was recorded in its database. You could do a query online on your own to find the Gettysburg Address so why does it help to have a tool to do the same thing?

In theory, the GPT-3 will potentially be able to modify the anticipatory text, honing it to your particular circumstances. Now, in the case of the Gettysburg Address, it would seem unlikely that you want the words to be altered. Likely, you want to have the words as they were originally stated. This then brings up one of the possible downsides of a tool like GPT-3, namely that it might or might not retain the word-for-word version of what it earlier saw, and might too opt to provide a modified variant of what it previously had seen.

I believe it was the immortal Forest Gump that famously said that you never know what you might find in a box of chocolates.

So, the good news is that the autocomplete of GPT-3 can do some might impressive feats of writing, potentially generating an entire passage of text that was prompted by your only needing to enter a few words. The resultant passage could be unique and innovative, blending a cornucopia of many other texts that the GPT-3 algorithm has opted to intermix into the resulting output. From just a few starter words as a prompt, GPT-3 could craft an entire thousands-long worded story or narrative for you.

GPT-3 is making use of artificial neural networks, a type of software approach that tries to somewhat simulate the neural network capabilities of the brain, though in a far less capable manner and dissimilar in many material respects. In the AI field, these techniques are referred to as Machine Learning and Deep Learning. Via computationally analyzing the text that it has been fed, the GPT-3 has attempted to identify mathematical patterns and perform a richness of pattern matching across and among the plethora of text input that it has uncovered.

GPT-3 and similar auto-generating tools can be used in spurious and nefarious ways. A simple example might suffice. Some are worried for example that students in schools will opt to use GPT-3 to write their essays for them. A sneaky student merely accesses GPT-3 (side note: it is not widely available per se while still in beta) and could generate a ten-page essay based on inputting a handful of instigating words or phrases. The output is typically being sown together via the GPT-3’s Natural Language Processing (NLP) capacity, such that the text is relatively fluent looking and has the appearance of being written by a human hand.

Of course, like all of today’s NLP, there are still times at which the computer-based translation is detectable, perhaps due to repeated wording or awkward or oddball phrases. Inexorably, these kinds of giveaways or gotchas are gradually being excised and the AI is being advanced to produce text that seems nearly indistinguishable from something a human might have written.

Downfalls Aplenty

A big problem with this system-generated text is that the AI of today has absolutely no semblance of reasoning, no capability of invoking common-sense, and produces text on a pretty much monkey-see-monkey-do basis. You might be aware that theorists have oft postulated that a monkey armed with a typewriter could ultimately produce the works of Shakespeare, known as the infinite monkey theorem in the computer science field, and we are seemingly getting closer to that day.

In the case of the student using GPT-3 to create an essay, the end-result might seem well crafted and inspirational, yet the AI system has not done this in any purposeful way. Instead, the GPT-3 has managed to weave together snippets of text and done so to the degree that it sure seems like it makes a lot of sense.

Well, it ought to potentially be sensible since it is based on what humans have written. If humans are writing sensible things, and if the AI is parroting those writings, the outcome should presumably be sensible too.

Unfortunately, this kind of rote mimicry also means that the untoward stuff written by humans is getting carried lock, stock, and barrel into the GPT-3 generative output too. There are already numerous instances of GPT-3 readily producing outputs that smack of racism, sexist remarks, and a plethora of other disgusting stereotypes and biases. This should not be surprising in that if you look at what is posted on the Internet, there is plenty of crazy and ill-advised text that can be absorbed by an automated crawler that just wants any text that it can find.

This old and yet trusty line still works: Garbage in, garbage out.

There is plenty of textual and narrative garbage on the Internet. Imagine all those nutty manifestos, the conspiracy theories about the world, and all of those incendiary commentaries and postings that rattle around. Much of that blithering flotsam is being potentially mixed into heralded works such as the Gettysburg Address and all other prized writings of humanity via these generative transformer tools.

Now that you’ve got the gist of GPT-3, keep in mind too that there are other similar generative AI-based tools in the marketplace, let’s consider how these pertain to the practice of law.

Legal Profession Interest And Impacts

First, when you are next tasked with writing a new contract, using a tool such as GPT-3 could be a big booster, allowing the AI to compose a contract draft that might be quite close to what you had in mind. This is more likely if the generative tool is focused on legal contracts, rather than being trained across all kinds of texts and documents, since the language of a contract is not usually the same language used in Shakespeare or manifestos.

Second, some believe that a generative AI tool could answer questions. Right now, some have been using GPT-3 to answer medical oriented questions. You type in a health-related question, and the GPT-3 spits out text that pertains (hopefully) to the medical question you have posed. This is not hard to envision since the generative tool likely might have picked-up the same question being asked at numerous web medical sites and has been able to find consistent patterns as to the answers usually logged.

Suppose that this same capacity was used to try and answer legal questions. Yes, it could certainly be attempted. Of course, similar to qualms about answering medical questions, one has to recoil somewhat at the veracity of whatever answers might be given via a tool such as GPT-3. There is also the open question as to whether this kind of AI tool is going over-the-line, as it were, and entering into the act of practicing the law, something that remains an ongoing debate inside and outside of the law profession.

Another legal facet involves the potential for legal liability associated with a generative AI tool like this. Suppose a company sets up a generative piece of software and offers it to the public as a means of answering questions about medical matters. What kind of liability might this create for the company taking such actions? Despite any potential disclaimers, there might nonetheless still be liabilities to be incurred.

A recurring issue with much of the AI that is being fostered into the marketplace has to do with the inscrutability of the systems. An AI package being used to decide who gets a car loan or a home loan might be using hidden biases based on training data that contains those biases, but no one necessarily realizes those biases existed. Efforts to try and get AI to be explainable, known as XAI or explainable-AI, are being pushed by those in the AI Ethics realm.

A generative tool like GPT-3 has already received criticism for its lack of XAI, meaning that when it produces its output, there are no direct means to trace where it came from and how it was derived. Meanwhile, the output can readily be jammed with misinformation. The convincing nature of how well it seems written will cloak the underlying biases and falsehoods.

If you were already worried about the Internet and its writings, generative tools are going to up the ante and be able to produce even more gobs of narratives to be found online and infused into social media and tweets. An autocomplete capability has a lot of potential handiness and likewise can be a bear to deal with. You can simply turn off the feature when composing an email or a document but doing likewise for the Internet is not feasible and we might end-up with incredibly “new” writings of soaring maxims while getting overfilled with outrageous rants and untoward raves.

Conclusion

AI-enabled tools such as GPT-3 are going to continue to be fostered and incrementally improved.

Meanwhile, the application of these auto-generative text capabilities will increasingly be applied to the field of law, showing up as standalone LegalTech or immersed into existing LegalTech.

In addition, you can expect that when these auto-generative machinations are used in various domains, inevitably there will be legal questions raised about what the AI is doing, how the AI came to be, and will spur the judicial system to step into the matter and aid in resolving a likely cacophony of disputes and difficulties that will inexorably appear.

For the latest trends about AI & Law, visit our website www.ai-law.legal

Additional writings by Dr. Lance Eliot:

And to follow Dr. Lance Eliot (@LanceEliot) on Twitter use: https://twitter.com/LanceEliot

Copyright © 2020 Dr. Lance Eliot. All Rights Reserved.

Dr. Lance B. Eliot is a renowned global expert on AI, Stanford Fellow at Stanford University, was a professor at USC, headed an AI Lab, top exec at a major VC.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store