GPT-3 is like looking into the future... What a time to be living!

Here are two poems by Wallace Stephens, except one is written by AI. Guess which one was written by Stephens?

1

Barque of phosphor 

on the palmy beach,

Move outward into heaven

Into the alabasters,

And night blues

Foam and cloud are one

Sultry moon monsters

Are dissolving.

Fill your black hull 

With bright moonlight.

There will never be an end

To this droning of the surf.

2

I must have shadows on the way

If I am to walk I must have

Each step taken slowly but alone

To have it ready made

And I must think in lines of grey

To have dim thoughts to be my guide

Must look on blue and green

And never let my eye forget

That color is my friend

And purple Must surround me too

The yellow of the sun is no more

Intrusive than the bluish snow

That falls on all of us. I must know

Grey thoughts and blue thoughts walk with me

If I am to go away at all.

AI continues to do things we never thought possible. GPT-3 does what your brain is doing now or more accurately what your brain is doing when you write. It is producing the next word in the sentence. Except it’s not. It is using brute force and a huge amount of data from the internet, to process what your brain does with relative ease. Nevertheless what GPT-3 does is astounding. What GPT-50 will do may be really astounding.

GPT-2 was impressive, a Natural Language Processing model that took a massive amount of text from the internet, at least the quality text, and it performed well, really well. So well, they didn’t release the software, for fear of its power. Open AI kept on going and created GTP-3, which is 117 times bigger that GTP-2. Bigger appears to be better. How much better it can become no one really knows.

Benchmarking GPT-3, on writing articles, humans can distinguish ‘human from AI’ created content accurately 52% of the time. In other words only as good as guessing, in other words, indistinguishable. Whether it’s a poem by Wallace Stephens or an essay, this is a stunning, if not frightening, achievement. Basically, AI is producing texts that are often hard to distinguish from human texts. Let that sink in, as the consequences of that 'competence' are massive.

The researchers claim that GPT-3 has learnt how to learn and that that the bigger the model, the better its adaptive abilities. This hints at the possibility that we are moving slowly towards AGI (Artificial General Intelligence) here. The trick is to focus on what they call the 'few shot' problem. In standard machine learning, you give it loads of data. However, for many real-world problems, the data is small, only a few examples. An example of a few shot learning problem, its digital ID. You give it only a few photos or image of your face, and it recognises your face to give you access to your phone. The larger models, like GPT-3, make better use of fewer examples in the data. What’s more, when it gets things wrong, it tends to get them wrong in plausible ways. Now that is interesting. We may not only get right answers but common misconceptions, which are useful in teaching and learning. 

In learning it can create questions and provide answers to questions, so that quick assessments are possible - lots of them. There's an application of GPT-3 that allows you to speak to people from the past - have a conversation with Marx, Freud... whoever. Its ability to create structured text means that future iterations will write good answers and essays. This could blow the whole 'essay' model in Higher Education.

It is easy to anthropomorphise GPT-3 but the deeper you delve and extrapolate, the more scary it becomes. It feels like a child, a smart, fast learning, precocious child, feeling its way forward in the world. It's not, of course, as a child never gets access to this much language when learning to speak. It may even work without the absence of semantic power. Such power, creeping towards AGI, from just predicting the next word (or token) in a sentence. As the efficacy of the model is still trending upwards, it may still have a long way to go, a very long way. It would seem that ‘competence without comprehension’ is with us in a big way. We could even just stumble upon AGI if the internal ‘learning how to learn’ feature, that seems to be emerging, gets exponential. What a time to be living.