Monday, July 27, 2020

GPT-3 is like looking into the future... What a time to be living!

Here are two poems by Wallace Stephens, except one is written by AI. Guess which one was written by Stephens?

1
Barque of phosphor 
on the palmy beach,

Move outward into heaven
Into the alabasters,
And night blues

Foam and cloud are one
Sultry moon monsters
Are dissolving.

Fill your black hull 
With bright moonlight.

There will never be an end
To this droning of the surf.

2
I must have shadows on the way
If I am to walk I must have
Each step taken slowly but alone
To have it ready made

And I must think in lines of grey
To have dim thoughts to be my guide
Must look on blue and green
And never let my eye forget
That color is my friend
And purple Must surround me too

The yellow of the sun is no more
Intrusive than the bluish snow
That falls on all of us. I must know
Grey thoughts and blue thoughts walk with me
If I am to go away at all.

AI continues to do things we never thought possible. GPT-3 does what your brain is doing now or more accurately what your brain is doing when you write. It is producing the next word in the sentence. Except it’s not. It is using brute force and a huge amount of data from the internet, to process what your brain does with relative ease. Nevertheless what GPT-3 does is astounding. What GPT-50 will do may be really astounding.

GPT-2 was impressive, a Natural Language Processing model that took a massive amount of text from the internet, at least the quality text, and it performed well, really well. So well, they didn’t release the software, for fear of its power. Open AI kept on going and created GTP-3, which is 117 times bigger that GTP-2. Bigger appears to be better. How much better it can become no one really knows.

Benchmarking GPT-3, on writing articles, humans can distinguish ‘human from AI’ created content accurately 52% of the time. In other words only as good as guessing, in other words, indistinguishable. Whether it’s a poem by Wallace Stephens or an essay, this is a stunning, if not frightening, achievement. Basically, AI is producing texts that are often hard to distinguish from human texts. Let that sink in, as the consequences of that 'competence' are massive.

The researchers claim that GPT-3 has learnt how to learn and that that the bigger the model, the better its adaptive abilities. This hints at the possibility that we are moving slowly towards AGI (Artificial General Intelligence) here. The trick is to focus on what they call the 'few shot' problem. In standard machine learning, you give it loads of data. However, for many real-world problems, the data is small, only a few examples. An example of a few shot learning problem, its digital ID. You give it only a few photos or image of your face, and it recognises your face to give you access to your phone. The larger models, like GPT-3, make better use of fewer examples in the data. What’s more, when it gets things wrong, it tends to get them wrong in plausible ways. Now that is interesting. We may not only get right answers but common misconceptions, which are useful in teaching and learning. 

In learning it can create questions and provide answers to questions, so that quick assessments are possible - lots of them. There's an application of GPT-3 that allows you to speak to people from the past - have a conversation with Marx, Freud... whoever. Its ability to create structured text means that future iterations will write good answers and essays. This could blow the whole 'essay' model in Higher Education.

It is easy to anthropomorphise GPT-3 but the deeper you delve and extrapolate, the more scary it becomes. It feels like a child, a smart, fast learning, precocious child, feeling its way forward in the world. It's not, of course, as a child never gets access to this much language when learning to speak. It may even work without the absence of semantic power. Such power, creeping towards AGI, from just predicting the next word (or token) in a sentence. As the efficacy of the model is still trending upwards, it may still have a long way to go, a very long way. It would seem that ‘competence without comprehension’ is with us in a big way. We could even just stumble upon AGI if the internal ‘learning how to learn’ feature, that seems to be emerging, gets exponential. What a time to be living.

5 comments:

Stephen Mitchelmore said...

It was a giveaway to me Stevens (not Stephens) wrote only the first because his poems are notable for the lack of the first person. I recommend the Library of America Collected Poems and Prose.

Williams Pater said...

Hello everyone i Am williams pater and i am from USA i am here to give my testimony about an herbal doctor called Dr,olu I was heartbroken because i had very small penis,not nice to satisfy a woman, i have been in so many relationship, but cut off because of my situation, i have used so many product which doctors prescribe for me, but could not offer me the help i searched for. i saw some few comments on the internet about this specialist called Dr,OLU and decided to email him on his email i saw on the internet,(drolusolutionhome@gmail.com ) so I decided to give his herbal product a try. i emailed him and he got back to me, he gave me some comforting words with his herbal product for Penis Enlargement, Within three weeks of me use it, i began to feel the enlargement, " and now it just 4 weeks of using his products my penis is about 8 inches longer, and i had to settle thing out with my ex girlfriend , i was surprised when she said that she is satisfied with my performance in bed and i now have a large penis.thanks to DR OLU for is herbal product. you can also reach him with emsil  drolusolutionhome@gmail.com though is..number WHATASPP him today on this number [ +2348140654426 ]

cheap erectile dysfunction pills online said...

You are so awesome! I don't believe I've truly read through anything like this before. So wonderful to discover somebody with unique thoughts on this topic. Seriously.. many thanks for starting this up. This website is one thing that's needed on the web, someone with a little originality!

buy erection pills said...

I was recommended this web site by my cousin. I am not sure whether this post is written by him as nobody else know such detailed about my trouble. You're incredible! Thanks!

Wales Publications said...

Wales publishers are offering optimized, Best Publication Services in UK to boost the researcher and research communities, by providing accelerated and efficient services to fasten the publishing process and to give more opportunities for research on different disciplines.