The Turing Test, also called the imitation game, is an exercise devised by Alan Turing in 1950 to determine whether a machine can exhibit intelligent behaviour. An interrogator sits in a room with two screens and keyboards. They know behind one screen is a human and behind the other is a machine. They do not know which is which.
Through sending messages back and forth to each screen, they have to determine which is the human and which the machine. If they can't distinguish the machine from the human, then the machine is said to have passed the Turing Test and capable of exhibiting intelligent behaviour.
Many philosophers debate the question of what intelligence is and whether machines can think. Because thinking is very hard to define, Turing answered the question with another question. "Are there imaginable digital computers which would do well in the imitation game?".
Given the rapid advancements in AI, we think there's a good chance that some algorithms may be able to fool humans and win the imitation game. One of the most promising algorithms is OpenAI's GPT3. There has been extensive media coverage of this algorithm being able to produce long paragraphs of writing indistinguishable from that of a human's.
We have been given access to GPT3 so have decided to run the world's largest Turing Test. The results from this experiment will not only help us figure out how good conversational AIs have got but also what kind of things they say that give away that they are not human. This will help us improve their conversational skills for when we run this experiment again.