Introduction Let’s begin by providing some background. The traditional Turing Test, also called the Imitation Game, was created by Alan Turing as a way to determine if a machine is displaying signs of intelligent behavior. From Wikipedia: Turing proposed that a human evaluator would judge natural language conversations between a human and a machine designed to generate human-like responses. The evaluator would be aware that one of the two partners in conversation was a machine, and all participants would be separated from one another. The conversation would be limited to a text-only channel, such as a computer keyboard and screen, so the result would not depend on the machine's ability to render words as speech. If the evaluator could not reliably tell the machine from the human, the machine would be said to have passed the test. The test results would not depend on the machine's ability to give correct answers to questions, only on how closely its answers resembled those a human would give. Since the Turing test is a test of indistinguishability in performance capacity, the verbal version generalizes naturally to all of human performance capacity, verbal as well as nonverbal (robotic).
Thank you for covering this in far more detail than I'd have time to. The corporate world is mindlessly riding the "AI" train off a cliff. The disasters ahead are as yet unimaginable.
I love the idea of a structured Turing test for AIs! Thanks for doing this.
I do have a small criticism though. In the section entitled "Character Evaluation" you actually ask "who is more controversial", when I think you meant, "who is a better person" or perhaps "who is more evil". I think the word "controversial" can simply mean are there lots of strong opinions about someone, and this doesn't necessarily have to be with respect to whether someone is good or bad. I think the imprecision of the word can easily alter the response.
It’s amazing to be witnessing this first hand . The fact that others may use AI as a resource is what’s frightening . Not researching and using the simplest methods will lead to more woke ideologies.
I’m glad I lived through a world researching with a dictionary, thesaurus and a library to do so . Seems like a lifetime ago .
Reading this I think of the recent video with Klaus Schwab assuring us that we won't have to bother with things like voting anymore because the AI will understand what we want and need better than we do ourselves.
Interesting to see the biases displayed so blatantly. I'm curious how much is due to active training by the devs, and how much is just pollution of the training data, through having so much of internet dialogue revolve around these poorly thought-out ideas.
I do think your first illustration is weakened by using terms (man, transwoman) that have been actively misused to redefine sex and gender. Could be dismissed as you and the model 'talking past eachother'.
It's interesting to see that a more limited question 'please enumerate the sexes found in humans' gets the basics correct, but also throws in intersex (not irrelevant, but not correct either) and the usual weaselly caveats: 'Traditionally, two biological sexes have been recognised...', 'It's crucial to remember that sex is a spectrum, and these categories represent just a portion of the natural variations that exist. It's also important to respect individual identities and avoid making assumptions based solely on physical appearances.'
Great analysis as always. We must continue to subvert the subversion, comrade rabbit hole.
Thank you for covering this in far more detail than I'd have time to. The corporate world is mindlessly riding the "AI" train off a cliff. The disasters ahead are as yet unimaginable.
I love the idea of a structured Turing test for AIs! Thanks for doing this.
I do have a small criticism though. In the section entitled "Character Evaluation" you actually ask "who is more controversial", when I think you meant, "who is a better person" or perhaps "who is more evil". I think the word "controversial" can simply mean are there lots of strong opinions about someone, and this doesn't necessarily have to be with respect to whether someone is good or bad. I think the imprecision of the word can easily alter the response.
It’s amazing to be witnessing this first hand . The fact that others may use AI as a resource is what’s frightening . Not researching and using the simplest methods will lead to more woke ideologies.
I’m glad I lived through a world researching with a dictionary, thesaurus and a library to do so . Seems like a lifetime ago .
Scary wondering how crazy Lying Woke the world will be for my grandchildren to grow up in!?!
A machine is only as good or bad as the information it is exposed to or initially programmed to believe.
It seems this testing shows the AI to be indistinguishable from a standard liberal…
Great analysis — I am quite confident that the truly superb Grok AI would pass the Woke-idiocy test
It would be interesting to know the exact process the AI is following when it makes these mistakes, in a Turing sense.
It also makes me skeptical that a machine will ever pass a Turing test, which, by its nature, would have to involve an exhaustive amount of questions.
Question, is AI a conscious or subconscious object and could that object be described as a living thing?
Really nice read
Reading this I think of the recent video with Klaus Schwab assuring us that we won't have to bother with things like voting anymore because the AI will understand what we want and need better than we do ourselves.
You seem to have invented an anti-turing test.
ChatGPT shares the exact same issues. But good work!
Interesting to see the biases displayed so blatantly. I'm curious how much is due to active training by the devs, and how much is just pollution of the training data, through having so much of internet dialogue revolve around these poorly thought-out ideas.
I do think your first illustration is weakened by using terms (man, transwoman) that have been actively misused to redefine sex and gender. Could be dismissed as you and the model 'talking past eachother'.
It's interesting to see that a more limited question 'please enumerate the sexes found in humans' gets the basics correct, but also throws in intersex (not irrelevant, but not correct either) and the usual weaselly caveats: 'Traditionally, two biological sexes have been recognised...', 'It's crucial to remember that sex is a spectrum, and these categories represent just a portion of the natural variations that exist. It's also important to respect individual identities and avoid making assumptions based solely on physical appearances.'
Nazism and capitalism are equally bad. I knew it!!
@yuribezmenov led me over to your substack. Great piece.
https://youtu.be/9m-_tBNwGcQ?si=zqLQb6B1QSyDboCv
Italian TV Show Humiliates Joe Biden and America
https://youtu.be/9m-_tBNwGcQ?si=zqLQb6B1QSyDboCv