The launch of Google‘s new AI chatbot Bard has sparked plenty of speculation about how human-like its conversational abilities really are. One way to evaluate this is by testing whether Bard can pass the famous Turing test. In this in-depth analysis, we‘ll take a close look at the Turing test, examine what we know about Bard so far, and determine if it has what it takes to fool humans into thinking they‘re chatting with a person instead of a machine.
A Brief History of the Turing Test
First proposed by British mathematician Alan Turing in 1950, the Turing test has become a landmark for artificial intelligence. The simple premise involves a human judge conversing with a machine and another human via text. If the judge can‘t reliably identify which is the bot, the machine is said to pass the test.
But the exact criteria for passing has provoked debate for decades. Some early versions were quite limited:
- 1966 ELIZA bot – posed as a therapist with canned responses
- 1972 PARRY bot – mimicked a paranoid person
These programs could fool people on surface topics through clever tricks, but had no real intelligence.
As AI advanced, versions of the test expanded to evaluate capabilities like:
- Knowledge – answering general knowledge questions
- Reasoning – logical thinking and causal understanding
- Planning – problem solving abilities
- Linguistic competence – fluid conversations
Different competitions also emerged with varying criteria, though none widely accepted as a definitive standard test.
Over the decades, AI systems have gotten closer to mimicking certain human abilities during text conversations. But the Turing test remains controversial in terms of evaluating true intelligence. Still, it provides a useful benchmark for assessing conversational AI.
Benchmarking AI Progress with the Turing Test
The value of the Turing test is less about defining true intelligence than benchmarking AI progress. By measuring how indistinguishable an AI system‘s conversational abilities are from a human‘s, we get an indication of advances being made with conversational AI.
|Deceived some users
|Displayed mental illnesses
|Won Loebner Prize
As this table shows, chatbots have progressed steadily, if slowly, in their ability to mimic human conversational patterns since the 1960s. A critical milestone was reached in 2008 when chatbots first fooled over 30% of human judges that they were human. And some now can even engage in meaningful exchanges beyond just scripted responses.
Google Bard represents the latest milestone in this decades-long quest to create conversational AI that comes ever closer to matching humans. To determine its progress, let‘s evaluate its conversational skills specifically.
Current State of Google Bard‘s Conversational Abilities
Google Bard aims to combine broad knowledge with eloquent, human-like language capabilities. According to Google, it can:
- Hold natural dialog spanning many topics
- Adjust its tone based on audience and context
- Cite trustworthy sources to explain concepts
- Admit when it doesn‘t know something
It builds on large language models like LaMDA, leveraging vast data and computational power. But does this translate into human-like conversational competence?
Early demos do show impressive coherence and versatility. In one conversation with a Google engineer, Bard smoothly shifted between discussing vacation tips, physicist Richard Feynman, and even composing a poem.
However, some limitations stand out that reveal its artificial nature:
- Hesitates on more complex questions requiring deeper reasoning
- Lacks subjective experiences to draw from
- Replies can be vague or repetitive
- Makes some unnatural topic leaps
While a solid conversationalist, its skills are still noticeably short of human capability based on current evidence. But it appears superior to predecessors like Siri and Alexa thanks to foundational progress made in AI.
Can Any Current Chatbot Pass an Unrestricted Turing Test?
Given the state of conversational AI today, could any chatbot reliably pass a rigorous, unrestricted Turing test? Most experts believe the answer is still no.
Chatbots like Mitsuku and Google Duplex demonstrate impressive conversational chops within limited domains. But when subjected to unpredictable open-ended questions on any topic, their artificial nature becomes apparent.
Key obstacles current AI still struggles with:
- Limited training data – No chatbot can yet span the immense variety of human conversations
- Context comprehension – Difficulty following dialog context beyond a few turns
- Reasoning gaps – Struggle with hypotheticals, complex inference and causality
- Opaqueness – Inability to explain reasoning behind responses
The Loebner Prize competition each year shows these issues remain barriers to chatbots convincingly passing the Turing Test. The few that get close exploit clever conversational tactics to mask shortcomings.
We‘re edging towards conversational AI that can pass limited forms of the test. But general human-level conversational competence remains beyond current capabilities.
Early Turing-Style Tests of Google Bard
Since being unveiled recently, Bard has already faced some informal Turing-style tests to gauge its conversational chops:
Psychiatric Times Test
Questioned about anxiety, Bard revealed gaps in mental health knowledge. It gave responses like "I do not actually experience emotions" showing its non-human nature.
Bard held up fairly well fielding unpredictable questions from CNBC hosts, demonstrating knowledge and humor. But some responses veered oddly off-topic.
When interrogated on Reddit, Bard relied heavily on searching internet sources to answer. Replies were often vague or repetitive.
Faced with hypotheticals from Wired to evaluate reasoning, Bard struggled. But it generated thoughtful responses to ethical dilemmas.
So far, these early interviews showcase areas of competence, like using sources and discussing principles. But clear shortcomings in reasoning, subjective experience and conversational depth reinforce its current artificiality.
Does Google Bard Pass an Unrestricted Turing Test Today?
Based on current demonstrated abilities, I would assess that Google Bard does not yet pass a fully unrestricted Turing test with skilled human interrogators aiming to determine machine from human.
A few factors give it away as machine over human:
- Limited conversational range – Still struggles with unpredictable questions and complex inference
- Transparent identity – Openly identifies as an AI system built by Google
- Response patterns – Conversation has a distinct computational, rather than human, flow
- Knowledge gaps – Lack of subjective experiences and opinions reveal its limitations
Of course, it shows impressive progress in human-like conversation over previous chatbots. I could see it potentially fooling some interrogators in more limited tests. But as of today, its conversation still lacks the depth and breadth to reliably pass an unrestricted Turing assessment.
Advancing Towards Human-Like Conversation
Google will undoubtedly continue improving Bard‘s conversational abilities over time. What enhancements might close the gap to reliably passing an open-ended Turing Test?
Expand Training Data
Access to more training conversations and world knowledge will minimize gaps and non-sequiturs. Integrating multiple data sources can achieve this.
Improving causal understanding and hypothetical reasoning will lead to more rational responses.
Natural Dialog Patterns
Mimicking human timing, turn-taking, interruptions, repetition and reactions will make exchanges more natural.
Generating personal facts, experiences and opinions would allow it to converse subjectively like humans.
Allowing real-time user ratings and critiques of responses could pinpoint weak points for improvement.
Explanations for responses and reasoning could inspire more confidence in its capabilities.
With continued advances in foundational AI, plus Google‘s resources, I see Bard‘s conversational competence advancing rapidly in the coming years – perhaps eventually even crossing the Turing Test threshold one day.
The Exciting Road Ahead
The launch of chatbots like Google Bard represent major milestones in the decades-long quest to achieve human-level conversational AI. While shortcomings remain that reveal its artificial nature, Bard demonstrates impressive progress in mimicking human dialogue.
Looking ahead, I see conversational AI following an exciting development path:
- 2025: Chatbots gain truly robust world knowledge and reasoning,convincingly passing limited Turing tests with over 50% of human interrogators.
- 2030: Systems exhibit comprehensive conversational competence rivaling humans, reliably fooling over 90% of Turing test judges with increasing durations.
- 2040: Conversational AI becomes indistinguishable from humans for most practical purposes, passing even rigorously unrestricted Turing tests.
The coming years promise to be an amazing period of progress for conversational AI. While chatbots like Bard still have limitations today, they represent a major step towards machines that can converse as fluently as people. I look forward to seeing what the future holds as this technology continues advancing.