When Google CEO Sundar Pichai unveiled artificial intelligence-generated features for the company's search engine last month, he and his colleagues demonstrated the new functionality with six text-based queries that the public could try out.
The questions included “How do I clean an upholstered sofa?” and “What can I use to remove coffee stains from carpet?”, aimed at highlighting how Google's new AI summary feature can generate complete and useful summaries of information on top of traditional search results.
But by Friday, only one in six queries still displayed the AI summary, according to a New York Times test. Rather, the feature was not visibly widespread: a search for “What can I use to remove coffee stains from carpet” brought up a text excerpt from the website JDog Carpet Cleaning & Floor Care, while a search for “How do I clean an upholstered sofa?” was replaced with a link to an HGTV website with the answer. (Results may vary by user and location.)
The disappearance of AI summaries from some searches appears to be part of a broader rollback after the new technology produced a number of falsehoods and inaccuracies, including recommending glue as part of a pizza recipe and ingesting rocks for nutrition. Users loudly complained about the errors on social media, often outright mocking Google.
Liz Reid, who was recently promoted to head of Google's search division, wrote in a blog post on Thursday that the company has trimmed down its AI overview in several ways, launched “additional trigger refinements” to provide more thoughtful answers about health, disabled misleading advice and limited the inclusion of satire and user answers from forums like Reddit.
“We will continue to improve when and how we show AI summaries to strengthen our protections,” she wrote, adding that Google is working on updates to improve a broad range of search results.
Google spokeswoman Ashley Thompson said in a statement Friday that the company has made more than a dozen technical updates to its system.
“AI Overview currently helps users with a number of queries in Search and acts as a launch pad for content across the web.” The company added that while it is making adjustments to improve AI Overview, it has no plans to move away from the feature in the long term.
The shift was a blow to Google's efforts to catch up with rivals Microsoft and OpenAI, maker of the chatbot ChatGPT, in the fierce race for AI leadership, and highlighted the difficult strategic choice the company faces: embrace AI techniques that may be less reliable, or keep its hugely popular search engine and risk falling behind competitors.
Google has chosen to move at a slower pace than Microsoft, which rolled out conversational AI-heavy features to its Bing search engine early last year. Google, which has many more users than Bing, tested AI features in its search engine a year before introducing AI Overviews. The company said the new features will roll out to U.S. users immediately and to more than 1 billion people by the end of the year.
But ultimately, Google “should have rolled this out more slowly,” said Patrick Hall, an assistant professor of decision sciences at George Washington University's School of Management. “Once something like this happens, it really forces you to back away, and at the very least, it's reputational damage for the company.”
Google, which has led internet search for over two decades, has been in a flurry since OpenAI released ChatGPT in 2022. Some tech industry insiders believe that the chatbot's answer-generating capabilities are a serious threat to Google's search engine, the most common way people get information online.
Since then, Google has been working aggressively to regain its lead in AI, releasing a suite of technologies called Gemini that includes new AI models for developers. The company has also built the technology into YouTube, Gmail, and Docs, helping users create videos, emails, and drafts with less effort.
Speaking at Google's Developer Conference last month, Reid said the AI Overview feature will allow the search engine to do more of the searching on users' behalf, highlighting the increasingly complex requests that the feature will help Google address, though it has not yet rolled out to users.
On stage, in response to simple questions such as “How do you get the campfire smell off your clothes?” the AI responded with answers such as exposing it to air, adding baking soda or spraying it with lemon juice.
But when users tried the new service, they found that AI Overview sometimes produced incorrect or downright dangerous answers, like recommending non-toxic glue to keep cheese on pizza. It also misquoted websites and got presidential history wrong.
Of the six questions Google made available to the public, the only one that consistently triggered the AI Overview on Friday was “What are some interesting science projects I could do with my 12-year-old son?”
The answer? Grow crystals, extract DNA from saliva, and write messages in invisible ink.