Industry Insights

Blog, Technology

AI Search Isn’t There Yet

by | Friday, April 4th, 2025

People do this all the time: if they don’t know an answer, they just make something up that sounds right. It turns out AI has the same bad habit.

A Study Put AI Search to the Test, and It Did Not Go Well

Researchers at the Tow Center for Digital Journalism (part of Columbia Journalism Review) wanted to see how well AI-powered search tools could find real news articles. They tested eight AI search engines, including ChatGPT Search, Google’s Gemini, and Microsoft’s Copilot.

Here’s what they did:

  • They picked 200 news articles from well-known publishers.
  • They asked the AI tools to find the article’s title, publisher, publication date, and web link.
  • They compared the AI’s answers to see how accurate they were.

The results that came back didn’t fill the researchers with a lot of confidence.

AI Search Tools Are Overconfident and Wrong

More than 60 percent of the answers were incorrect, but the AI acted like it was completely sure of itself. Some answers were worse than others. Here are some of the results:

  • Perplexity was wrong 37 percent of the time.
  • Grok-2 messed up 94 percent of the time.
  • Microsoft’s Copilot was the only one that refused to answer when it didn’t know (which was actually better).

Some AI tools even made up fake website links or credited the wrong source. That’s like someone giving you directions to a place that doesn’t exist.

AI Was Worse With Blocked Content

Here’s the weirdest part: when publishers blocked AI crawlers from reading their content, the AI was somehow better at finding the right information than when it had free rein of the underlying information.

Even in cases where publishers had partnerships with AI companies, misattribution was still a problem. Some AI platforms linked to republished versions of articles on different websites, making it harder to track down the real source. Obviously, this is problematic for people using the platform to search for facts.

The Big Takeaway: Just Don’t Trust AI Search

The researchers expected AI to either find the right answer or admit it didn’t know. Instead, it just made things up. This matters because AI search tools are becoming more common, and if they’re unreliable, they could mislead people and harm publishers. The last thing society needs is more misinformation out there from supposedly reputable platforms.

AI can be useful, but when it comes to searching for facts, be skeptical and double-check the answers yourself. If you would like to talk to one of our knowledgeable experts about integrating artificial intelligence into your business, give us a call today at 1-855-405-8889.

A Glimpse Into What Compliance Looks Like for Businesses

Apr 7, 2025 | ,

It’s easy to see all the reasons why you should make data regulations and compliance a priority. After all, you want to ensure you don’t violate the trust and security of your customers, as well as the integrity of your operations. If you make even one mistake, it...

Remote Work Is Great, but There Are Some Pitfalls

Apr 2, 2025 | ,

Do you have employees working remotely? If you do, the real question is, are you doing everything you can to keep them productive and secure? Remote work is awesome, but it comes with its fair share of risks. Today, we get into how to competently confront them. Remote...

Hiring IT is Hard (Here’s How to Make It Easier)

Mar 28, 2025 | ,

Do you have someone on your staff who can handle most IT-related issues for your business? If not, we’re sure your organization feels it in more ways than one. The issues that come from not having IT help are only made more frustrating when it comes time to find IT...

Let’s Take the Lid Off of CAPTCHA

Mar 21, 2025 | ,

We've officially reached the point where humans have to prove they're, well, human just to access websites. One of the most common ways to do this? CAPTCHA. CAPTCHA stands for Completely Automated Public Turing test to tell Computers and Humans Apart. It might sound...

Automation Isn’t Always the Best Business Option

Mar 17, 2025 | ,

Automation makes sense from an operations standpoint, and people see this despite the many who advocate for scaling back to save jobs. For every task that can be completed, however, less than half can be automated. When you consider all the tasks that a human might be...

Is Your Cloud Hosting Reliable?

Mar 14, 2025 | ,

If your business uses the cloud, you need that cloud to be reliable. Otherwise, you’re just wasting resources. Let’s examine how you can make your cloud solution as reliable as possible so you’re not leaving any money on the table. What Makes a Cloud Solution...

More Reading from Industry Insights:

Hiring IT is Hard (Here’s How to Make It Easier)

by | Mar 28, 2025 | ,

Do you have someone on your staff who can handle most IT-related issues for your business? If not, we’re sure your organization feels it in more ways than one. The issues that come from not having IT help are only made more frustrating when it comes time to find IT...