WIRED reveals Perplexity is an Artificial Intelligence-Powered Search Engine, with an Application to the Condé Nast/Analysis

A WIRED analysis and one carried out by developer Robb Knight suggest that Perplexity is able to achieve this partly through apparently ignoring a widely accepted web standard known as the Robots Exclusion Protocol to surreptitiously scrape areas of websites that operators do not want accessed by bots, despite claiming that it won’t. Perplexity is almost certainly responsible for the machine that WIRED observed onwired.com and other Condé Nast publications.

Perplexity CEO Aravind Srinivas talked to WIRED earlier this year about his company’s answer engine, which it says is a natural-language chatbot that gives answers to questions in real time. He told Forbes that shortly before the funding round valued the company at a billion dollars it was almost like they had a kid. More recently, after Forbes accused Perplexity of plagiarizing its content, Srinivas told the AP it was a mere “aggregator of information.”

Perplexity is more specific. Prompted to talk about what Perplexity is, it says that it is an Artificial Intelligence-powered search engine. It provides concise, real-time answers to user queries by pulling information from recent articles and indexing the web daily.”

Web crawling—the act of indexing information across the internet—has been around for decades. It has been used by search engines to catalog the contents of the open internet and make it easier to find. Web crawling is usually seen as benign, as websites rely on the process as a way to find their content. But now crawling tech has been subsumed by the great AI-ening of everything, and is being used by companies like Google and Perplexity AI to absorb whole articles that are fed into their summarizing machines.

Randall Lane, Hacks on Max, GabgetLab, and Solar Keys: Podcast for the Week of November 17 – 23rd

Randall’s new league is the National Thoroughbred League. Kate recommends the book Victim by Andrew Boryga. Lauren loves the show Hacks on Max.

Randall Lane can be found on social media @RandallLane. Kate Knibbs is a person. LaurenGoode is a person. Michael Calore is @snackfight. The main hotline is atGadgetLab. The show is produced by Boone Ashworth (@booneashworth). Our theme music is by Solar Keys.

You can always listen to this week’s podcast through the audio player on this page, but if you want to subscribe for free to get every episode, here’s how:

If you’re on an iPhone or iPad, open the app called Podcasts, or just tap this link. You can also look for gadgets with the help of an app or search on the internet. You can find us on the Podcasts section of the app on the phone. We are also on a service called Spotify. And in case you really need it, here’s the RSS feed.