Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives
We are observing stealth crawling behavior from Perplexity, an AI-powered answer engine. Although Perplexity initially crawls from their declared user agent, when they are presented with a network ...