Read the latest posts - ”Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives”

News from the Cloudflare Blog:

Monday, August 4, 2025

Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives

Perplexity is repeatedly modifying their user agent and changing IPs and ASNs to hide their crawling activity, in direct conflict with explicit no-crawl preferences expressed by websites.

By Gabriel Corral

Read More

Copyright © 2025 Cloudflare, Inc. 101 Townsend Street, San Francisco, CA 94107 www.cloudflare.com | Community | Unsubscrib