Free Robots.txt Generator Tool | Control Search Engine Crawling

🤖 SEO Tool · GrowwithBA

Robots.txt Generator

Control what search engines can crawl on your WordPress site. Build a clean, optimized robots.txt in seconds.

Site Info

Site URL

Sitemap URL

Crawl Delay — seconds (leave 0 to omit)

WordPress Default Rules

🔧 Toggle rules on/off for all crawlers (User-agent: *)

Disallow Specific Paths

Blocks crawlers from these URLs

Allow Specific Paths

Explicitly permits crawling these URLs

Block Specific Bots

Creates separate User-agent entries with Disallow: /

⚠️ Blocking bots only works for bots that respect robots.txt. Malicious scrapers will ignore it.

Generated robots.txt

Fill in the fields above...

📁 Upload this as robots.txt to your WordPress root directory, or use Yoast SEO → Tools → File Editor to paste it directly.