WordPress SEO guide

WordPress Robots.txt Generator Guide

WordPress sites usually need a simple robots.txt file: keep content open, block admin noise, and point crawlers to the sitemap.

Short answer

A safe WordPress robots.txt usually disallows /wp-admin/ and /wp-login.php, allows /wp-admin/admin-ajax.php, and includes the XML sitemap URL. Do not block themes, scripts, images, or public posts that Google needs to render and rank.

Safe WordPress example

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-login.php
Allow: /wp-admin/admin-ajax.php

Sitemap: https://example.com/sitemap.xml

Related crawl-control pages

SEO tools hub Robots.txt Generator Block AI crawlers guide Schema Markup Generator

Frequently asked questions

What should WordPress robots.txt block?+

Most WordPress sites can block /wp-admin/ and /wp-login.php while allowing /wp-admin/admin-ajax.php. Public posts, pages, CSS, JavaScript, and images should stay crawlable.

Should WordPress robots.txt include the sitemap?+

Yes. Add the absolute sitemap URL, usually https://example.com/sitemap.xml or a sitemap index generated by your SEO plugin.