robots.txt maker
Choose what search engines may visit and get a robots.txt file ready to upload.
Runs in your browser. Nothing is uploaded.
How to use
- Choose whether search engines may visit the site.
- List any paths to keep them out of, one per line, such as /admin/.
- Add your sitemap address.
- Copy the result into a file called robots.txt at the top of your site.
Good to know
- robots.txt asks well-behaved crawlers to stay out. It does not hide or protect pages.
- To keep a page out of Google results, use a noindex tag instead.
Questions
Where does robots.txt go?
At the root of the site, for example example.com/robots.txt.
Will blocking a page remove it from Google?
Not always. Google can still list a blocked address if other sites link to it. Use noindex.
Should I include my sitemap?
Yes. It helps search engines find all your pages.
Also known as: robots.txt generator, create robots.txt, robots file maker.