Please read the following guidelines carefully before you start using this tool:
1. The first and second rows are "Default - All Robots" and "Crawl-Delay", you should choose the "Default" value for all robots. If you don't want to modify them, just leave them as they are.
2. The third row is about Sitemap. Make sure you put Sitemap link of your site in the text box. If you don't have one, let create it with our XML Sitemap Generator tool.
3. Search Robots row. For example, you choose "Google" for search engines.
The first row is for search engines, the second row is for pictures, and the third row is for the mobile version of your website. Set the "Allowed" or any option you want.
Do the same for other search engines.
4. Restricted Directories: As the last option, you might disable crawlers from indexing certain sections of the page. Remember to add the forward-slash before entering the directory or page's URL.