robots.txt generator online free

Search Engine Optimization

Robots.txt Generator

Default - All Robots are:  
Sitemap: (leave blank if you don't have) 
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo MM
  Yahoo Blogs
  DMOZ Checker
  MSN PicSearch
Restricted Directories: The path is relative to root and must contain a trailing slash "/"

Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.

About Robots.txt Generator

Free online robot.txt generator - robots.txt maker - robot txt file creator

In web world robots exclusion protocol (REP) plays a major role in crawling of your website and it's web pages. Robots.txt file gives instructions to all search engine robots on which web pages / folder should be crawl or which pages / directories should be ignored.

Where to place robots.txt

First please make sure the file name as a robots with an extension .txt (robots.txt)is correct

By default all search engine bots search for robots.txt file in the highest level directory i.e root directory of your website (i.e You need to transfer the text file to the root directory of your domain through any FTP client (File Transfer Protocol Client Software).

Note : Do you know that robots.txt files are very important and a single text file that could slow down or destroy your website complete ? so be caution while your are creating the file.

Online robots.txt file maker Key Features and Benefits

  • Robots.txt file can be uploaded to your root directory of web server through any FTP Client software or C-panel.
  • The file give instructions to all web bots about their website, so that which of your website's pages or directories should and should not show up in searches results.
  • Will give you correct direction when you decide to add new instruction, file, or file path to either the new or existing robots.txt file.
  • When using our robot.txt builder tool, you can customise as to which search engines you want to include in your robots file.
  • After creating your robots.txt file, upload it to the / (root) , all search engines will be pointed to as which web page's, folder or directories of your site should or should not be shown in searches result.