Header Ads

How to create robots.txt file

How to create robots.txt file

Here in this blog post you can get detailed information of robots.txt file. Actually lot people don't know importance of robots file.

Like us For Regular Updates of Digital Marketing

What is Robots.txt?

How to create Robots.txt file

Robots.txt file is a instructions file to search engines. Based on Robots.txt instruction only search engine cralws/robts will read our website. Without knowing to the Google we can maintain a page, we can store in the form of a text document. We need to upload this txt.file in the server. We can inform or instruct the search engines to what page of our website should be indexed or crawled and what pages to be not. So we need robo.txt file only if our website includes content which we do not want search engine to index like Admin Page. (Content by - Indu)

For example:
If we don’t want our login page to index then we have to save file as

User-agent: *


/-is nothing but a folder


  1. Such a great articles in my carrier, It's wonderful commands like easiest understand words of knowledge in information's.
    Digital Marketing Company in India

  2. Thanks for sharing the information and keep sharing the blogs. Robots.txt is a file where we can block the pages in our website which search engine will not read that page and will not index the specific page. Through this robots.txt file,search engines will read our website.
    digital media marketing agency