Javatpoint Logo
Javatpoint Logo

Robots Meta Tag

SEO Robots meta tag 1

Robot Meta Tag is used to give instructions to the web spiders. It is implemented on a page to page basis by incorporating a meta tag in the head of an HTML file. It was proposed in 1996 at the W3C workshop and its use was described in December 1999 in Appendix B of HTML 4.01.

Robots meta tag plays an important role in on page optimization as it is the robots meta tag who tells the search engines how to treat the content of a page.

Basic Syntax: It has a very simple basic syntax.

In the syntax, you can put different values as a place holder as we have written "instructions to be followed" as a placeholder. The different values can be index, noindex, follow, nofollow, all or none-thing.

Implementation and meaning of Robot meta tags:

This tag says "Do not index the page and do not follow the links on the page". It is used to instruct the web spider not to index the page and not to follow the links to other pages". So, your page will be dropped out of the search index and rest of your page will not be crawled. This tag is mostly used when a site is in development mode and the developers don?t want the spider to pick the page for indexing.

This tag says "Index the page but do not follow the links on the page". It instructs the spider to index the given page and not to follow the links to other pages. Thus, your page will be included in the search index but the link path to other pages will not be created.

This tag says "Do not index the page but follow the links on the page". So, the spider will not index the given page but will follow the links to other pages, i.e. your page will not be included in the search index but a link path to other pages will be established.

This tag says "Index the page and follow the links on the page". So, the spider will index the given page as well as follow the links to other pages, i.e. your page will be included in the search index and a link path to other pages will be established.

Next TopicSEO Robots.txt




Please Share

facebook twitter google plus pinterest

Learn Latest Tutorials


Preparation


Trending Technologies


B.Tech / MCA