How to use the robots META tag
September 1st, 2005 | Filed under PHP, Code snippets, HTML, Search Engine OptimizationThe robots <meta> tag in the <head> of your web page tells well-behaved robots (like the Googlebot) whether or not to index a page and whether or not to follow links on a page. For bloggers and zine publishers, robots should only index the permanent locations of posts, not the ever-changing front page or archive pages.
To use meta tags to direct the Googlebot to your permalinked post locations, insert the following in the <head> on your front page and archive pages:
<meta name="robots" content="noindex,follow" />
That says, “Hey bot, don’t index this page, but follow the links.” You don’t want these pages indexed because they will change every time you post to your site. The posts’ permanent locations, which should be linked on your front page, will never change however, and so the robot should follow the links and index them. Those permanent locations of your posts should have the following in the document:
<meta name="robots" content="index,follow" />
Alternately, if you don’t want the links in your posts followed but you do want your posts added to the search engines’ indices, use:
<meta name="robots" content="index,nofollow" />
To achieve this in a WordPress template or any PHP page where $single is set if you’re viewing a single post’s permalinked location, insert the following into the templates that include <head> tags:
<meta name="robots" content="<?php echo (!$single?'no':''); ?>index,follow" />