As explained on the Google blog.....
In addition to the robots.txt file -- which allows you to concisely specify instructions for a large number of files on your web site -- you can use the robots META tag for fine-grain control over individual pages on your site. To implement this, simply add specific META tags to HTML pages to control how each individual page is indexed. Together, robots.txt and META tags give you the flexibility to express complex access policies relatively easily.
To prevent a webpage from showing up in the search results, you can insert this line within HEAD tags:
<meta content="NOINDEX" name="GOOGLEBOT">
Add the NOARCHIVE tag to a web page and Google won't cache copy of a web page in search results :
<meta content="NOARCHIVE" name="GOOGLEBOT">
As a result of the above line, that web-page will appear in a search result but the Cached link that typically appears in a search result will not be shown.
Also see:
HOW TO find "dofollow" blogs/websites
rel="nofollow" - one non-standard HTML attribute value that the Big 3 search engines support
90