You'd think the robots.txt directive it pretty cut and dry. It hasn't changed much in the past several years......
-
Google Talks About The Robots.txt Directive For One Whole Hour
seroundtable.com