Google's John Mueller said on Twitter that if your content on your site is being outranked by robotted out pages, then you got work to do. In short, if you have a page on your site that you used the robots.txt file to tell Google not to crawl and it still outranks other relevant content on your site for a query in Google - that is an issue....