Google: URL Parameters Tool Is Not A Replacement For Robots.txt

14:56 01.06.2020
Google's John Mueller said on Twitter that using the URL parameter tool is no replacement for using a robots.txt file for blocking content. John was asked "how reliable is it" when setting "crawl no urls" of a certain type of URL pattern. John said "it's not a replacement for the robots.txt -- if you need to be sure that something's not crawled, then block it properly."...
Теги: Google twitter
  763