Google: Crawling At Large Scale Is Complicated; Don't Oversimplify It

15:56 08.01.2020
Google's John Mueller responded to a question related to crawl budget around large scale web sites saying "crawling at a large scale is much more complicated." This was when he proposed two ideas for improving crawl budget but John "both of those are too simplified" and there is more to it....
Теги: Google
  762