Google Does Not Support Crawl Delay But Bing Does

Dec 13, 2019 - 7:43 am 0 — by
Filed Under Search Engine Optimization

Crawl Delay

Google's John Mueller said on Twitter that Google "never supported crawl-delay in robots.txt." Bing's Frédéric Dubut said on Twitter "Bingbot supports crawl-delay." That is not to say Google does not let you slow the crawl, you can and here is how.

Here are the tweets:

The crawl rate setting is still in the legacy Search Console, but you can still access it over here. You can learn more about Google's crawl rate over here.

Forum discussion at Twitter.

 

Popular Categories

The Pulse of the search community

Follow
Subscribe Options

Search Video Recaps

 
- YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: March 10, 2025

Mar 10, 2025 - 10:00 am
Google Ads

Google AI Mode Ads Will Be Coming

Mar 10, 2025 - 7:51 am
Google Search Engine Optimization

Google Expands Site Reputation Abuse Policy Manual Actions Next Steps

Mar 10, 2025 - 7:41 am
Google

Google Responds To Publishers Concerns On AI Mode

Mar 10, 2025 - 7:31 am
Google Maps

Google Knowledge Panel With Expandable Places Map

Mar 10, 2025 - 7:21 am
Google

Google Search Knowledge Panels Larger Font

Mar 10, 2025 - 7:11 am
Previous Story: Google: Speakable Markup Works Outside Of News Content
Next Story: Google Local Panel With "X Minutes From Landmark"

The content at the Search Engine Roundtable are the sole opinion of the authors and in no way reflect views of RustyBrick ®, Inc
Copyright © 1994-2025 RustyBrick ®, Inc. Web Development All Rights Reserved.
This work by Search Engine Roundtable is licensed under a Creative Commons Attribution 3.0 United States License. Creative Commons License and YouTube videos under YouTube's ToS.