This seems logical, but I'll say it anyway, since Google's John Mueller said it on Twitter. The faster your server responds to GoogleBot, the faster GoogleBot can crawl through your site. Google's John Mueller said "the faster we can crawl, the more we can crawl."
Here is his tweet:
Generally not. However, for sake of fast crawling, I'd still try to get that response as quickly as possible. The faster we can crawl, the more we can crawl.
— John ☆.o(≧▽≦)o.☆ (@JohnMu) May 15, 2018
Of course, Google will crawl what it can without slowing down your server. But you don't want to have Google slow its crawl if you don't have to. So making sure your site is super fast either or both through faster servers or optimized code, is best not just for your end users having a super fast experience but also for Google chowing down on your content.
So make your site fast.
Forum discussion at Twitter.