Google is out with a new video in The Non-Canonical Cabinet series, this one is on 404 pages and why they are not scary. I like how they also timed the video with a Halloween theme. In the video they discuss common misconceptions about HTTP 404 errors, crawl budget, and how website owners should handle deleted URLs.
The video is less than 10 minutes long and is broken up into these sections:
- 1:15 – Search Console 404 panic: Are dead pages really bad?
- 2:08 – Technical indexing vs. business & user experience impact
- 3:40 – Redirect all 404s to the homepage? To do or not to do?
- 4:03 – The 3-Step Framework for managing dead URLs
- 6:13 – Myth busted: Do 404s drain your crawl budget?
- 7:11 – Why 500 server errors and 429 limits are the real monsters
- 8:03 – Recap and Gary's cowboy moment
Here is the video:
I should note, there is nothing specifically new here but it is a nice recap video.
Here are the key takeaways:
1. 404 errors do not cause Google penalties. A 404 means a page was not found. Google expects pages to disappear from the web and will eventually remove missing URLs from its index. A 404 does not, by itself, harm other healthy pages on the site, Google explained.
2. You don't need to fix every 404. Fixing one is primarily a business and user-experience decision, not an SEO requirement. If a deleted product page still receives traffic, consider redirecting visitors to a relevant replacement or using a helpful custom 404 page.
3. Don't redirect every 404 to the homepage. Redirects should lead to genuinely relevant destinations. Redirecting unrelated, deleted URLs to the homepage can confuse visitors and produce misleading data.
4. Use a three-step framework for handling deleted URLs.
- Redirect to a relevant replacement when one exists.
- Keep the 404 or return a 410 when the page is permanently gone and has no suitable replacement.
- Improve the 404 page with a helpful message, search box, and links to relevant categories or content.
5. 404 errors do not drain crawl budget. Google says that encountering normal 404 responses does not consume crawl capacity in a way that reduces crawling of the rest of the site. This is a common myth, Gary added.
6. HTTP 500 and 429 errors are different. Server errors (500-class responses) and HTTP 429 (“Too Many Requests”) can indicate that a server is struggling or limiting requests. Google may slow crawling in response.
7. Don't panic over Search Console reports. A list of 404 errors is not automatically evidence of an SEO problem. Evaluate the URLs based on their traffic, links, business value, and whether a genuine replacement exists.
Here is the full transcript cleaned up a bit:
Chapter 1: Welcome Back to the Cabinet (Where Is the Chicken?)
Cherry Prommawin: Gary, if you are hiding in the infrastructure archives again, I'm locking the drawer from the outside.
Gary Illyes: Boo! Be scared of me. I'm a ghost.
Cherry Prommawin: Gary, it is 2:00 in the afternoon. Why are you wearing my good tablecloth?
Gary Illyes: Are those holes? I'm illustrating a 404 page, Gary, a digital ghost, an empty entity, a phantom of a server room. Be scared of me, please.
Cherry Prommawin: No.
Cherry Prommawin: Fine, but this brings us to a terrifying question in the non-canonical cabinet. Is a 404 status code the ultimate disaster for your website? Does it matter? Should people be afraid of it?
Cherry Prommawin: You know, let's go find out.
Cherry Prommawin: You know that you owe me a new tablecloth, right?
Chapter 2: Search Console 404 Panic — Are Dead Pages Really Bad?
Cherry Prommawin: All right, all right, all right. Let's face our fears. You open Search Console, see a list of 404 errors, and boom, instant panic. People think the domain will get a manual action or it will be cursed and never rank again. Gary, a 404 is not that scary, is it?
Gary Illyes: No, we don't care at all. An HTTP 404 status code literally just means "not found." It is a perfectly natural, healthy part of the web. Pages die, products sell out, all URLs get deleted.
Gary Illyes: Googlebot encounters a 404, realizes the page is gone, and eventually we drop it from the index. There is no penalty, no secret Google jail, and no negative impact on your other healthy pages. From our perspective, it's completely fine to just leave them be.
Cherry Prommawin: But that's not the full picture, is it?
Gary Illyes: Yeah, maybe we should talk about the business side of things. But I really miss our chicken.
Chapter 3: Technical Indexing vs. Business and User Experience Impact
Cherry Prommawin: Anyway, from the business point of view, the problem is like our chicken situation right now. Chicken is gone.
Cherry Prommawin: Chicken has left the office like a 404. He's not found. Technically, our office still functions perfectly fine without him. The structural integrity of the office is intact. If Googlebot is coming for chicken and chicken is not here, Googlebot just doesn't care. But if a human user comes explicitly looking for chicken, they might get disappointed and go look somewhere else.
Gary Illyes: Exactly. Fixing a 404 is entirely optional, but it can be a smart business move depending on your site.
Gary Illyes: For example, if you run an e-commerce store and you delete a popular product page like "Ultimate Deluxe Chicken Feed," leaving a blank error page means you might be dropping potential sales, right?
Gary Illyes: It's more about user experience, less about Google Search. If the dead URL still gets traffic from links or social media, you might choose to guide those users to a newer product model using a redirect. But if there's no replacement, just let it be a 404.
Gary Illyes: Also, don't take the lazy shortcut if you're thinking, "Huh, let's redirect all your dead pages to the homepage." That just confuses humans and makes your data very confusing.
Chapter 4: Redirect All 404s to the Homepage? To Do or Not to Do?
Cherry Prommawin: Um, hey Gary.
Gary Illyes: Hmm?
Cherry Prommawin: Why are you wearing a business suit over your hoodie? And why am I wearing a cowboy hat?
Gary Illyes: I don't know about your cowboy hat, but I'm suiting up to talk about business framework. Scary. Yee-haw!
Cherry Prommawin: So, if a site owner does want to manage their digital graveyard, they can probably benefit from a logical framework.
Chapter 5: The Three-Step Framework for Managing Dead URLs
Cherry Prommawin: A three-step framework.
Cherry Prommawin: First, one-to-one relevant redirects, but only if a true replacement exists. If that page has a direct, natural replacement, like an updated version of the same article or a newer model of the exact same product, you set up a redirect only to that specific, highly relevant destination.
Gary Illyes: But we'll get into the actual mechanics of redirects, the 301s and 302s, another day.
Cherry Prommawin: Okay. What's the next step in the framework?
Gary Illyes: The next step applies if the page is permanently gone and there is no true successor. For example, you stop selling the item entirely and there is absolutely no natural replacement. In that case, let it be a 404. It's fine. It is what we expect.
Gary Illyes: Return a 404 or a 410 status code. The crawler drops the dead URL from the index, and the website life cycle stays completely healthy. Don't fight the natural course. Let the dead page pass on peacefully. Let it go. Let it go.
Cherry Prommawin: Okay, but what about the human users who do happen to stumble onto that dead end?
Gary Illyes: That's step three, and that should have been yours.
Gary Illyes: Make the 404 page as helpful as possible. Instead of hiding the error from humans, embrace it. Design a custom 404 page with a friendly message, a search bar, and maybe links to your top categories or latest content.
Gary Illyes: Don't trick the user. Just give them a clean map to find their way back on their own terms.
Cherry Prommawin: Gary, you're lacking a bit today. What's the hold-up?
Gary Illyes: I don't know. I feel like it's really hard to think without the chicken around. I miss chicken.
Chapter 6: Myth Busted — Do 404s Drain Your Crawl Budget?
Cherry Prommawin: Gary, when you're back from daydreaming, let's tackle a classic myth.
Cherry Prommawin: People assume that if a site accumulates thousands of 404s, those digital ghosts are actively draining that crawl budget.
Gary Illyes: Well, that's just not true. A 404 error does not drain your crawl capacity. Many people think that if our crawlers spend energy knocking on dead doors, Googlebot will lose interest in the rest of the site over time. We won't.
Cherry Prommawin: Yes, so 404s don't actually affect the crawl budget. It's perfectly normal for the web to have a digital graveyard: removed pages, sold-out products, or articles that have passed on to the great archive in the sky.
Gary Illyes: To be more blunt, Google understands that 404s are normal on the web. So when we encounter a 404, it doesn't drain the site's crawl capacity or crawl budget.
Gary Illyes: This is unlike HTTP 500-class errors. That is the technical equivalent of your server becoming a burning building.
Chapter 7: Why 500 Server Errors and 429 Limits Are the Real Monsters
Cherry Prommawin: Precisely. A 500 error affects crawl budget because it tells us the server is gasping for air. It's essentially saying, "Stop knocking. I'm on fire." So we slow down to avoid making things worse.
Gary Illyes: HTTP 429 does the same thing. Actually, even if that's a client error code, that's your server telling Google that we're sending too many requests. We treat that like a 500 because, frankly, it's the same thing.
Gary Illyes: Basically, you are telling us, "You're too much for me right now."
Cherry Prommawin: Gary, you can take that cowboy hat off now.
Gary Illyes: No, I like it.
Chapter 8: Recap and Gary's Cowboy Moment
Cherry Prommawin: Recap time.
Cherry Prommawin: All right, all right, all right.
Cherry Prommawin: First, 404 status codes are technically normal and won't cause sitewide penalties.
Cherry Prommawin: Second, fixing 404s is completely optional. It's a user-experience and business choice. It's less about Google Search.
Cherry Prommawin: Third, 404 URLs do not drain your crawl budget, so don't panic over your crawl reports.
Cherry Prommawin: For more reading about 404s, go to the link in the description. And if a page is gone and has no replacement, let it 404. If it has a natural successor, redirect it. Just grab the towel and don't panic.
Gary Illyes: Cherry, look who's home.
Cherry Prommawin: Chicken!
Gary Illyes: Welcome back. Next time, we will discuss the messy and confusing world of redirects, moving URLs permanently or temporarily, and I might wear a sensible hat.
Gary Illyes: If you have a broken URL, a haunted server file, or a mystery error hiding in your crawl report, leave a comment below. Subscribe for more.
Cherry Prommawin: Say bye to your friends, Gary.
Gary Illyes: They all ghosted me.
Forum discussion at LinkedIn.
