Back in the day, there was a difference between how Google responded to a 404 vs a 410 server header response code. 410s were faster and more permanent.
It seems as if that may have recently changed.
A Google Webmaster Help thread has Googler John Mueller giving us an update on the differences, or lack there of, between the 404 and 410.
He said, "even back when we differentiated between the two, the practical difference was just a very short time-difference, the overall process was the same."
To be honest, it isn't 100% clear if Google stopped differentiating between the two but it seems they have.
Why have they? I assume because most sites probably don't bother 410ing and just 404, at best.
Forum discussion at Google Webmaster Help.
Update: John Mueller got back to me to say he was wrong about this, he told me:
It's good to double-check these things, and it appears I was wrong there -- we do treat 410s slightly differently than 404s. In particular, we'll sometimes want to confirm a 404 before removing a URL from the index, and we tend to do that faster with a 410 HTTP result code. In practice, the difference is very small, and it's not critical to use a 410 if a URL is permanently removed. If you want to speed up the removal (and don't want to use a noindex meta tag or the urgent URL removal tools), then a 410 might have a small time-advantage over a 404.