There is an interesting tidbit you can learn about Google's web rendering service (WRS) in the latest JavaScript SEO hangout with Martin Splitt of Google where Martin said he is "spilling the beans a little bit" by sharing that there is a layer on top of WRS that handles "prioritization, error retry, quality of service guarantees, caching and it also interacts with a crawling subsystem to actually fetch things."
To be clear, Martin seemed to know what this was at a high level and did not go into too much detail on this layer. He said "I can't say the name, but there's like an... it's larger than the WRS." He added "the WRS really is just the rendering service and then there's like a slightly bigger layer on top of it and that actually has prioritization, error retry, quality of service guarantees, caching and it also interacts with a crawling subsystem to actually fetch things."
He mentioned this at the 16:30 mark into the video when he was asked "Are we talking about dynamics serving looking at UA WRS caches resources and that's okay but do you cache url using UA as a key?"
Here is the embed at the start time followed by the transcript:
Are we talking about dynamics serving looking at UA WRS caches resources and that's okay but do you cache url using UA as a key?That's a question that… I love the questions that are coming because that's the question that I don't have the answer to. And that means that you've just given me a taskfor the next couple of days because I think they might have separate caches but I don't know because caching does happen within the crawling infrastructure but I don't actually no no no that's not that's not true.
That specific cache that we're talking about because you're talking about WRS. WRS has its own catch which is interesting because so the architecture, just like spilling the beans a little bit.
The architecture is roughly that there is the WRS which caches nothing and has no network interactions whatsoever. And then there's like a layer around it, that's what the, I can't say the name, but there's like an it's it's larger than the WRS. The WRS really is just the rendering service and then there's like a slightly bigger layer on top of it and that actually has prioritization, error retry, quality of service guarantees, caching and it also interacts with a crawling subsystem to actually fetch things.
And I don't exactly know how this caching layer this caching component of that larger thing that wraps around WRS is actually organizing its cache.
So that's a really good question. I would expect it to be different per UA but I don't know.
Forum discussion at YouTube Community.
