Many SEOs are obsessed with pagespeed but as you know, mobile rankings don't even look at mobile page speed these days and they won't when the mobile first index comes out, at least not to start. Either way, fast pages are important for users even if Google doesn't have it in their rankings right now.
Google's John Mueller was asked if it is possible that YouTube embeds can slow a site's load and reduce the page speed score Google gives it. John Mueller said "maybe" because Google doesn't make exceptions for YouTube or AdSense or other embeds that they run versus competitors or third-party companies. He did add that YouTube embeds "shouldn’t be slowing things down too much" because "YouTube players are pretty well optimized." This was said around the 45:13 mark into the video.
Here is the video embed:
Question:
What’s really on my page is a lot of YouTube videos. And I am wondering, is that something that can be weighing down the page enough to affect the mobile page speed?
Answer:
Maybe, maybe. So from our point of view, we try not to make any exceptions for Google services. So whether or not you are using YouTube or AdSense or anything else on those pages, if that is slowing your pages down, then we will report it. So it doesn’t matter if it is a Google service or some third-party service or you are hosting the videos yourself, essentially, we would treat that kind of the same.Usually, the YouTube players are pretty well optimized, in the sense that just loading the page won’t necessarily blow your page speed down just because it has YouTube videos embedded. It uses a player where it loads the dynamic parts later so that you can click on it and then play the video.
So just by loading the page which has the videos embedded, it shouldn’t be slowing things down too much.
Forum discussion at Google+.