John Mueller from Google said on Twitter that you don't need to add alt text to all your images. Specifically, adding alt text to small images used for positioning stuff on your pages or placeholder images are not necessary.
Obviously, you should add alt text to important images, images that are part of your content, all should have alt text. I know it takes extra time and resources but sometimes it is worthwhile.
Here is what John said:
@GeorgeAsimakos Should be fine, but no need for alt texts on tiny images used for positioning / placeholders) regardless.
— John ☆.o(≧▽≦)o.☆ (@JohnMu) April 5, 2017
John then added that you should also make sure that your normal alt text also shows on your mobile site. He added that "many mobile sites skip over that completely," which is not a great idea.
@GeorgeAsimakos FWIW (kinda related) alt-text makes sense & is important on mobile pages too; many mobile sites skip over that completely.
— John ☆.o(≧▽≦)o.☆ (@JohnMu) April 5, 2017
He made no mention of rankings here.
Forum discussion at Twitter.