Google's John Mueller said on Twitter "we generally don't use color as a factor." He then added "but CSS tweaks can include a lot more than just color changes." Here is that response in context:
We generally don't use color as a factor, but "CSS tweaks" can include a lot more than just color changes.
— 🐐 John 🐐 (@JohnMu) February 17, 2022
We know that making site changes can impact your rankings but John has said numerous times before, just making color changes should not impact your rankings.
Of course, if you move content around, hide content or something else with CSS, that can impact your rankings. This not just applies to the body content but also the main navigation and architecture.
Heck, I change the overall design of this site on some important days. Those types of changes should have zero impact on the overall rankings of this site. If they do, so bit it, I like making it fun here. But I highly doubt they do have any impact on rankings by me changing the background colors and tweaking the design of the logo.
Forum discussion at Twitter.