We know Google has an HTTPS Ranking Boost. Initially when it launched, all it checked was to see if HTTPS was in the URL, but it didn't check if it was valid or working. A couple months ago, Google began sending out notifications of SSL certificate issues, which shows they may start validating it before giving the ranking boost.
John Mueller said on Twitter that self-signed certificate, which is an identity certificate that is signed by the same entity whose identity it certifies, would typically be flagged as invalid by Google's detection tools. He said, if it works in a browser, than you are safe, if not, then not.
@Ebrarkahn Self-signed usually won't work in browsers, so we'd flag that. I'd use https://t.co/IHdqVIuJjB to check the current state.
— John Mueller (@JohnMu) January 6, 2016
Bottom line is, if it doesn't generate an error in a browser, it should work for Google as well.
To be safe, don't go with self-signed certificates.
Forum discussion at Twitter.