By book 7 of Harry Potter, we know that James had a Stag Patronus and we know that dear old Snivellus had a Doe Patronus, confirming his undying feelings for a certain redhead by the name of Lily Evans. We know that Lily had a Doe Patronus because she fell in love with James, but the books never said, "LILY EVAN'S PATRONUS WAS A DOE." Harry just knew that Lily's Patronus was a doe because offfffffffffffffffff *drumroll* RON AND HERMIONE!!!
In the 5th book, it's stated that Ron's Patronus is a Jack Russell Terrier and that Hermione's Patronus is an Otter. Harry, being the one to teach them how to do Patronuses, must have known what their Patronuses were, despite being the oblivious brick wall that he is.
After Hermione and Ron first kissed, Harry must have been able to put two and two together (after all, he's not that stupid) and realized that Terriers chase Otters; Romione is meant to be! Because of this fact, Harry could conclude with absolute certainty that his parents really did love each other.
Harry knew that if he ever asked himself if Lily actually did hate James, in the end, the answer would be nah, she didn't.