Why does James give so much credence to Liz's words as it's obvious that she dislikes Lily and is extremely jealous of their relationship and doesn't want them to be together, so it seems like James shouldn't really trust her view of the situation. And I think it's worrisome that he still lets her have so much control over his actions - like he has to defend himself and account to her for all his dealings with Lily- at some point he needs to establish way better boundaries with her.
It's not that James is just blindly listening to what Liz says--he knows that she is extremely biased, that she doesn't understand the situation, that she doesn't really know Lily...but the things Liz says aren't all entirely crazy, and Lily does a lot of things in the following days to have James doubting even more. These things tap into just about every insecurity James has about their relationship, so having to hear them from someone who he does trust at a time when everything else around him is pretty much going to hell...well, it gets to him.
His relationship with Liz definitely has boundary issues, but that's a bit of a bigger issue than just in relation to this specific incident.