Obviously the amount of realism you put in a D&D game is entirely up to you (and everyone else at your table), but I will say that if a particular monster is attacking the party because it wants to eat them, it would almost certainly try to run away and look for a less dangerous food source once the players do enough damage to it. I'd say that reducing it to half its max HP should be enough to get it to change its goals from "eating the party" to "running away"
(If I'm a big monster and I take a fireball to the face, the weedy little twink that threw it is not gonna be a good enough meal to justify the risk of that happening again)
Two exceptions to this would be if 1. it can't run away, for whatever reason, or 2. it's already starving... and if it's the latter, consider how that would affect its appearance and abilities. And whether any of the PCs might be able to notice those effects
And if the monster has a motivation other than "want food" then it might operate differently - and might have different conditions for ending the fight. For instance, a monster that's trying to chase the party away from its den will back off once they're far away enough; gods know what might happen if it keeps chasing you and leaves the den unguarded
A monster that's sapient and has Sapient Being Reasons for wanting to kill the party... is a person and thus can potentially be reasoned with. The DC may be high, but it's possible
The only time there wouldn't be a realistic way to get the monster to back off until either it or the party dropped dead is when that monster was specifically created to kill. Maybe kill anything it came across, maybe kill the party specifically, point is that if its main objective is anything else you can get it to back off by fulfilling that objective














