I think a large part of the reason Phineas is so patient is because he rarely has a reason to get angry. As he says in The Phineas and Ferb Effect, "We... have no frame of reference for that [when things go wrong]." Usually he and Ferb just try to have fun every day. And when they do find themselves in a sticky situation, they can usually invent their way out of it. They have those positive probability ions that make anything they try to do, succeed.
So Phineas gets frustrated and angry when life doesn't work out. When something is truly at stake. When he can't take the lead. Especially when he needs someone else to help, who isn't helping. We see that in situations like:
Candace messing up the power assignments in the Marvel episode
"It was all a dream! Christmas WASN'T canceled!"
A spider standing between Candace and saving the world from eternal winter
"You're a secret agent?!"
Him freaking out over losing Isabella to the pharmacists
Baljeet's mountain climbing adventure
And of course, "GET ON THE TRIKE!"
(Note that he's still fairly calm and collected in special episodes where he CAN lead a charge or help out in any way. It's only when he's NOT in control that he loses it. That's probably why Ferb is much more emotionally stable than Phineas, because Ferb prefers to be a supporter, while Phineas prefers to be the leader.)
That's also why I admire Candace, Milo and Doof a heck of a lot more: because they have the negative probability ions. Life has not been kind to them. But somehow, they still find ways to give more back to the world than what they got out of it -- Doof and Candace mainly with their families, but it still counts, and Milo with just about everybody.
But given how much Phineas idolizes Candace, and he has still gotten angry with her (albeit for very justifiable reasons), I think it is possible he would get angry at Isabella. Of course it would require her to either refuse to cooperate with him, which she has never done in 5 seasons, or to try to help but mess things up, which she has also never done in 5 seasons.
But it could still happen.