Ok, I've been turning the Hagen-Clamenza-Tessio meeting in my head overnight and come up with a few things
1) Having realized and planned the upcoming transition to Michael's rule… these three men, his closest and most reliable courtesans, now become his biggest threats and liabilities. Transitions are where it makes the absolute most sense to try for the throne, and if anyone's going to pull it off it's one of these three. In a very real way, the Five Families meeting was an appetizer--here is where the true maneuvering happens
2) For the first part, before Tessio and Clemenza leave, he's actually encouraging shots at the throne--by telling them he will be semiretired and growing tomatoes, he's telling them his influence is down and not going to stop shrinking, he doesn't have the heart for this business anymore. If they're going to be a weak spot, he wants them exposed and out of the action soon
3) This is also why he pretends he is so amazingly mad at Freddie and wants him to stay in Vegas. Trying to break in with a resentful passed-over son is textbook overthrow, and he highlights it--possibly even keeping Freddie safer in the process than he'd otherwise be. (Unless he goes for it, of course.)
4) But Tom is a bigger problem. He flatly doesn't believe this put-on, and he is in a lot better position to screw him than the caporegimes. It's in the consigliere job description: "Between the head of the Family, Don Corleone, who dictated policy, and the operating level of men who actually carried out the orders of the Don, there were three layers, or buffers. In that way nothing could be traced to the top. Unless the Consigliere turned traitor." The Consigliere is the man with the keys, and there's probably no shortage of mobster fables Vito's witnessed where they used them.
The way things have been, he would never; Vito's shaped him, built him, and has all his buttons and levers down dead to rights. But that strength is the weakness in a transition. This creature you've built to care only about your wishes, to lay your own good as the center of his moral universe… what happens during this shift, where True North goes spinning out of alignment? It is difficult to predict, and messy. Tom cannot get wind while he can still mess things up, and he's too smart and involved not to pick up on it. ("You know me better than anybody else" develops a double-meaning.) So… you smother his brain, just a little. Some men can be distracted with girls or entertainments. Tom Hagen can be baffled by spoiling out just a bit more of that paternal affirmation.
It's important to keep him on the back foot, to state it more clearly, because Vito doesn't trust him to keep the Corleone's needs at heart absent his specific machinations to keep him there.
He's not family.