i'm clearly not the only one to have this theory, but i really don't think Kaeya is literally a Cavalry Captain
we already all memed to death how silly it is to take all the cavalry but leave the captain - like either take the whole cavalry or leave a portion of it in Mondstadt to defend the city with the captain, and take the other half. take the cavalry without the captain seems like the worst possible choice
but more importantly - have you noticed how good Kaeya is at gathering intel and that he's pretty much doing it anytime we see him?
but it wouldn't do to have a "Favonius Spy" or "Favonius Intel Gatherer" as someone's official role. it would make people wary around him and thus less likely to give him information
Yelan is similar, but in her case, she doesn't appear in any registries or files at all, and isn't very well-known to the general public. i suggest Kaeya is in a similar boat, but instead of being completely absent from the Knights' roster (might be harder as everybody seems to know everybody in Mondstadt, at least moreso than in Liyue), he's instead labelled as the "Cavalry Captain"
and maybe he sometimes acts as it, y'know so people will see him with the Cavalry and won't become suspicious. but his true role is to gather intel.
and that's why Varka didn't take him along: the Intel Gatherer of the Knights of Favonius would be better continuing to gather intel in the city than coming with the expedition.
