I could be entirely wrong with what I'm about to say, but this feels like he's [maybe not intentionally] making a dig at Gwen. Or at the very least, Gwen takes it as a dig.
Because when Brett dealt with Lena, her presence made him feel small, made the office feel small. Because she was strict and to the point, and didn't have time for idle chatter.
But with Gwen? Gwen "I'm Busy [not knowing what to do]" Bouchard? As much as she tries to emulate the same presence, it falls flat, because she is so out of her element. Her presence doesn't make Brett feel small, nor even the office they're standing in, so he can take the moment to look at the space around them. He can see how big it really is when there isn't any sense of power/authority coming from the person he's speaking with.