I daresay, that really does look like outright scorn. The reaction was instant- like Takumi didn't want Karua to be there, or to see him waking up like that. Though he explains later on that he just wanted an advance notice of her being there- no matter the time of day.
Part of the problem with being extremely close-knit to one another on the regular, throughout life, is a trouble with setting boundaries- which could explain the reflexive annoyance. Not being able to get enough space can sometimes sour even very close relationships.