When writing a scene, make sure you always know who’s involved in a particular conversation. Even if someone involved is small random character and not particularly important, still know their name, age, basic personality, etc. Most of this information will not make it onto the page explicitly, but it will help you develop a believable dynamic. It’s not like in a large group setting the only people who say or do anything are the characters who play a direct role in the plot.
For example, if you have a character who is master of the house with tons of servants beneath them, try to know the servants’ names and general personality so not every servant is a generic “Character A.” Even if they never come back.
Sometimes, if you have a large group and certain characters of that group haven’t been introduced yet, still know everyone and make them behave like themselves.
You need to know who these people are, but the audience probably won’t care since they don’t know these small characters, so don’t necessarily show everything about them. Make sure they need to be present and participating but the narrative details, descriptions, etc. need to be weighted toward the ones that are important.