Normally the only people I'm at war with are the goblins, but now I have some elves and a big human civilization mad at me too. I think the elf civilization got mad at me during worldgen (doesn't matter because they only have like 2 sites... I think they sent their entire population to ambush me a couple years ago), but I pissed off the human civilization by killing a criminal that visited me because I didn't have any crimes to use to interrogate them at the time...
The goblin presence in the part of the world I reside in is quite low, too (there are like 3 sites within 3 days travel, and the largest one has about 50 population), but the human civilization that is mad at me is almost entirely within 3 days of my fort, so they keep sending these massive seiges. I had to actually set up a "junk in need of melting down" stockpile near my dedicated melting magma smelter because my usual strategy (just designate stuff for melting and let whoever picks up the job haul it inside) was actually too slow to keep up with the goblin+humanite influx