I am very talking out of my ass here, just going with books I've read rather than any actual research, but my theory:
Fae Romantasy comes down to Sarah J. Maas. Maybe (probably) there was more of it going on before her, but she mainstreamed it and got to define the tropes. From that we get fae as sexy, powerful, sort of primal people referred to as males and females, who have soulmates, often look down on humans, use magic, and have an elaborate structure of monarchy and nobility. Sarah J Maas had a successful YA fantasy series that abruptly pivoted in book 3 to include fae, and then her next series was fae romantasy from the start, and also caused incredible discourse due to having explicit sex scenes in a book marketed as YA.
But where did she get this fae archetype from? My argument would be that prior to being romantasy characters, fae were urban fantasy characters. Jim Butcher gets mentioned here for possibly codifying the summer/winter court structure, and also just having a bunch of humanoid human-sized fae nobles in his Dresden Files books. But IMO the stronger connection would be Holly Black.
In 2018, post Sarah J Maas fae romantasy wave, Holly Black publishes a YA fae dark romance which has many many elements that seem recognizable to existing fae romantasy. A human girl raised in the fae realm, a fae prince who hates her even as he can't resist her, lots and lots of court politics and power dynamic swings. The difference here is that Holly Black has been writing these kinds of books since 2002 (which makes her earlier books old enough to have been influences on the beginnings of fae romantasy). She's maybe best known for her Spiderwick Chronicles series of children's books, which feature all kinds of creepy and gross fae creatures, which feels similar to older folklore. But at the same time she's also writing the Modern Tales of Faerie series, which are YA dark romances about humanish girls and the powerful (but vulnerable) fae boys they meet. Notable here is that the fae here are not monolithic in species: you've got humanish fae (iirc most main characters are in this category), sure, but also more classic creatures like trolls (I remember there being others but not the specifics).
Notable for these books is that they aren't secondary world fantasy: iirc the Modern Tales of Faerie books are set in New York. There's also a sensibility about them that I want to describe as punkish? The protagonists aren't relatable everywomen, or destined princesses: they're mostly homeless teenagers, squatting in subways and trying to survive on the edges between fae society and human society.
And so let's go one step further back. What influenced Holly Black? And here we have a definite answer, because she was co-editor of a Welcome to Bordertown, a 2011 remake/tribute to the Bordertown series, done as a collaboration between some of the original authors and younger authors, like Holly Black, who had grown up with them. The original Bordertown books were a 1980s series of anthologies, with each chapter a short story by a different author. They were set in Bordertown, a city founded on the edge of our modern world and a resurgent magical one, full of strange magic meeting modern technology, populated largely by outcasts and runaways. The summary I have pulled up describes Bordertown as "a place of half-lit neighborhoods of hidden magic, of flamboyant artists and pagan motorcycle gangs", which should give you an idea of the vibe.
This is very clearly the predecessor to Holly Black's Fae books. Only this is published in 1986, and so the magical world that Bordertown sits at the edge of is Elfland. And that's where I think the root is, taking folkloric elves, making them sexy feudal intrusions on the world, and then to avoid confusion with the better known elves of Tolkein, pivoting the name to fae. After all, older sources use the two interchangeably: if you look at variants of Tam Lin some of them have a Queen of Fairies, some an Elfin Queen.
A coda: I think Wen Spencer's 2003 book Tinker is illuminating here. It starts an unusual but modern young woman who meets a powerful, domineering elfin lord when he is uniquely vulnerable, then struggles between her attraction to him and the political and magical dangers he brings. The love interest here is very in line with romantasy fae males! But it's 2003, so he's still an elf, and the book is largely set in Pittsburgh.
annaxmalina
gummygunk
jacobvanloon
vaporwave
