(OPINION INCOMING) Some issues I have with the "Light should have won" and "L himself should've beaten Light" crowds are that, the obvious aside...
1) I sincerely believe killing L off was necessary for the plot and the best choice possible for his (and Light's) character.
2) Let's say Light really did end up "perma-winning" (which, to begin with, is undeniably unrealistic, granted, but this is fictionāspeculative at thatāand everything's possible within it). No more successors, no more opposition, no more nothing. He's on top of the world as he's always thought himself to be. This isn't news to him and, surely, nothing else will be anymore. Would it be a service to his characterāhe, who has monotonously been at the top for his entire lifeāto remain unchallenged and to lack any semblance of humbling for the rest of his life?
3) With the former two points established, had Light been defeated by L himself, it would have been massively unsatisfactory as it would've aligned more with Light's selfish personal wishes. I don't know about you, but I do not believe it'd be anything spectacular or unexpected to witness. It's no secret to us as spectators that Light, in his own way, held L in high regard, as a worthy match, the only person posing a challenge worth his effort up to that point, his one true rival. Maybe he never acknowledged so much explicitly, but such a dynamic, such a view was undeniable. L left a deep mark. We even see him essentially headcanoning what L would've done, had he been in Near's shoes. And, naturally, his analyses had hit the bullseye. Light couldn't help but increasingly compare Near to L, dismissively seeing the former as small, soft and lacking in comparison to the latterāof course, this is not to undermine Light's own overconfidence and the fact that he was under the sincere impression that he had completely fooled Near. Which, to an extent, he had, only making this worse. Meaning: it wasn't just Near, it also was himself, naturally.
What makes Light's death so satisfying and fair, because Light is the most unfair man in the world, is that he was not defeated by who or what he would've preferred. Quite the opposite; he was brought to an end by the person he had come to underestimate and gradually thought less and less of. The one he was fairly detached from, the one who dared to hide himself behind a literal mask of whom Light, succinctly, in his own way, missed and was fairly haunted by right in front of him. Namely, Light himself prematurely thinks of his "victory" over Near as an unsatisfying one.
I like to think of one of L's motivations for solving the Kira case as him not wanting his main hobby to turn basically uselessāthink about it; if Kira's going to completely flatten the crime rates, how is L possibly going to get his supply of excitement? Of criminals to dissect and study? Knowing this was a personal threat Light posed to L, it's no wonder he immediately decided that the Kira case was of instant interest, that Light was of interest. Someone he saw some of himself in, for better or for worse, while also witnessing how radically yet complementarily different they were. Of course, had L defeated Light post-Yotsuba arc, it's safe to assume that he, too, would've thought about the Kira case, about Light, for years on end. Light and L were simultaneously the worst and best thing to happen to each other.
Back to Near. He lacked, in his complete favor, this sort of connection with and interest in Light, and vice versa (& to a chagrin Light would rather not put into direct words, in a sense). With this, it was far easier for Light to see himself consistently frustrated (and ragebaited) by this "rival" he came to deem a pebble in his shoe by the later instances, not to mention how he had completely declared the Task Force to be even less than a secondary threat. It is thoroughly necessary for Light, especially second half Light, to be brought back to Earth. He had gotten too cocky, too confident, too careless (no matter how much people love to blame Mikami for Light's failure, it absolutely also was on Light for lacking further foresight and communication. Emergencies are possible, Light!), especially after his father's death. The absence of the "one and only match" type of bond that Light and L had (whether they'd will themselves to accept it or not) completely fueled Light's negative and dismissive views on Near, therefore resulting in a complete, thorough and above all deserved humiliation when Near successfully flips the tables on him, because of him. Not only that; in the Yellowbox Warehouse, Near gracefully concedes that Light did manage to have the major upper hand at some pointāand he also acknowledges that he, on his own, cannot quite match L. Near had no qualms with admitting such things, just like he had no qualms with telling Light right to his face that he was nothing but a deranged murderer with a power far bigger than he was, laying out the fact that they truly weren't all that different from each other to their core. Coming from a person who Light at that moment thought of as nothing more than a joke (which is funny; initially, Near seemed to be a worse threat than L in his eyes and he naturally seemed pretty excited by the prospect of a new challenge), this clearly must've struck a nerve; he continues to think of the people around him as hopeless fools who would never get it. Oh, but they do know what he means; they just don't subscribe to it. They never would.
And then he gets shot, multiple times at that, by the goddamn comic relief. If that isn't just beautiful then I don't know what is. Matsuda couldn't have been less of a real problem to Light. Light saw him as disposable, malleable, naive, easy, foolishāyet Matsuda was also the one who was ever constantly willing to give Light the benefit of the doubt, to trust him, to believe in his innocence. He was comfortable enough with Light to sleep around him. With Aizawa having no reason to be surprised or angry anymore, it's even more delicious to see Matsuda of all people be given his deserved spotlight and to have him be the one to put Light in the most literal pain he's ever been in. Light even has the nerve to continue attempting to get Matsuda to see eye to eye with him, to absolutely no avail, and to Matsuda's further rage and disillusionment. And this is without bringing up the monumental topic that is Soichiro Yagami.
The piece de resistance is that Light eventually begs Ryuk to kill everyone. Deliriously and desperately, he thought of the possibility that even a God would bend to his will. But he didn't. Light saw himself defeated and humiliated from every angle. No L to satisfy his longing for a challenge; no person left to believe in him; the person to take him down was the one he conclusively thought was inferior to L, inferior to him. When you live your entire life winning effortlesslyānot just winning, but consistently remaining foreign to the idea of failureāhow could you take such a thing in stride? How could you possibly expect that such a thing would ever happen to you? That you had successfully overestimated yourself, even when you never thought it was baseless in the slightest to have comically large amounts of confidence (and... frankly, throughout his life, he never had any reason to doubt himself), and been completely humbled by what you thought was lacking in comparison to you? And that, in the end, you got a taste of your own medicine, your very fearādeathāfrom the very entity who promised it to you from the start, right in front of everyone, as it demonstrates that you were always a mere mortal just like everyone else? That your role in his life had gone beyond obsolete the moment you begged it to save you?
Light deserved that far more than he deserved to be beaten by L or than he deserved to win; it was only fair. It was needed. Light should not have been given any leeway to continue. He should, and he did, have seen himself fall completely because of what was unfair and beneath him in his eyes and, of course, because of himself. In fact, this fate was merciful to his story. No matter how I see it, I fail to believe that Light would've been truly, wholly satisfied with any life he had been able to live.
He always wanted more, and it always was bound to work against him.