That's why he is gonna go from beach city at the end of Future. The whole SU plot shouldn't have happened anyways, it's not his world, Yes he has powers, yes he has a gem in his belly, but he is a human, he wasn't destined to be a CG, he wasn't destined to be the replacement for Rose Quartz, he was destined to be Greg's son and to have his own life. I am certain Rose would be FURIOUS with the gems for what they expect Steven to be, because she didn't want his son to be a Crystal Gem she wanted him to be a human being.
"...you're going to be something extraordinary, you're going to be a human being."
It's all in the tape she made, there is no hidden message, there is no magical destiny (as stated in the episode itself) there is only Steven, no Crystal Gems, no Rose Quartz, just a human.
Did you notice that we never, and I mean NEVER, see Greg's grief in the show? It's because we see it from Steven's eyes, from Steven's perspective and Greg never shows Steven what he doesn't need to know, but in the other hand we know almost every problem and trauma the gems have. I kept thinking while I was seeing the video "Why did nobody talked to him about what was he feeling" "Why nobody realized that he was just a kid, and didn't have to deal with any of that", because they are aliens, they didn't even know what a child was, HOW COULD THEY RAISE ONE?, I mean at the moment of their creation they have a purpose and are capable of doing complex thinking, they are super smart and kinda like Adults, innocent adults that can deal with problems, and Connie was a child as well so no fault on her whatsoever, she couldn't even realize what was happening. I only wish Greg was more in his life, and that he understand that Steven is not just a Crystal Gem, it's also a human and not just a human, but a human child.
Greg made a mistake, the gems made mistakes, and in doing so they Raise a child to be the fucking saviour of the galaxy, but at the cost of the child's childhood.
I kinda get what he goes through (I just said kinda like the base of it, not in it's entirety) and I understand why he has to go.