Okay I apologise for this but I have… (more) thoughts? I hope it’s not obnoxious this is so long by now.
My brain just kept turning over things in terms of the basic “what we do get in canon” for Luke & Vader. Because fact is, with the limited screentime Vader has and the bits of that that is with Luke, most of it is spent fighting.
The first time when neither of them know who the other is, so this is pure-baseline Darth Vader as he’s existed for the last decade+ (or more, but we should give him some in-universe time to actually settle into what he becomes); goal oriented, ruthless, the enemy shall die because That’s What Has To Be Done in the service of the Empire/order/the Emperor.
The second time is when Vader knows (and later on during this when Luke also knows), and it’s already been pointed out how you can interpret Vader’s acting during the fight on Cloud City in different ways, but we also have the aftermath of the telepathic conversation through the new force bond.
Third, the second Death Star. Again, we can have multiple interpretations, all probably underblown by a certain amount of apathy/Vader having given up on any ambitions he harboured in those three years up to the climax of Cloud City. So we go anywhere from “goading Luke into turning to keep him alive”, kill him because he’s only been a tool (up until Palpatine tries to kill him, as that part is unquestionable) and Vader don’t actually want Luke to replace him at Palpatine’s side, to getting Luke to kill him because, again, that will buy Luke time and might keep him alive.
All of these of course with the additional ways Vader chooses to interact with Luke during the fights - in Cloud City, they are alone, and Vader’s riding high on what he sees as his inevitable victory; you have a blend of praise of Luke’s nascent skills and trying to get him to stand down in among the way he chooses to respond combat-wise. Said praise can be read as both genuine (and still some amount of manipulation!) as well as purely calculated with no actual fatherly emotions behind them, because canon gives us nothing - in isolation, that is!
(Because we have to account for Vader and Palpatine’s earlier discussion about Luke here; there’s no real reason for Vader to argue that Luke is “just a boy” and get official sanction to try to turn him. Palpatine now knows, but what will he know of what Vader might do in the immediate moment he may meet Luke, unless there’s witnesses? How is he to know if Vader immediately kills the boy (which could weigh equally between what a good Sith would do, and what Vader shouldn’t do as “a good Sith” since taking Luke as a secret apprentice for eventually killing Palpatine is all within good order and does not require Palpatine’s knowledge or consent - hell, it SHOULD be a secret, if Vader was a “proper Sith”.) or tries to talk him to his side/whatever you want to argue might be his motivation?
Yet Vader chooses to go for “please give me sanction to do this we do not need to kill him he’s JUST A BOY.” Sir please, you are a Sith, perhaps act like it?)
In the DSII fight Vader and Luke aren’t alone, and Vader is doing a lot of taunting to get Luke to genuinely fight him, so you have freedom to go between that this is a play all for Palpatine’s sake, that Vader is genuine and doesn’t care and is doing it to hurt Luke, etc. Many choices, all equally possible.
(Again, you have another conversation between Palpatine and Vader to add as context; “he will be here/he is on the moon, I have felt him.” and “strange that I have not, are your feelings about this clear?” I’m sure you can do several different takes on this conversation as well, but it still stands here as context and adding complexity to anything Vader chooses to do during the fight prior to Palpatine trying to kill Luke. There is a genuine connection between him and Luke, one which clearly worries/displeases Palpatine.)
The only time Vader and Luke are actually talking without fighting prior to the DSII climax is on the bridge in the Imperial base on Endor, and they are alone here. Vader deals with this conversation by half avoiding Luke’s points (delaying addressing the point of bringing Luke to the emperor by focusing on the lightsaber for example) and simply cutting the conversation short when it becomes too uncomfortable (sending Luke off to spend the rest of the night in a cell after Luke goes ~then my father is truly dead), and this conversation, like pointed out earlier, has to be viewed in the light of the fact that Vader is lying - to Luke as much as himself, for he sacrifices himself for Luke and shows Luke is right.
None of this gives any huge pointers to how they could interact in a more prolonged relationship, nothing that is any more or less IC or OOC no matter what your preferred interpretations to Vader and Vader and Luke’s relationship are.
I think it’s pretty damn great that, even with fanfic being fanfic and that we can do whatever the hell we like, wanting to be as IC as possible while indulging our wants or going more crack, there is absolutely support for either a softer, more fatherly bent on Vader’s actions, and a more callous/ambitious and cruel one.