#mxao troubleshooting

[pictureamoebae]

TS4 and MXAO: Some Issues?

I was recently contacted by someone who had been experiencing issues in TS4 since the stairs update, where MXAO was disappearing when they moved their camera up, and reappearing when they moved their camera down.

I’ve updated my game and started a new save with no mods and checked myself (because I haven’t played since before the stairs update). I can confirm I’m seeing the same behaviour.

If any of you have used MXAO with the Indirect Lighting (IL) option in the past you’ll be familiar with how the IL effect can change rapidly when moving the camera. This is because ReShade and MXAO make hundreds of calculations constantly, working out where solid objects are, where in the distance they sit, how far they are away from other objects, and the angle from which we are viewing them. With this information MXAO renders Ambient Occlusion (AO) and IL to create semi-realistic shadows and light reflections based on how objects interact with each other to block light or reflect it. With IL, it works to bounce light off one object onto another, and as we move the camera, even just changing the angle, how light will bounce and reflect changes -- and so we sometimes see rapid changes in the strength and colour of the reflected IL.

It seems that the same is happening with the AO part of MXAO now too. It may well have always done this (it’s been a while since I’ve played so I don’t remember), but perhaps a recent update has done something to the game’s depth buffer (which is what ReShade hooks into in order to let us use depth effects like MXAO) making those calculations more sensitive. I can’t give a solid reason why it’s happening, and I don’t know if it’s affecting everyone and every version of ReShade (I use 3.4.1), but I can understand why it would be annoying.

I had some relief from it by editing my graphicsruls.sgr file to turn off ssao and dof, but it was still happening. One thing I noticed was that it seemed to affect shadows in the distance more than in the foreground, and between objects that have a wide distance between them. Now, in reality if I hold a cup in front of me I would not expect it to cast a shadow on a hill in the distance, but the limitations of ReShade mean that sometimes that can happen with MXAO if the settings aren’t tweaked to avoid it (look for the tool tips that talk about viewing angle and self-shadowing). What this new behaviour seems to be doing when it weakens MXAO when moving the camera up is actually applying a more realistic MXAO, so that cup in front of my face isn’t shadowing the hill in the distance any more. With the camera moved up, I still saw nearby foliage shadows but distance shadows disappeared or became weaker. 

I know of no solution at the moment that will give a consistent MXAO at all camera angles. Some things I suggest, however:

edit graphicsrules.sgr to turn off ssao and dof

play around with the MXAO settings that relate to viewing angle and self-shadowing

get used to editing your MXAO settings regularly as you set up shots -- I’ve always done this, and it’s a necessity if you use exaggerated AO and IL settings anyway, but it may be a necessity with more subtle settings now too

I’m sorry if this is affecting some people’s enjoyment when using ReShade. If I find another solution, or at least a way to mitigate the worst of it, I’ll let you know. In the meantime, I hope you have a bit better understanding of how MXAO works, and why it may be doing this, so you can try to work with the shader yourself and get the best out of it in difficult circumstances.

If you’re not experiencing this behaviour, it would be useful to know the version of ReShade you’re using, and also your MXAO settings. Perhaps we will be able to narrow down what’s going on in that way.

[pictureamoebae]

I have tried several different TS4 reshade presets and setting various settings but cannot get MXAO to work properly, there are black bars/white bars from IL stretched across the screen. I have a nvidia 1070 GTX graphics card, and I have no idea what is causing this, any ideas?

I’m sorry you’re having problems getting MXAO to work.

First thing I would check is whether you have the depth buffer set up properly or not. In versions since 3.1.1 it’s set to be reversed, but TS4 doesn’t have a reversed depth buffer, so we have to change that setting back. I made a tutorial on how to do that. It’s quick and simple to do.

Secondly, make sure to double check that Edge Smoothing is turned off in your in-game graphics settings. (Even if you’re certain it is, double check–I had someone once say it was but when they checked again it was turned on.) Edge Smoothing must be off in order for depth-based effects to work properly. Usually they just won’t work at all while it’s on, but sometimes it might cause glitches instead.

If you’ve checked both of those and are sure everything is set up as it should be, the next thing I’d check is that you have any preprocessor definitions relating to MXAO and IL set up. These settings can be edited either manually in the mxao.fx file (they are in the top-most section), or you can add them to the preprocessor definitions box (the same box you looked at when checking to see if the depth buffer was reversed). 

You can see a list of those definitions as they should look in the preprocessor definitions box at the top of this tutorial. Sometimes different versions of MXAO will have different preprocessor definitions that you can use. You can check which are applicable to your version  by looking at that top section of the mxao.fx file (found inside reshade-shaders > shaders).

If you’ve gone through these steps and you’re still having problems, please send me a screenshot of what you’re seeing in-game, and I’ll see if I can work out what might be going on.

Edit: I forgot to say, make sure Laptop Mode is also turned off in your game settings. It can interfere with MXAO.