As a learning mathematician, I want to try and explain the deal with math, see if I can make it clearer. But I'm not really sure what makes this counterintuitive to you? I'll try to say some things which feel relevant and please tell me if that adresses the confusion at all.
It seems to me that you see math as something about numbers. That sudoku couldn't be math because the numbers don't matter there, you could write any old symbol and sudoku would stay the same. And you're right! In that numbers, indeed, don't matter for sudoku.
But math isn't just about numbers. Modern math is actually mostly not about numbers. I'm not sure what exactly you know about math, but you probably had at school (or elsewhere) some interaction with geometric proofs: where you have a bunch of shapes and through reasoning you prove some facts about them: for instance, that if two triangles have two equal sides and the angles between those are equal too, you can move one of them around to coincide exactly with the other. So the shape of a triangle only depends on two sides and the angle between them. You probably heard a bunch of proof like that, and they're not about numbers at all!
I could give more examples but for that, please tell me what experience you've had with math: at what level were you taught it, what areas of math you had, stuff like that.
But the general point is that math is generally about speaking precisely about precise things. Math deals with all situations where you have a strict set of rules, and you want to see where those rules lead you. In geometry those rules are geometric axioms (stuff like "for any line and any point outside it, you can draw exactly one line parallel to the original through the chosen point"), in arithmetic they're the rules for addition and multiplication (there's a lot of theory to explain exactly the rules, so I won't distract us with that), and so on. Math is when you say "right, so here's a bunch of objects and here's when we call two objects "the same thing" and here's what we can do with them. What conclusions can we draw?" Numbers are just one kind of object that can be described this way: very useful in many areas, but for from the only useful thing to think about in this way.
And you can see that sudoku fits the description perfectly. We have an object - a 9*9 board that's partially filled with characters. There's 9 characters to use, usually they're the digits from 1 to 9. You're right to say that if we replace the digits with some other symbols, it's "the same sudoku" - what matter is that every kind of symbol is in the same cells as before. And we have rules: what board counts as a correctly-filled sudoku and what doesn't.
And this is exactly the kind of thing math is made to answer! You could ask questions like "can we always tell if a board is filled correctly or not?" (obviously yes) and "if there's a partially filled board, can we always tell if you can fill it correctly?" (yes, we can try every single way of filling empty cells and check each of them if it's correct) and "how many operations does it take to do that? can we avoid checking this overwhelming number of boards?" and "can there be multiple ways to fill the same board?" (yes; for example, every correct sudoku is a filling of an empty board) and "for a given board, can we tell how many ways to fill it there are? how?" and "is there an algorithm that correctly fills every board that can be correctly filled?" (you probably know much more about this than me)
This, by the way, does mean math is an abstraction. That's what rules basically are: abstractions, where you talk not about some specific sudoku board, but about every single sudoku board you can think of. Not about some specific triangle made out of sticks, but the general properties that make up a triangle, whatever it may be made out of, even if it's merely imagined. You say math is an exact science that can give exact answers, and you're right. But it's exactly this abstraction that gives it such power: the real world is riddled with details and measurement errors and complications, so you can only know anything approximately. But if you abstract those away and only keep the rules of the game, then you can be sure rules apply always and fully, and therefore, you can be sure about your conclusions.
You also mentioned that we can't always have precise answers about infinite sets. Why do you say that? We very much can, there's a lot of cool things we know about infinite sets! For example, the rules for arithmetic: they apply for all numbers, and there's infinitely many of those. Or that favt about triangles I mentioned earlier: there's infinitely many triangles, and it works for all of them.
So, hope you tell me how you feel about these ideas, maybe you can explain more specifically what makes you feel such a dissonance about sudoku being math. There's a lot I could say on the matter.
