I have a REALLY hard time with geography and directions and distances, even in places I know really well. I'm working on a fantasy novel, and I know world-building is a huge part of fantasy, but how much does setting really NEED to be developed? No matter how many guides I read I just CANNOT create a map because I have zero sense of distance/direction/scale/ANYTHING RELATING TO GEOGRAPHY, but I can't just ignore it in my novel. Any advice?
Setting and world building do need to be developed, but neither of those are necessarily talking about geography. Culture, descriptions of a place and climate, the people (i.e. a non-human race) - these are integral to world building. Unless it's something really integral to the plot like the hero undertaking a physical journey, I would say hammering out all the geography nooks and crannies could be something that's easily left until your second draft. And in the end, maps may have been a main feature in High Fantasy settings, but nobody has made it an explicit rule that you have to have make one. For now, I would work on the story and if you find it necessary, either find an editor/beta writer/someone who has a better idea of these things to help you afterwards. And when that point comes, we have a few links to help:
World Building Considerations
World Building: Map Generators, Software and Tutorials