Gynephobia (gynephobe, gynephobic, gynephobiac, ginephobia, ginefobia), gynophobia (gynophobiac, gynophobic, gynophobe, ginophobia, ginofobia), gynecophobia (gynecophobic, gynecophobiac, gynecophobe), gynaecophobia (gynaecophobe, gynaecophobiac, gynaecophobic, gynรฆcophobia, gynรฆcophobic, gynรฆcophobiac, gynรฆcophobe) or phobogyny (phobogynist, phobogyne, phobogynic, or phobogynous): fear of women.
Androphobia (androphobic, androphobe, androphobiac) or phobandry (phobandrous, phobandre, phobandric, or phobandrist): fear of men.
Not to be confused with misogyny/gynomisia or misandry/andromisia (opposites of philogyny or philandry), these are the counterparts of gynephilia or androphilia. These are also clinical phobias. Based on this.