A huge reason why so many people hate sex workers is because people who are part of the reserve army of labor make up a huge amount of sex workers. This includes disabled women, trans women, queer members of the female sex, and racialized members of the female sex.
In other words, sex workers are often people excluded from the normal labor market. Some people may say, "Cool, so sex work is non-consensual." Yes, but that has more to do with class society than the nature of the work itself. Sex work is non-consensual for the same reasons working at Walmart and McDonald's is—if the alternative to working is not having your basic human needs met, while we live in a country that over-produces everything, then all work is non-consensual, not just sex work.
But doesn't that mean it's paid rape? No, rape requires direct violent force or direct coercion. If I buy a chicken sandwich from McDonald's, am I assaulting the fast-food worker? If I go to the doctor because I am in a medical emergency, am I assaulting the doctor? Secondly, the people who are saying this are radfems whose anti-queer stance helps create the reserve army of labor. Secondly, this assumes that no one would ever choose to be a sex worker if they were not forced into it.
On top of this, criminalizing sex work harms sex workers. If sex work is legal but consuming it is not, people who consume sex work might resort to violence to protect themselves. On top of this, it would reduce the income sex workers make. This is in addition to reinforcing purity culture.