A hateful association between Blacks and monkeys or apes was yet another way that the antebellum South justified slavery. Blacks were considered by some Whites to be more simian than human, and therefore had no self-evident rights, including freedom. After the Civil War, the emancipation of slaves, and passage of the 13th, 14th, and 15 amendments to the Constitution, White bigots used the association to justify Jim Crow laws, and the use of violence, such as the lynching of Blacks who challenged or threatened the status quo. The general acceptance of the evolutionary theories of Charles Darwin was easily twisted into a means of identifying further "evidence" of the primitive status of Blacks.
The depiction of Blacks as apes & monkeys found expression in mainstreamed popular culture around the turn of the century, especially in postcards. Often it was the zip or urban coon that was being caricatured, for the amusement of White consumers.
Why people are mad is because stereotyped anti-Black imagery that depicts Blacks as less than human.
If you want more info why calling a black person a monkey/ape is considered racist, I will defer you to google to look up "The Coon Caricature."