Having bought and read The Viscount Who Loved Me, having only watched the show, never knowing these books existed before watching Bridgerton, I gotta say, I have no what what people are on about complaining that the show doesn't stick to the book enough. I fully agree that I think there could have been something different done besides sister drama, but literally everything else in the show was superior to the book, besides maybe wishing Anthony and Kate had spoken a little more in the library scene.
In the book, there are several instances where Anthony is just a full blown mean, abusive asshole, who sexually assaults Kate, kicks her in the stomach on purpose, complains about his right to sleep with her after they are married, and is all around a very bad dude. I'm so, so glad they changed the majority of the book. Had they followed the book, I would not have watched.
Edit: Also! Had they followed the book, it would have just been the same storyline of season 1, one of two people caught in a garden and forced the marry?! I don't get it.