Oh this is very much a known and oft-lamented problem in science. You're far from alone here. The main reason I left science is because I couldn't handle the politics of it; a lot of what you're doing is trying to get your research looking important enough so you can qualify for grants and pay rent, or trying to release enough citeable papers to be important enough to get grants and pay rent. How "good" a scientist you are depends a lot on a) where your papers are published (which is largely a grift from the prestigious papers who take money from everyone involved and give very little back) and b) how much your papers get cited by other scientists (which means there's a lot of networking and result-generalising and stuff to get those numbers as high as possible so you can keep making enough money to live and do more science).
The most dangerous part of this sort of publication, though, isn't academic dishonesty and exaggeration (which is ferretted out by other scientists after a few years, who tend to keep an eye on each other and know all the tricks) or missing something in the unpublished data (which is always going to happen). It's the opposite -- accidentally inventing trends that don't really exist.
In science, there is always the possibility that your result is random chance. I could give ten tomato plants fertiliser A and ten fertiliser B and mix them all up and it's possible that all ten fertiliser A plants will do great and all ten fertiliser B plants will die by completely random chance, even if fertiliser B is better, or even if both fertilisers are exactly the same. Most non-medical sciences have a confidence interval floor of p=.05 -- that is, the chance that your result was just coincidence has to be one in twenty (5%), or lower. (In applied medicine they're usually stricter).
So, as an example, let's say that 40 people do experiments that involve MRIs. One person might be checking the effect of sugar on the pleasure centres of the brain, one might be looking at some new medication, one might be pinpointing how imagining stuff translates into brain activity; it doesn't matter. Beforehand they do some baseline tests, and one of them finds, completely unexpectedly, a major difference int he baseline brain activity of their male and female subjects (p=.025)! P=.025 is pretty damn confident! They have to publish that, of course they do! So they do!
Running forty experiments, you expect this result if there is no difference, with a confidence interval like that. In context, this result means absolutely nothing. There's no difference. But the other 39 people aren't going to publish papers about a difference they didn't find while looking for something else, are they? They might mention having done preliminary tests on their subjects in the unrelated paper they do publish, and leave it at that. So what the scientific community sees is a paper finding a difference, and no papers contesting it.
Now, what you're supposed to do in a situation like this, is consider this preliminary and run another experiment. But this lab is studying something else with the brain. They threw this paper out because it's an interesting thing they found, it's not their focus. It's not anyone's focus. And a few years later, another lab does the same thing. Until you end up with a handful of pretty solid-looking papers about this difference between male and female brains, and nothing contesting it.
"Oh, but surely someone will eventually run the experiment properly and -- " and what, write one single opposing paper? Maybe someone will, if for some reason there turns out to be enough attention tot he topic to make it worth it. Maybe a few people will. Maybe someone will write a meta-analysis, comparing all the papers on the topic and diving into a few that weren't on the topic to try to find what the standard results are, trying to find the truth. It's unrewarding work, but people do it. In the meantime, ten years of popular science books have been written casually citing the "fact" that male and female brains think differently and it's been proven with MRI, and second and third generation tests have been built on this utterly failing premise to say things like "MRIs show that women are better at social stuff and men are better at physical stuff because [horrible misreading of the data from some such paper]" and "trans women's brains behave like female brains and not male brains in an MRI" (yes, it's still bullshit even if it's used in support of something good instead of something bad; bullshit science is bullshit science on both sides of the fence). And while the MRI stuff is being debunked, is the stuff about diet that help autistic kids being debunked? Or the differences in metagenomic analyses of the microbial life in different parts of the world? Or the effect of boats on dolphin populations, or whether the colour blue is calming for infants, or what types of music different fish can distinguish, or data on lion pride social structures, or whether or not almonds help with high blood pressure?
A lot of what we 'know' is confirmation bias, even in science, because it depends on what gets published, not what gets discovered.
