#practice safe research

[datamuse]

There ain't no License to ILL in the world of e.

[datamuse]

#4: That Facebook "mood" study. #5 really makes me wonder if all that money we're paying IEEE is worth it.

[datamuse]

A paper by Maggie Simpson and Edna Krabappel was accepted by two scientific journals

There are legitimate journals which charge author fees. These are not among them.

[datamuse]

Remember kids, when doing your research, watch out for confirmation bias! (From chainsawsuit.)

[datamuse]

Please, think of the kittens.

[datamuse]

Anatomy of a Research Tempest

On June 17, 2014, an article appeared in the Proceedings of the National Academy of Sciences and almost immediately garnered mainstream and popular news site coverage. It’s not unusual for PNAS-published research to get media attention; the site is open access and the research being published is often sexy in an attention-getting way. In this particular case, what was sexy about it was that the research involved Facebook doing the sort of thing people automatically assume that Facebook is always doing.

Everyone loves to hate Facebook, but so many people use it that it’s become the default platform for interacting with others online. Other sites use it to handle user logins, commenting, and networking, not to mention marketing. At the same time, the site’s obscure algorithms and perpetual-beta approach to usability provide useful hooks for the mass discomfort over how much personal information we’re giving up to a site that uses that information to make money. (As the saying goes, if you’re not being charged for it, the product is you.) Everyone complains about Facebook, but everyone uses it; it’s troublesome but it’s convenient, and as is so often the case, convenience trumps just about everything else.

Thus, the reaction to the news that Facebook researchers had intentionally manipulated some users’ news feeds to see whether what they saw on the site impacted their moods--as read from their subsequent posts on the network--was fairly predictable: a sort of unsurprised outrage that both acknowledged the ethical hinkiness of the manipulation, and classified it as business as usual. Of course Facebook was manipulating everyone’s news feeds. It’s the sort of thing that Facebook does.

More interesting was the view that the research study and the coverage of it provided into research processes that are to most people fairly obscure. It’s possibly the first time that many social media users ever encountered concepts like informed consent, institutional review boards, and various social science research methodologies. One of the issues raised at the outset concerned whether agreeing to Facebook Terms of Service constituted informed consent to the study; while the ToS does state that user data may be used for research purposes, the consensus emerged that this did not include having what information a user was exposed to manipulated in the name of research. What stories a Facebook user sees vary widely and are only partially under the user’s control, so users had no way of knowing this manipulation was happening at the time.

A related question was whether the authors of the study, two of whom were affiliated with Cornell University, had secured the approval of Cornell’s Institutional Review Board to conduct the study. IRBs are supposed to prevent abuses of human subjects in the course of research; accordingly, a researcher working for an institution that has an IRB submits a description of the study to the board for review. However, in this instance, the researchers were using already-existing data provided by Facebook; therefore, Cornell concluded that IRB approval was not necessary.

Then the question emerged almost immediately as to whether the research had any value. The study’s methodology (a form of textual analysis previously used on much longer samples than the average Facebook post) was called into question, as was the whole idea of whether Facebook status updates really communicate any useful information as to the poster’s emotional state anyway.

What interests me about this event and events like it is the successive waves of information and understanding as word about the study and its implications spread. First out of the gate was the OMG! reaction as people communicated their initial shock over Facebook’s shenanigans. This reaction gained traction quickly, for reasons already outlined: Facebook is very widely used, but much of its user base is cynical about the platform’s methods and motives. (Unsurprisingly, a great deal of this early reaction propagated through Facebook itself.)

The more analytical opinions took longer, and emerged in other locales--initially, chiefly in the comments on the PNAS article itself, where a debate quickly brewed over the study’s research methods, whether the Facebook ToS constituted informed consent, and whether the study had (and needed) IRB approval. Commentary emerged on AV Club and The Atlantic addressing these questions, as well as on subject-specialist site PsychCentral. Some of these commentaries also spread, via Facebook, Twitter, and other social media, but overall this wave was smaller and engendered a lesser reaction in terms of size.

Facebook’s user base does not appear to have diminished at all.

It’s a frustrating phenomenon if you’re in the business of getting people to seek out the best information on a topic, and to thoughtfully consider what they read. The most troubling element here isn’t that emotion might be contagious across social media--the study is, at best, inconclusive. The most troubling element isn’t even that this research was conducted, though the circumstances under which it was conducted, approved, and published are problematic. The most troubling part for me is how little people seem to care, beyond that initial OMG! reaction, to find out what really happened. It’s a side effect, perhaps, of the drinking-from-the-firehose experience of living in the Information Age, though I question whether people were really any more thoughtful about what they read in the past--certainly they found their sources more trustworthy, but whether they actually were is a different question. There’s always some new incident, news item, or outrage coming along, and by the time the analytical writing comes out, people have moved on to the next thing. Some sources have responded to this by speeding up their own analysis--The Atlantic’s coverage is quite thorough, and came out relatively quickly. Too, discovering what was going on with this particular research hardly required a great deal of investigative journalism; all that was really needed was a close reading of the study and a few follow-up questions.

There’s been a fair amount of research on the emotional effects of social media. As with most human endeavors, the results are a mixed bag. This, however, seems to be the first study that attempted to manipulate people’s responses, rather than simply observing what they did.

It won’t be the last.

[datamuse]

How rumors spread via sloppy citation practices @insidehighered

Most of the times a faculty member has asked me for assistance in tracking down a citation, the source in which they found it referenced it incorrectly.

[datamuse]

A rough guide to spotting bad science -- from Compound Interest

If I've ever remarked that a particular study was bad science or shouldn't have been published, chances are it possessed several characteristics from this list.