So we’ve had a couple of questions regarding our interview with Lisa Pearl and what she had to say about textual analysis and writeprints, the ways in which we signal who we are by how we use language. Like, for example, Gretchen McCulloch on All Things Linguistic asked about telling apart the different writing done by different characters in Rainbow Rowell’s Fangirl vs. Carry On. And in the YouTube comments, Valdagast asked about Good Omens, a book co-written by Neil Gaiman and Terry Pratchett, regarding whether we could use this kind of analysis to figure out which parts were written by which author.
I went ahead and ran these questions by Dr. Pearl, and I’ve got her answers below here! And I’m glad she answered them, because wow, this is not my area of expertise.
Let’s start with the various Simon Snow work. If you don't know, Fangirl is a book about a first-year university student, Cath, who writes a lot of fanfic about a series starring Simon Snow, a young man attending a wizarding school. In the text, we have Rowell's story, some (short) sections of Rowell writing as Gemma T. Leslie, the "real" Simon Snow author, and some longer sections of Cath's Simon Snow fanfics. Then, Rowell wrote a follow-up book called Carry On, which is a Simon Snow story, but not really as written by either Cath or Leslie; it's Rowell's own version of the story, not translated through any proxy characters.
According to Dr. Pearl, this is definitely something you could try to use stylometric writeprints for, at least in theory. See, the main limiting factor is the amount of text you have of each kind. (This is because you need enough of a sample to get the writeprint for that kind of text.) Just to give you a sense of how much data, in the datasets from her epistolary novel study, the character who had the smallest amount of text probably still had about 90,000 words. For another study, she and colleague Mark Steyvers looked at deception imitation attacks, where people attempted to imitate a writing sample by Cormac McCarthy. The training data for Cormac McCarthy’s own true writing was about 80,000 words, and they were able to easily tell the fake-Cormacs from the true Cormac using stylometric writeprints. However, if you have less than that to learn from for any one type of text, things might get tricky.
So (speaking for myself here), with regards to Fangirl and Carry On and the various Simon Snow writers, I imagine it’d be very challenging to actually pull it off. Comparing Carry On and Fangirl in totality could work, but Cath’s fanfics and particularly the portions written as Leslie are, from this report, going to be too short to work with. I haven’t counted, but I doubt even the sections of Cath’s fanfic you get are more than several thousand words in total.
Okay, but what about books that are actually written by two authors? Gaiman and Pratchett both wrote about half of Good Omens, but we don’t really now which parts were whose. Can we tell using this kind of analysis? Let’s go back to Lisa, who says that she suspects this might be tricky (additional editing to make the two of them sound potentially more similar aside), but not theoretically impossible.
The specific problem reminds her of a technique that Steyvers helped develop, called topic modeling. Using topic models, he can automatically determine which words or phrases within a document belong to which topic. For example, he’s shown that even within the same sentence, certain phrases come from one topic while other phrases come from another. Along with some other colleagues, he’s used topic modeling to identify which author wrote a particular scientific paper. The basic idea is that certain topics are more likely to be in the writing of one author vs. another. So, theoretically (and she wants to emphasize the theoretically part — the exact details of this are non-trivial), we might be able to use this to separate out chunks associated with topics by Terry Pratchett vs. chunks associated with topics by Neil Gaiman.
On the stylometric side (i.e., using writeprints where all the features are style-based rather than content-based), she believes something similar might be possible (where we’re looking at pieces within a larger document like a book). Certainly it’s easier to implement a stylometric model as long as we have plenty of Terry Pratchett-only and Neil Gaiman-only texts to learn from, and there, we are lucky, because they’ve both done a lot of writing.
So thanks again to Lisa for helping out answering these, and for making the boundaries of what we can do clearer. It’s really cool stuff. ^_^
