#brown corpus

[visualizing-archive-data]

He Said, She Said

What are the most common verbs following he/she pronouns in a given text? Following inspiration from here (is it strange how just the other night I was thinking Jane Austen would be great text to test on?), I implemented this in nltk for the Brown fiction corpus, a collection of 29 texts.

First of all, I was a bit surprised that there were only ~ 200 two-word phrases starting with ‘he’ and ~ 100 starting with ‘she’ in the entire corpus... but this is just a training exercise, so let’s see what the most common actions for ‘he’ and ‘she’ to take are!

Looks like whatever the gender, people talk a lot, but women think quite a bit more! 🤣

[mreyus]

Screw the Brown Corpus

brown_dict={}

iterations=0 for word, pos in brown_news:     working_word=word.lower()     if working_word not in brown_dict.keys():         brown_dict[working_word]=[pos]     brown_dict[working_word].append(pos)     iterations+=1     if iterations == 25000: print 'one quarter done'     if iterations == 50000: print 'half way done'     if iterations == 75000: print 'three quarters done'     if iterations == 90000: print 'ninety percent' print '\nbrown_dict complete' for word, tags in brown_dict.items():     brown_dict[word]=set(tags) print 'tags are now in sets' pos_number=0 for word, pos in brown_dict.items():     if len(pos)>pos_number:         pos_number=len(pos)         word_most_pos=word         all_pos=pos print '\nThe word with the most tags is:','"',word_most_pos,'"', 'with', pos_number, 'tags, and these tags are:', all_pos #The word with the most tags is: " to " with 7 tags, and these tags are: set(['TO-TL', 'IN-TL', 'TO', 'IN-HL', 'TO-HL', 'IN', 'NPS'])