One of my bosses took a pdf index of documents with academic and legal citations and ran it through Copilot, for the purposes of getting it into a spreadsheet without having to manually type it all out. Not a terrible thought I guess since it seems like a simple enough task but obviously proper citations are super important especially in the legal system so she asked me to take a glance to make sure there were no errors.
There was an error in literally. Literally. Almost every. Single. Citation. (And there were hundreds of them) Hereโs a non exhaustive list of things the AI did instead of just transcribing the exact text:
- Rewording or shortening the titles of articles and legal cases
- removing authorsโ middle initials
- Removing the specific dates from newspaper articles (like making it just 1966 instead of January 8 1966)
- In cases where there were multiple citations per entry, only transcribing the first one and then adding the words โAnd othersโ like? Did you get tired?? Youโre a robot
- (By far the worst one) instead of populating all the specified page numbers it just wrote โVarious pagesโ. Imagine if we went to court and when asked to cite page numbers we were just like โoh itโs somewhere in there trust us your honorโ
TLDR do not fucking use AI for shit that actually matters (I mean imo donโt use it at all for anything ever but especially not for serious stuff) because we wouldโve gotten in a ton of shit if I hadnโt checked this. And it took me forever to correct all of these errors which in the end definitely took longer than just typing it up myself in the first place.















