Don’t always trust your libraries to do what you expect.
I recently implemented chunked or “multi-part” uploading to a video uploader. As part of that I made use of dropzone (this was already there from work I had done earlier) and resumable.js (the glue to make support chunked uploading). This was a great win, users with spotty internet connections could now upload large files without worry of the upload failing halfway through. As part of my testing for this feature, I would start an upload, disconnect from the internet, wait, and reconnect and ensure the upload finished. Hooray!
This made it out to production and everything seemed okay. However, a few weeks after it was rolled out we got a few reports of “This file just wasn’t working when we uploaded it”. Initially it seemed a lot of those reports were due to encoding and our transcoder not playing nice with it. Eventually a report came in that one of the videos that was uploaded, when played back, was missing the last few seconds of video! Uh oh.
This immediately smelled of something going foul with either the chunking, or the rebuilding of the file server-side. We reached out to our transcoding service and learned we were missing a few parameters in the request we were making prior to the actual upload. Well, cool, I went about starting to fix that when I noticed that the exact number of chunks I was uploading was off by one.
It turns out that resumable.js will, by default, will set the last chunk to upload’s size as in between n to 2n-1 (n being the chunk size). https://github.com/23/resumable.js/issues/51 Is the issue where some unfortunate soul discovered this odd behavior. Anywho, the company behind resumable.js decided that because of their specific use-case around the last chunk’s size, they wanted this to be the default behavior. This caused me some headache, but at least they stuck in the forcechunksize option in there to override their silly default.
So, lesson learned, always check to be sure that library you’re trusting to work, actually works the way you are expecting it to.
