Memory use is often overlooked when people compare JSON libraries
TL;DR: In decode oriented use-case with big payloads JSON decoders often use disproportionate amounts of memory. I gave up on JSON and switched to Msgpack.
You should draw your own conclusions by running the test code yourself.
Based on various feedback [*] I've did the benchmarks again, using ru_maxrss instead of Valgrind and with few more implementations.
processing one large file - 240MB JSON
Peak memory usage (Python 3.5):
marshal: 372.1 Mb pickle: 372.9 Mb msgpack: 376.6 Mb rapidjson: 668.6 Mb yajl: 687.3 Mb ujson: 1,578.9 Mb json: 3,422.3 Mb simplejson: 6,681.4 Mb Speed (Python 3.5) ----------------------------------------------- Name (time in ms) Min ----------------------------------------------- test_speed[msgpack] 69.0613 (1.0) test_speed[pickle] 69.9465 (1.01) test_speed[marshal] 74.9914 (1.09) test_speed[rapidjson] 337.5243 (4.89) test_speed[ujson] 902.8647 (13.07) test_speed[yajl] 1,195.4298 (17.31) test_speed[json] 4,404.9523 (63.78) test_speed[simplejson] 6,524.9919 (94.48) -----------------------------------------------
Bottom line
Both speed and memory are affected by data shape. Speed is not always proportional to memory use.
Again, don't trust the numbers, run the benchmarks yourself, with your own data. Even if your data is identical in shape your hardware might behave differently than mine. Even the memory use can be different on your machine (example: different architecture, different shared libraries). And what's the chance your data has the exact shape as whatever was used in the benchmark?
...
Feedback from commenters here, Reddit, HackerNews and Google+.









