VPork
VPork background With the wide range of distributed, non relational databases out there it is hard to know which one to choose. One part of the puzzle is of course performance. Personally I'm interested in low response times. A couple of weeks ago Jon Travis put up a useful program called VPork on his github repository. It's a fairly straight forward performance testing tool for Voldemort, written in Groovy. You can find the announcement on the Voldemort mailing list. Short description of how it works from the wiki: * A single JVM is started, with any number of client threads * Each thread executes for a given number of iterations * For each iteration, the thread can read an existing record, and/or create a new one * The probability of each read/write is configurable * The location of where reads happen is configurable (by default, it reads the most recently written records, trailing off to less frequent reads of writes which occurred long ago) At the end of the run it gives you info such as average, 99th percentile and standard deviation of read and write latency. VPork + Cassandra I'm interested in both Voldemort and the Cassandra project. Thus, it seemed like a good idea to add the ability to benchmark Cassandra to VPork. The result can be found my branch here. How to run a one node Cassandra test (on Ubuntu) First let's fetch Cassandra, there's no official release yet so we'll use a nightly build. sudo apt-get install sun-java6-jdk wget http://hudson.zones.apache.org/hudson/job/Cassandra/lastSuccessfulBuild/artifact/cassandra/build/cassandra-0.3.0-dev.tgz tar -zxvf cassandra-0.3.0-dev.tgz sudo mkdir -p /var/cassandra/logs sudo chown -R `whoami` /var/cassandra/ cd cassandra-0.3.0-dev bin/cassandra -f Now Cassandra should be up and running, let's start VPork. sudo apt-get install git-core groovy git clone git://github.com/johanoskarsson/vpork.git cd vpork ./vpork.sh configs/cassandra/30-thread-pork.groovy configs/cassandra/nodes.conf After a while hopefully you'll get some meaningful results. This is of course a very basic test, you probably want to add more Cassandra nodes, run the client on another node etc. Where do we go from here? VPork has some drawbacks such as suffering from the power-of-ten syndrome and it doesn't warm up the databases before it starts measuring, but it's a good start! I'd love to see other interesting storage engine tests added to VPork to give users a simple way of comparing them with the load pattern they expect. There's an issue open for it over in HBase land. It's fairly easy to do, create a class that implements a createClient() method returning a client that implements the basic get(String key) and put(String key, byte[] value) methods. Another possible use is to alert developers of performance regressions. There's been discussions about setting up nightly benchmarks on multiple servers to do just that. Disclaimer: I'm no performance testing expert or a statistician (or a magician for that matter, but it's not important right now).