After one and a half years of development I'm handing over the responsibility of developing and maintaining the Ruby driver for Cassandra to Datastax.
They've worked tirelessly for the last few months to create a new Ruby driver to match the API of the Java, Python and other official drivers, using cql-rb as a foundation.
The new driver has a few new cool features that I never had the time to add to cql-rb, including configurable strategies for which node to send a request to, for request retries and for reconnections.
Besides a blog post announcing the release of the first beta, there's lots of new documentation of the new API and the new features.
cql-rb v2.0 has finally been released. This is the first release to support all the new features in Cassandra 2.0 like result paging, mixed batches, SASL authentication.
Read the full release notes on v2.0
The next step is to add LZ4 compression support – it's already completed, but the lz4-ruby gem didn't release a compatible version in time to include it in v2.0, and there will be support for connection strategies, routing strategies, and hopefully support for encrypted connections, as well as support for the new features that will be introduced in Cassandra 2.1.
This release is forwards compatible with all versions of Cassandra that support v2 of the CQL binary protocol, which includes Cassandra 2.1 and most likely all releases in the coming year or two.
It's been way too long in the making, but v2.0 of cql-rb is very nearly here. I released the first release candidate yesterday, and the final version should be released this week.
The reason it has taken so long is twofold: first I wasn't sure enough that I'd found all the bugs, but didn't have the time to take it through the process I usually use before releasing a version, and after I had come that far I wanted to be sure about the public API.
What I do to satisfy myself that a new version is completely stable and reasonably bug free is a blog post I've been meaning to write for a while, but suffice to say that I don't take it lightly. v2.0 is a big release, there's a ton of new code and feature support, and lots of things have had to changed to accommodate supporting two protocol versions at the same time. The code that does the initial connection and peer discovery was also heavily refactored, it's by far the most complex part of the driver, but now it's at least possible to understand it again. Being confident that you've weeded out most of the bugs after such a big change requires more than just some integration tests.
Since bumping the version to a new major means the possibility of making backwards incompatible changes I also wanted to make sure I took full advantage of that and removed things I didn't want to support, dropped deprecated features, and changed things I wasn't too happy about. In the end I decided on keeping the API backwards compatible with v1.2, and just remove the deprecated features from the documentation. Some pieces that may have accidentally been in the public API documentation have also been marked as private.
Once v2.0 is released I will start working on new features, and I should probably get going on supporting Cassandra 2.1 features like user defined types.
cql-rb v2.0.0.pre1 has just been released, it's the first version with an external dependency.
Before I tell you more I want to give you some background about why cql-rb has until now been completely standalone: When I started cql-rb I was hoping to use some existing gems for things like futures, IO, byte buffers, UUIDs, etc., but I didn't find any that fit.
For example, there are many UUID gems for Ruby, but all that I looked at had issues mostly around performance and most of them were overengineered for something so simple. I've been asked a few times why I didn't just fix the problems of one of the existing gems and made the world a little better, but considering that the Uuid and TimeUuid classes in cql-rb, including the TimeUuid generator is only around 150 lines of code, it feels like it would be much more trouble than it would be worth depending on something else.
In contrast, the case with byte buffers seems it seems like everyone is happily using strings and arrays for buffers. With a bit of help from a friend I build a pretty performant byte buffer implementation that is orders of magnitude faster than appending and slicing from a string. Using strings and arrays as buffers and queues in Ruby is not a good idea, but something that most people don't need to worry about, which means that those that do don't know that they should worry about it.
When it comes to concurrency and IO I had an idea that I might try to use Celluliod and Celluloid::IO as a core and build the driver around that. In the end I decided against it, mostly because it would be such a heavy dependency and I wouldn't use much of it. In hindsight I'm very happy about this decision. I've played around a bit with Celluloid since then and either I don't understand how to test it properly, or the performance is just not very good. From what I've understood it uses fibers in its message dispatch, and that could explain things.
One of the benchmarks that ship with Celluloid shows it making 8K async message sends per second, within the same process. Compare that to a simple client/server setup that uses cql-rb's IO reactor and does over 50K async RPC calls between two processes, over TCP. That includes encoding frames and everything. 1
This brings us to cql-rb's new little sister, Ione. I have used cql-rb to build network clients for other things and it feels a bit strange depending on a Cassandra driver when the application doesn't talk to Cassandra. From the start I've tried to keep the parts of cql-rb that aren't strictly related to the CQL protocol as generic as possible. The UUID implementation is usable by itself, futures and byte buffers are completely generic, and the IO reactor doesn't assume anything about who it's talking to. As proof of that, and to keep myself straight, the integration tests have included a full-featured Redis client.
With v2.0 of cql-rb around the corner it felt like the right moment to extract the reactive core, including the futures implementation, the IO reactor and the byte buffer. These are the building blocks of network clients and reactive applications, and I've already got code that makes it possible to run servers too (a feature I had built for an application which didn't fit at all to merge back into cql-rb).
The UUID implementation remains in cql-rb, it doesn't fit in Ione, and I don't think the Ruby world need another UUID library, whatever I think about the existing ones.
So, does the world really need another IO reactor and futures implementation? I don't know. I felt like I needed one, and I didn't find the right one for cql-rb. Now what I build to solve that problem is available as a standalone project, and hopefully it will be useful to someone else but me.
About Celluloid, don't read too much into what I'm saying here, I think it's a very interesting project with many successful users. All I'm saying is that it looks to me like it wouldn't have worked for this use case, and not with the kind of performance I need from a Cassandra driver. Making something tailor made for a purpose will always make it better for that purpose than a general tool. ↩︎
It's been long in the making, but now you can get full Cassandra 2.0 support with cql-rb. Including batches, paging, SASL authentication and bound variables in non-prepared statements.
cql-rb, the Cassandra driver for Ruby, just reached v1.2.0 and is now feature complete with respect to the v1 of the CQL binary protocol.
It's been about a year since I set out to build a new, high performance, Ruby driver for Cassandra. v1.0 was released eight months ago and was about delivering the bare necessities with great performance. v1.1 followed a few months later and had everything you could expect from a driver for a distributed database. v1.2 is a tiny release in comparison to those two, but wraps up the feature set with support for request tracing and compression.
The the goal posts always move, as it were. Cassandra 2.0 came out in September with lots of new features. The next release of cql-rb, besides bug fixes and performance improvements, will also be 2.0. We'll have to see if Cassandra 2.1 comes out before, but the idea is not to have synchronized version numbers, I just think that it will be a significant number of changes and new features and a major release seems appropriate.
The plan is to do a release with support for all of the features of v2 of the CQL binary protocol, including batching, bound variables for non-prepared statements, result set paging, and support for SASL authentication. The new version also opens up some great new optimization possibilities. Keep a look out for the prereleases, but if you can't wait you can peek at the protocol_v2 branch, which already has some of the features mentioned.
Video of my talk on building a CQL driver from Cassandra Summit Europe 2013. You can find the slides here, and the videos for the other talks from the conference can be found on YouTube.
Sorry about the lack of light, I turned down the lighting in the room to make sure my slides would be readable -- it looks like it was pitch dark, but it's mostly the camera focusing on the brightest source of light in the room.
