Second level Hibernate Caching for Coldfusion on AWS using Hazelcast.
If you’re unfamiliar with Hibernate, or Coldfusion, feel free to skip over this post.
I work on a coldfusion 10 application. Recently we’ve made the push to have distributed sessions shared in redis. In the past, the site used ehcache as a second level cache for Hibernate. This was fine when the application ran on only one server. Now there are three and this caching was turned off. I had the fun task of figuring out how to enable a distributed l2 Hibernate cache. Oh how much fun I had!
In the documentation for this feature, http://help.adobe.com/en_US/ColdFusion/10.0/Developing/WSCAD9638E-2D2C-48d8-9069-AE5A220B75A6.html#WS4C3B91C4-E209-4449-B1EE-E44F4F5D3D14. They list a few cache providers:
ormsettings.Cacheprovider
This setting defines the cache provider that needs to be used for secondary cache. This defaults to EHCache. The other values for this setting are JBossCache, OSCache, SwarmCache and Hashtable. You can also specify the fully qualified class name of the cache provider.
Great! They have given me some suggestions on what I could use! A little bit of googling later and I found out that OSCache & SwarmCache are dead, Hashtable is not intended for production use and is not safe for clusters, and Ehcache is not “cluster safe”. Well that last bit about Ehcache is not true, Ehcache became part of Terracotta’s enterprise solution: Bigmemory Go or was it Bigmemory Max. Either way, the open source version of it was limited to a single server and failover. This would not suffice. Oh, and the other option listed for a cache provider, JBossCache, is now infinispan. Great. This caused plenty of confusion. Maybe if I were using the most up to date version of CF their documentation could have pointed me in the right direction. I don’t know, but this was terrible.
So, I do more googling and think “Maybe ehcache with replication will work” I tried to set this up and everything looked okay *except* replication. I was dumbfounded. I had no idea why it wasn’t working, I checked to see that multicast was enabled on the interfaces I was using. I tried so very hard to use the ehcache-debugger jar. Sidenote, I had plenty of fun inspecting the jar to see what other jars it was looking for and going out and finding them -- classpath fun. Once I resolved the classpath errors with the debugger I just ran into something about my defaultcache not supporting statistics.
So I gave up on replicated ehcache.
More googling later, I found out that Hazelcast and Infinispan both are open source Hibernate l2 cache providers. Infinispan clusters are set up via Jgroups, something I tried out earlier with ehcache’s replication and had no luck with it, so I decided to pass on that and started out with Hazelcast.
While getting hazelcast set up, I noticed they have a <aws> section in their default config file. My interest was piqued as our servers are in AWS. I looked up what this did, and found out:
Configuring EC2 Auto Discovery
Hazelcast either uses Multicast or TCP/IP for discovery, but EC2 does not support multicast. To configure Discovery using TCP/IP, you need the IP addresses upfront and this is not always possible. To solve this problem, Hazelcast supports EC2 auto discovery, which is a layer on top of TCP/IP discovery.
So *that’s* why my replication stuff didn’t work. EC2 just flat out does not support multicast. Good to know. Well I’m glad Hazelcast has a work around.
After loading up the jar and setting the cacheProvider to
I found out that it looks like CF’s version of hibernate didn’t like this, and was expecting some class that could be cast as a CacheProvider. The Hazelcast documentation points to using com.hazelcast.hibernate.provider.HazelcastCacheProvider as the class name. I tried setting this and got a no class found error. Yay! It looks like that class was removed in the Hazelcast 3.x branch. Thankfully, the 2.x branch was still being updated and I downloaded that and set it up accordingly and it worked! Even the aws autodiscovery! I could see the same objects in the cache across nodes and everything! Yay!
