behind the scenes of making an Bigg calculator and rule engine
Recently (at office) I had the chance to work with a couple of friends (from Taipei, Shanghai and Hyderabad). We invested lots of time in design, discussions, Proof-of-Concept, research and that resulted in a smoother and quicker implementation(comparatively). But what exactly did we make? We implemented a rule engine and calculator for our company. To give you an analogy, you could say we made a menu card for a sophisticated restaurant that served all the dishes of the world, in all the sizes, in all the varieties and for all the different type of people. You see it, the price for a dish depends on a number of things? Ingredients, Size of dish, Number of people it servers, Quality level of the Ingredients for instance are some of the levers that would determine the price of the dish. That explains why there is calculation involved, another instance to describe this is sales tax, service tax, food tax, currency tax, so many price points and each depend upon different attributes. But why did I say rule engine? Well remember I said this is a global sophisticated restaurant that servers to all the different type of people and culture in the world. Now there would be lots of rules for such a multi-cultural, multi-cuisine restaurant to function. For instance, for the Japan variety of tea, there should be a rule saying don’t put sugar(I heard they don’t), for the Chinese variety of a salad, there should be lots of soy sauce. So you see, there would be many such if-this-then-that rules defining the intricacies of a seemingly same/similar recipe based on some attributes, like audience of the dish, preparation style, ingredients etc. This could be implemented by a rule engine, which is what we did. I’m going to give a brief sneak peek into the implementation of the above calculator and rule engine, and leave the detail implementation for the next post.
First, we designed our language of representing the knowledge. Here we think how do we represent our recipe-Peking duck in terms of its constituents data/attributes (dishes, ingredients, number of servers, style etc). This leads to the formation of a generic grammar, which can be used to represent anything from Ma Po Tofu to Chicken Biryani or Pulihora
Secondly, we designed the mechanisms of data interaction or the rules. Here we think about things like how we exclude some recipe or pick the right ingredients given the recipe. This leads to the formation of your grammar rules or semantics.
To implement the above we used ANTLR and to keep everything fast we cached all the data into RAM using some interesting data structures. My friend(the Shanghai dude) did a lot of work on bits and data compression, I picked up some concepts from there to implement the in-memory layout. It consisted of RoaringBitmaps from the popular Apache Spark packages and our faithful AVL trees.
In the next blog post I’ll go more into the amazing open source package ANTLR ,till then stay tuned!
Side dish: We played around with #slack which was fun and amazing, since my friends lived in different parts of the world!
