Using JSON with Replacer/Reviver
If you want to build a dynamic application in any language (that is, one where end-users can extend the environment) you need to be able to build user interface and logic elements at runtime (and/or build an API mechanism that allows some type of plugins). Generally, you can build user interface information just using data but logic requires code. JavaScript is a very common way to do both of these things - but how do you manage the data and the logic (code) so ensure it is all kept together in, say, a database?
JSON (JavaScript Object Notation) has been used for a while now to represent rich data structures and modern JavaScript engines contain the JSON object that allows a developer to turn a JavaScript object into a string and take that string and turn it back into an object…. Or does it?
Unfortunately, JSON is not “recommended” for storing the logic (the functions and code) that are associated with the object - well, at least not out of the box. Fortunately, the methods in the JSON object can be easily extended to deal with functions (or any conversion of non-data elements). This is accomplished by using a Replacer callback function when converting to a string and a Reviver callback function when converting back from string to object.
Because my development focus at the moment is Meteor (and that Meteor is built on NodeJS which in turn is built on the V8 engine) I will focus on the JSON documentation from the Mozilla website. However, a good JavaScript JSON implementation should support the Replacer/Reviver callbacks (ECMAScript 5.1).
The focus is going to be on getting any function in the object converted into a string (when using JSON.stringify) and then getting a working function when we convert it back into an object (when using JSON.parser). Converting to a string is pretty simple and we will use new Function() and eval with a bit of parsing to turn it back.
First, our Replacer function looks something like this:
function FunctionReplacer(key,value) { // in most cases, we won’t change anything // so just copy the value to return it var newValue = value; // if we dealing with an actual function if(typeof value === ‘function) { // simply use ‘toString’ on the function // to get newValue newValue = value.toString(); } return newValue; }
The Replacer function takes two values:
a key (which is the name of the element being converted)
a value (which is the real JavaScript object the key is attached to)
The FunctionReplacer method above looks at the type of the value and, if it is a function, turns it into a string that is then returned to the JSON object doing the work. Otherwise, the value is unchanged.
A call to the JSON.stringify function would look something like this:
var objectAsString = JSON.stringify(myObject,FunctionReplacer);
You could, of course, write the Replacer function in-line as an anonymous function.
No so much magic here. Most objects (including functions) in JavaScript have a toString function to provide a string representation of that object. The toString on a function returns the whole function definition - including the function keyword at the front. This is critical for the next piece: the Reviver function.
A companion Reviver function to our Replacer is a little more complicated. When you recreate the function you need to have not only the function body but the named arguments to the function as well. Otherwise, when the revived function is run, the named arguments don’t exist and will cause a runtime error. The simplest (and safest) way to do this is actually using the new Function() constructor and eval (eval is not evil in this case as you are controlling what you are doing with it). The reason you need both is because your function arguments and body are variable. You don’t know how many arguments there are until you work that out from the stored string. So, compose a new string that represents the correct new Function() code and then eval that to get a real function.
Basically, if you have a function attached to your object that looks like this:
function(x,y) { return x*y; }
… you need a call to new Function that looks like this:
new Function(“x”,”y”,”return x*y;”);
Our reviver needs to compose that code as a string and then ‘eval’ it. This means we can’t really use the function string as it was stored. Our companion Reviver looks something like this:
function FunctionReviver(key,value) { // as before, start off with the original value var newValue=value; // identify if we have a function by checking // if it is a string and if it starts with ‘function’ if(typeof value === ‘string’ && value.match(/function/)) { // the following support functions just using // some simple parsing to get the BODY and the // ARGS from the string version of the function var body = getFunctionBodyFromString(value); var args = getFunctionArgsFromString(value); // start with the body - there may be no args // quote the body - it needs to be a ‘string” // in the call to ‘new Function’ var newFunctionParms = “‘“+body+”’”; // only add in the args if there are any // this should already // be a string a quoted arguments separated // by commas - e.g. ‘x’,’y’ if(args!=””) { // prepend the args and separate with a comma newFunctionParms = args + “,” + newFunctionParms; // the above gives us a string that looks like: // ‘x’,’y’,’return x*y’ } // create the real function using ‘eval’ newValue = eval(“new Function(“+newFunctionParms+”)”); } return newValue; }
The comments pretty much explain it. The two functions with no code provided simply just use basic parsing to locate the { } and ( ) to extract the body and args respectively using mechanisms like regex and substring. Again, the Reviver function could be created inline as an anonymous function but may be easier to maintain if it is separate (the support functions for the parsing are also required and may make the code messy if done inline and should also be separate).
The call to JSON would look like this:
var myObject = JSON.parse(objectAsString,FunctionReviver);
Obviously, there is a point to all this but I thought I would start it all off with the background material that others might find useful for their projects.
Next article I will talk about how this might apply to various Meteor modules like aldeed:simple-schema and we may even eventually get into how to build Templates at runtime (if I solve some of the issues around that)
