Simple Ember.js + Node.js + D3.js Dashboard Part 1 (Client)
I'm going to break this up into two parts because no one likes to read giant ass blog posts. This first part will cover the Ember.js client and the second part will cover the Node.js server and the Google API's.
I'm writing this because I wanted to play around with Google API's and D3.js (I am slightly obsessed with analyzing data) and while there are many many great examples around the web of one or two parts of this equation, there was definitely a shortage of coverage from end to end. I hope this code example is useful to someone starting out on the same path.
The entire project with installation instructions is on github.
I'm going to assume a basic familiarity with ember-cli, npm, and bower (and that they are installed) so that we can skip over a lot of the boring stuff. If you're just going to clone the repo, the instructions are in the readme on github. If you're going to write from scratch, here are some steps.
After creating a basic ember-cli app add Zurb's Foundation
npm install ember-cli-foundation-sass --save
ember g ember-cli-foundation-sass
npm install ember-cli-font-awesome --save
and finally D3 through bower
For D3.js there are a couple of extra steps. First add the following to your Brocfile.js
app.import('bower_components/d3/d3.js');
Then in your .jshintrc file, add d3 to the list of predefs
"predef": [
"document",
"window",
"-Promise",
"d3"
],
You can run then run the server
It won't do much without the Node.js Dashboard server running, but it should at least validate that you have everything installed and setup correctly.
There are three basic routes in the app; Sessions, Visitors, and Pageviews. Each route displays a graph comparing the current 30 days of data to the previous as well as callouts that show the totals for the current and previous 30 days of data. Finally there is a callout that displays the delta between current and previous data sets.
The three routes all basically do the same thing. They each make a RESTful API call to the Node.js server that returns the two data sets (current and previous) and then uses the data sets to calculate the totals and the delta. (The calculations could just as easily have been done on the server). The results of the RESTful API are formatted and returned via a Promise as the routes model.
Since all of the routes really only differ by the type of data they return, I created a parent route called "dashboard" that the sessions, visitors, and pageviews route all inherit from. The child routes then simply call the 'getModel' function of their parent passing in the type of data they are requesting.
Here is the pageviews route as an example:
export default Dashboard.extend({
renderTemplate: function() {
// render a default template for this route
this.render('default');
},
model : function() {
return this.getModel('pageviews');
}
});
And this is the dashboard route that is inherited by all other routes:
export default Ember.Route.extend({
getModel: function(endpoint) {
// return a promise
return new Ember.RSVP.Promise(function(resolve, reject) {
// setup variables
var url = ENV.APP.API_HOST + "/api/1/" + endpoint, // API url
parseDate = d3.time.format("%d-%b-%y").parse, // turns returned dates into D3 date format
currentCount = 0, // totals
previousCount = 0,
difference = 0;
// make the GET request
Ember.$.get( url, function( data ) {
// if the API call was successful
if(data.success === true) {
// loop through the results, format dates for D3, and add up totals
for(var x = 0; x < data.current.length; x++) {
// the only date we use is the current data (default 30 days) in the x axis
// so we only have to parse the date once for both data sets
data.current[x].date = data.previous[x].date = parseDate(data.current[x].date);
currentCount += data.current[x].value;
previousCount += data.previous[x].value;
}
// calculate the delta between the two time periods
difference = (currentCount - previousCount) / previousCount;
// resolve the promoise and return the model
resolve({'current': data.current, 'previous': data.previous, 'currentCount': currentCount, 'previousCount': previousCount, 'difference': difference});
} else {
// API call failed, return an error
reject('server error');
}
});
});
}
});
A couple things to point out about the getModel function. The single paramater 'endpoint' corresponds to an api on the Dashboard server (either sessions, visitors, or pageviews). The server API's have the format '/api/1/pageviews' as an example.
The hostname for the Dashboard server is stored in ENV.APP.API_HOST. You can set this variable in config/environment.js
APP: {
// Here you can pass flags/options to your application instance
// when it is created
API_HOST: 'http://localhost:9090'
}
The import at the top of dashboard.js makes ENV accessible
import ENV from "dashboard-client/config/environment"; // import ENV
I originally didn't want to have any controllers in this project, but I really wanted to have a reusable parent route and a reusable template (more on the template below). I also wanted to put the name of the route (which is really the type of data) in the reusable template. So instead of creating multiple templates I created 3 very basic controllers to hold a "title" (either Sessions, Visitors, or Pageviews). Doesn't seem very elegant, but it is what it is.
export default Ember.Controller.extend({
pageTitle: 'Pageviews'
});
There are only two page templates. An application template that holds the Foundation "left off canvas toggle menu" and the outlet for the default template.
The default template is a single template that renders each route
<section class="main-section full-screen">
<div class="row">
<div class="large-12 columns">
<h2>{{pageTitle}}</h2>
<p class="panel show-for-medium-up">
{{line-chart current=this.content.current previous=this.content.previous}}
</p>
<ul class="small-block-grid-1 medium-block-grid-3 large-block-grid-3">
<li>
<div class="current-metric">
<span class="current-metric-value">{{format-number this.content.currentCount}}</spam>
<span class="current-metric-title">CURRENT</spam>
</div>
</li>
<li>
<div class="previous-metric">
<span class="previous-metric-value">{{format-number this.content.previousCount}}</spam>
<span class="previous-metric-title">PREVIOUS</spam>
</div>
</li>
<li>
<div class="difference-metric">
<span class="difference-metric-value">{{format-percentage this.content.difference}}</spam>
<span class="difference-metric-title">DIFFERENCE</spam>
</div>
</li>
</ul>
</div>
</div>
</section>
<a class="exit-off-canvas"></a>
It's really pretty simple. You can see the {{pageTitle}} at the top which comes from the controller. Then there is the "line-chart" component which we'll get to next. It takes data from the model. Then the three callouts: current-metic, previous-metric, and difference-metric. These take data from the model as well but they also use Handlebars helpers to improve the display by adding expected numbery things like commas, decimals, +/-, and %.
The "<a class="exit-off-canvas"></a>" anchor is a Foundation thing that closes the left slide out menu when you click on the page.
The real Ember coolness part of using a single template for all three routes comes from this function in each route:
renderTemplate: function() {
// render a default template for this route
this.render('default');
},
This just tells the route not to look for a template that matches it's name but instead just use the template called "default" maximizing code reuse. I know ... right? ;-)
So the real magic here is the line chart ... that's why I did this entire project in the first place. D3.js is an amazing library ... there really isn't much it cannot do. Instead of reading a thousand pages of documentation, to get started I googled a couple simple D3 examples and combined the parts that I liked from each. I wanted to make the code reusable of course, so I wrapped them in a component that I could use in many different templates if need be. Here's the good stuff:
Here is the component in the template with model data being passed in:
{{line-chart current=this.content.current previous=this.content.previous}}
Here is the component code:
// component for display line graph for current and previous data sets
export default Ember.Component.extend({
didInsertElement : function() {
// wait until the component loads and then process the data
Ember.run.scheduleOnce('afterRender', this, 'updateGraph');
},
// update the graph with the data from the model
// in the future we could observe 'current' and 'previous' to update
// dynamically, but keeping it simple for now.
updateGraph: function() {
// setup variables
// get the chart data
var currentData = this.get('current'), // current for last n days
previousData = this.get('previous'); // previous for n days prior to current
// get the graph from the dom and set the dimensions so that
// the x & y axis can be calculated
//
// currently height & width are fixed and the graph isn't visible
// on mobile browsers, in the future we can adjust the size
// dynamically based on window dimensions
//
var graph = d3.select("#graph"),
WIDTH = 900, // make dynamic in future
HEIGHT = 250, // make dynamic in future
MARGINS = { top: 20, right: 20, bottom: 20, left: 50 },
// determine the current data range for the x-axis (date)
currentXRange = d3.time.scale()
.range([MARGINS.left, WIDTH - MARGINS.right])
.domain(d3.extent(currentData, function(d) { return d.date; })),
// determine the current data range for the y axis (number)
currentYRange = d3.scale.linear()
.range([HEIGHT - MARGINS.top, MARGINS.bottom])
.domain([0, d3.max(currentData, function (d) { return d.value; }) ]),
// determine the previous data range for the x axis
// NOTE: commented out because we actually don't need this because the x axis is the dates from the current data,
// and the previous data is the exact same number of days. if that changes, adjust
/*previousXRange = d3.time.scale()
.range([MARGINS.left, WIDTH - MARGINS.right])
.domain(d3.extent(previousData, function(d) { return d.date; })),*/
// determine the previous data range for the y axis
previousYRange = d3.scale.linear()
.range([HEIGHT - MARGINS.top, MARGINS.bottom])
.domain([0, d3.max(previousData, function (d) { return d.value; }) ]),
// format the x axis (dates)
xAxis = d3.svg.axis()
.scale(currentXRange)
.ticks(5)
.tickFormat(d3.time.format("%b %d")),
// format the y axis (values)
yAxis = d3.svg.axis()
.scale(currentYRange)
.tickSize(2)
.orient("left")
.ticks(4),
// function to format the data point by point in the graph for 'current' data
currentLineFunc = d3.svg.line()
.x(function (d) { return currentXRange(d.date); })
.y(function (d) { return currentYRange(d.value); })
.interpolate('linear'),
// function to format the data point by point in the graph for 'pevious' data
previousLineFunc = d3.svg.line()
.x(function (d) { return currentXRange(d.date); })
.y(function (d) { return previousYRange(d.value); })
.interpolate('linear');
// functions for formatting the graph
// add the x axis to the graph
graph.append("svg:g")
.attr("class", "x axis")
.attr("transform", "translate(0," + (HEIGHT - MARGINS.bottom) + ")")
.call(xAxis);
// add the y axis to the graph
graph.append("svg:g")
.attr("class", "y axis")
.attr("transform", "translate(" + (MARGINS.left) + ",0)")
.call(yAxis);
// add the line for the current data to the graph
graph.append("svg:path")
.attr("d", currentLineFunc(currentData))
.attr("stroke", "blue")
.attr("stroke-width", 4)
.attr("fill", "none");
// add the line for the previous data to the graph
graph.append("svg:path")
.attr("d", previousLineFunc(previousData))
.attr("stroke", "orange")
.attr("stroke-width", 4)
.attr("fill", "none");
}
});
Ok, so first is my lazy confession. The rendering piece of D3 needs to know the size of the canvas it's going to draw the graph on so that it can calculate where all the lines go. I cheated here and hard coded the size. In the component template you can see the height and width.
<div class="line-chart-wrapper">
<svg id="graph" width="900" height="250"></svg>
</div>
Which correspond to this component code:
var graph = d3.select("#graph"),
WIDTH = 900, // make dynamic in future
HEIGHT = 250, // make dynamic in future
Yes, this could be dynamic ... maybe in a future update. For now I just wanted to see some lines on the screen!! :-)
In the code, everything should be labeled pretty clearly. I calculate the ranges for the current and previous data sets that are passed in from the model as component parameters. Since we are comparing current to previous, we don't need to calculate the x-range for the previous data set ... we'll just display the data relative to the current data. The y range (values) is really what we're more interested in when it comes to comparing the data sets. If I ever add a 'hover' components that displays dates and values when you hover over points on the graph (yes, D3 does that too!!!) then I'll have to uncomment the x range for previous.
The next step is to add formatting to the x and y axis of the graph. The important part here is setting the scale so that the axises contain the highest and lowest values of all of the date. You can also do things like set the label format, how the 'ticks' look, etc.
The lineFuncs are callbacks used for pulling data values into the graph. Pretty straight forward.
Finally, all of the elements are added to the graph ... the x & y axises, the current data set, and finally the previous data set. Presto!!!
**Note: Is axises correct?
Last but not least are the Handlebars helpers used to help display the numbery stuff in the callouts. (I'm a huge fan of helpers in Ember so I couldn't do a project without adding a couple).
formatNumber adds a comma into 4 digit numbers in the correct place. If you have a really busy site that gets like millions of pageviews, sessions, and users you'll need to tweak this.
// simple helper to display 4 digit numbers with a comma
export function formatNumber(input) {
return input.toString().replace(/\B(?=(\d{3})+(?!\d))/g, ",");
}
export default Ember.Handlebars.makeBoundHelper(formatNumber);
formatPercentage takes the delta between current and previous, fixes it at 2 decimal places and adds the '%' symbol. Simple, but visually important ... no one likes looking at big decimal numbers.
// simple helper to display a decimal as a percentage
export function formatPercentage(input) {
if(input > 0) {
return '+' + (input * 100).toFixed(2) + '%';
} else {
return (input * 100).toFixed(2) + '%';
}
}
export default Ember.Handlebars.makeBoundHelper(formatPercentage);
There are still some more things that I could do. Loading and error states are an obvious next step. We could also cache the models in the controllers so that we don't have to pull from the server every time we visit a page. If you have any specific requests or questions, just let me know.
Stay tuned for part 2 (the server) ....