Graphing as a Service, all of your data in one place
The big release announcement is here, Graphing as a Service is released.
Your data is probably stored in different locations across multiple services, or maybe potentially you are not tracking the data because you do not want to go through the hassle of setting up your own custom reporting service, it's a lot of work, trust us! Let Graphdat help you get everything into one place.
No longer are you limited to just what metrics our agent provides for you. No longer are you limited to only the SDKs that we have available, now you are truly free to graph as you wish.
We have opened up our platform, provided a REST API to add your own metrics and allow you to create your own graphs, with whatever data you is important to you.
Do you want to graph MongoDB? Maybe Redis or Memcache? Or maybe you just want to track your disk usage on your server? With our new Graphing as a Service (GaaS), you can do just that.
This is the first release of many, so let me get a couple of things out there first.
Pricing. This is a beta release of GaaS, so we are offering it for free to all of our customers while we are in beta. When we move out of beta, we will offer two separate prices based on what you need. Same as before, if you are running a Graphdat agent, we will continue to charge the same low hourly pricing (nothing has changed) on a per agent basis. If you add in your own Graphs (or do not use an agent and only use the custom graphs), we will charge per metric for the custom metrics. Pricing will be somewhere around $0.25 / 10 Million data points (so about what the current agent is costs right now). We will still offer 1 free server or the equivalent in data points, so nothing changes there, same sweet deal.
Drill down support. The first round of graphs do not support the ability to drill into them (like you can for CPU processes or see the code level in Application Responses). Support for drill downs is coming, but this is just the first version so you have something to get started with.
With those out of the way, lets start creating our first custom graph. Disk Space is something that we care deeply about at Graphdat, with multiple terabytes of data and growing. Lets start by graphing the amount of free space on our test server.
Firstly, lets create the metric that we want to graph. In the Settings dialog, choose the Metrics tab and create a new Metric.
There is a lot of information here, lets go over what it all means.
name: what should the metric be called. This metric name will show up on your dashboard and in your alert emails.
short name: sometimes we have less space to work with, so we need an abbreviation, the dashboard builder is one example. disk space is pretty short, so we can use the same text for both.
identifier: this is the primary key for the metric. It is global across all of Graphdat. This is how we will identify your metric when you send us data (it will also be used when the plugin system comes into play to make it easier for you to write and share plugins). Create your own personal namespace and make it easy for yourself. In this case, all of my metrics will start with SIGURD as they will come from our test environment, code name sigurd. So lets create this metric in that namespace.
description: we display this along side the metric in the list screen so you know a little bit about your metric.
unit: what type of data are we tracking and how do we want to roll up the numbers so they make sense when the values get larger?
Count of Bytes tracks things like network traffic. When you send larger values, we roll it up into K and MB. ex. 8192
Percentage tracks the percentage of something. How busy is the CPU or how full is the disk? ex. 0.12
Duration tracks time. How long did that HTTP Request take in milliseconds? When you send larger values, we roll it up into seconds and minutes. ex. 1000
Number is a count. How many requests did you receive, how many cups of coffee did you drink? ex. 1
default aggregate: if you normally only view things in real time, you may not have even noticed that you can change the aggregates (in real time we are showing you the raw numbers). If you zoom out with your mouse, or select a larger time span from above, a new drop down appears next to your graphs and let you manipulate the data. Do you want to see the average of the values in that interval, or maybe you want to see just the min or the max value at in time period, just select it from the dropdown. In this case, selecting the `default aggregate` lets us pre-select the drop down for you to make you life just a little easier.
Now that we have our new metric, we need to create a graph to be able to see it. Because we are creating a disk usage graph, we'll need to create a dashboard with all of our disk metrics (for a refresher in how to build dashboards, you can check out this post)
Go into the dashboard builder and drag on your metrics. Your newly created metric now appears on the left, so lets drag that over and create our disk stats dashboard, give it a name and hit Save.
With our dashboard ready, we can send a couple of test metrics to make sure things are working correctly. Documentation can be found on the developer site
With a little bit of cURL love we can test it out nice and easy.
curl https://api.graphdat.com/v1/measurements \ -u [email protected]:api.896a98bf4f \ -X POST \ -H "Content-Type: application/json" \ -d ' { "source": "sigurd-db", "metric": "SIGURD_DISK_SPACE", "measure": 0.10, "timestamp": 1378262618 }'
Lets go over what those values actually mean:
email: what email address do you use to sign in with
api-token: this is YOUR personal API Token. If you are running this as an automated process, create a user for it and use that token instead. That way you can set that account as a non-admin account, user accounts are free to create, so create as many as you need. You can find your API token in the Settings -> Account tab.
source: where did the data come from? In the legend at the top of your dashboard there is a list of servers (sources), what name would you like this metric to be associated with.
metric: what identifier did you give for the metric? What is the metrics primary key.
measure: what value are we sending? Given this is a percentage, lets try with 0.10 (0.10 == 10%).
timestamp: timestamps are a little tricky, they are Unix time. If you leave the timestamp out, we do the right thing and use the current date, but when you send a batch of metrics, you need to send a timestamp so we know where to place the metrics.
So lets run the curl command a couple of times and check the graph for the results (you can remove the timestamp field for this test, otherwise you will need to keep updating the timestamps for each request). Success, we have data!
Now that we know this works, lets write a little node app to run so we can leave it running in the background and report our stats.
https://gist.github.com/6433040
And it worked, now we have our first graph. Lots of free space on the server, just what we want to see.
Next, we need to make sure we are alerted when things go wrong. If the disk fills up, we need to know. To create an alarm, open Settings -> Alarms and create a new alarm (for a refresher in how to create an alarm, you can check out this post).
With our new dashboard, our new custom graph, our new service and our new alarm, we have successfully extended Graphdat.
GaaS for the win!
Log into your Graphdat account to see the magic.
For support, drop us an email at [email protected] or on twitter @graphdat
For pricing, checkout our sweet per hour pricing.