#restberry

[materik]

RESTful JSON API Design (PART 3)

PART 3. This is the third and last part in a series of articles on designing RESTful JSON APIs, here is the first and second part. This part will be about query strings and authentication.

QUERY STRINGS

Query strings are a set of keys and values present in your url that affects the output or the action of a request, for example: /endpoint?offset=0&limit=10. Here, the query string begins with a question mark (?) and is followed by a key and a value separated by a equal sign (=). Two different key-value pairs are separated by an ampersand (&).

I have already mentioned one type of query string before and that is the action key which I talked about briefly in the first part. I will mention it in this part as well. Since query strings aren’t following the same rules as the resource structure we developed in part 1, it’s important that we stick to a predefined set of keys. The reason for this is so we can instinctively know which keys can be used and know what to expect by using them, this is the whole idea of the resource structure so let’s implement it here as well.

These are the predefined query string keys: fields, expand, offset, page, limit, sort and action. Here is an explanation of each of them.

Fields

This key can be used on either resource type, its value is a comma separated list of keys you wish the output JSON object to contain. Ex:

GET /tracks/123?fields=nr,name -> 200 {     track: {         href: “/tracks/123”,         id: 123,         nr: 1,         name: “Track Name #1”,     } }

Notice that href and id are always present in the output but that no more keys than nr and name, which is specified in the query strings, are returned.

Expand

Expand is used when you want to show more data of a nested object. The value is a comma separated list of keys you want to expand. By default, a nested object should only show href and id if expand is not defined. Ex:

GET /albums/123 -> 200 {     album: {         href: ‘/albums/123’,         id: 123,         name: ‘The Marshall Mathers LP’,         artist: {             href: ‘/artists/12’,             id: 12,         }     } }

With expand, ex:

GET /albums/123?expand=artist -> 200 {     album: {         href: ‘/albums/123’,         id: 123,         name: ‘The Marshall Mathers LP’,         artist: {             href: ‘/artists/12’,             id: 12,             name: ‘Eminem’,             fullName: ‘Marshall Mathers’,         }     } }

This is also true for collection resources which should not be expanded by default, ex:

GET /albums -> 200 {     ...     albums: [         {             href: ‘/albums/1’,             id: 1,         },         {             href: ‘/albums/2’,             id: 2,         },     } }

A single resource response shouldn't need to have expand defined. It is implied that you want the single resource to be expanded by default.

Offset/Page

Offset and page are integer values which can only be applied to a collection resource and is used in combination with limit to enable pagination. Offset is the exact index from where the objects should be returned from and page times limit gives the start index of the returned objects. These values have a relationship as such: offset = (page - 1) * limit. You can only use one of them in a request. Ex:

GET /albums?offset=2 -> 200 {     …     albums: [         {             href: ‘/albums/3’,             id: 3,         },         {             href: ‘/albums/4’,             id: 4,         },         {             href: ‘/albums/5’,             id: 5,         },     ], }

The default values for these should be set to offset=0 and page=1.

Limit

Limit is an integer value and goes together with offset and page to enable pagination and is the number of items the user wish to be returned from a collection resource. Ex:

GET /tracks?limit=1 -> 200 {     …     tracks: [         {             href: ‘/tracks/1’,             id: 1,         },     ], }

You can choose the default values for limit which best suits your project but a recommended default value is 10. Remember to also define a maximum limit so a user can't retrieve too many objects in too large of a response.

Sort

Sort is a comma separated list of keys that you want the returned list from a request to be sorted by. The order of the list decides the priority of the sort where the first one has the highest priority. You can decide if the sort should be ascending or descending by having a minus sign infront of the key for descending and leaving it out for ascending. Ex:

GET /artists?limit=3&expand=track&sort=name,-age -> 200 {     …     artists: [         {             href: ‘/artists/1’,             id: 1,             name: ‘Arnold’,             age: 43,         },         {             href: ‘/artists/3’,             id: 3,             name: ‘Arnold’,             age: 35,         },         {             href: ‘/artists/2’,             id: 2,             name: ‘Boris’,             age: 38,         },     ], }

Sort should be set to timestamp created by default or similar value.

Action

Action is the key where you have more leeway in defining your own functionality to a request. The value should be something descriptive of what will happen and the HTTP method should tell if an action or a retrieval will happen, GET=retrieval and POST=action. The value should be a lower case string with dashes instead of spaces. Ex:

POST /albums?action=export-covers -> 202 {}

AUTHENTICATION

There are many ways of doing authentication and some of them follow the RESTful guidelines more than others. As always, my feelings on it is that it should be as easy an intuitive as possible. The authentication workflow consists of two API calls, POST /login and GET /logout. A session token is generated and returned by the login request and passed in the header to other API calls. The logout request destroys the session token and ends the ability to use the token for authentication. Login ex:

POST /login {     username: ‘materik’,     password: ‘********’, } -> 200 {     user: {         href: ‘/users/123’,         id: 123,         username: ‘materik’,     } } HEADER {     …     Set-Cookie: ‘connect.sid=<token>; …’, }

Notice that the user object is returned by the login request and that the session cookie is returned in the header. The key of the header returned by the login is Set-Cookie but the authenticated requests after this should have the token returned in the header with the key: Cookie, ex:

GET /users/123/albums HEADER {     Cookie: ‘connect.sid=<token>’, } -> 200 {     …     albums: [         …     ] }

To kill the session you request logout, ex:

GET /logout -> 204

The logout request returns 204 NO CONTENT which says that the request was successful.

THE END

That is all the parts I deemed necessary to know about to create a good and scalable RESTful JSON API. If you have something you want me to add or something else you want to discuss about this series, please contact me on twitter (@thematerik). Thanks for reading and happy coding…

Planning on writing an epilogue that will summarize this series include some more thoughts on API design, stay tuned…

[materik]

RESTful JSON API Design (PART 2)

PART 2. This is the second part in a series of articles on designing RESTful JSON APIs, here is the first part: http://materik.tumblr.com/post/98324672516/restful-json-api-design-part-1. This part will be about URL design and responses.

URL DESIGN

There are two ways of designing the base URL of your API for easier adoption and usage. You can either have it be:

http://api.myapp.com

Or

http://*.myapp.com/api

The first one is the preferred option but the second one has the benefit of being able to exist on the same domain as your web application which makes it easier to handle domain specific session cookies. If this is not an issue for your implementation however I would definitely use the first option.

Versioning is an important strategy to help consumers of your API be sure that the functionality is consistent over time and leaves you room to extend and improve the API without breaking older versions. By including the version number in your API paths the consumer can be sure that the response from a particular API call will always keep the same structure. This is particularly important so the user doesn't have to update his code as soon as you do a refactoring of your API. The URL should look like this:

http://app.myapp.com/v1

Or

http://*.myapp.com/api/v1

The version number shouldn't need to contain minor values or more detailed versioning. Use v1 and v5, not v1.2 nor v5.2.2. This type of detailed versioning is not necessary and the consumer of your API can't and shouldn't need to know the difference between these fine incremental version updates. Only bump the version number when new functionality has been added or a response structure has been changed, anything that will break legacy.

RESPONSES

HTTP Status

An API response should always be accompanied by a relevant HTTP status so that the response can easily be identified as a success or failure. There are many statuses to chose from and they can all be found in this w3 provided list: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html.

Some of the more useful and important ones to know about are:

200 OK The request was successful and a response body was returned. Ex: GET /albums -> 200 { ... }

201 CREATED The resource was created and a resource representation was returned. Ex: POST /albums { ... } -> 201 { ... }

202 ACCEPTED The request was received and is being processed. Ex: POST /tracks?action=build { ... } -> 202 { ... }

204 NO CONTENT The request was successful but no response body was returned. Ex: GET /logout -> 204

400 BAD REQUEST The request was not successful and the error response contains more details. Ex: POST /tracks { ... } -> 400 { error: { ... } }

401 UNAUTHORIZED The request was not successful because you need to be logged in to access the resource. Ex: POST /tracks/:id { ... } -> 401 { error: { ... } }

403 FORBIDDEN The request was not successful because you have not been permitted access to the resource. Ex: POST /tracks/:id { ... } -> 403 { error: { ... } }

404 NOT FOUND The resource that was requested does not exist. Ex: GET /tracks/:id { ... } -> 404 { error: { ... } }

409 CONFLICT The resource was not created because it is conflicting with another object due to lacking uniqueness or due to some other reason. Ex: POST /albums { ... } -> 409 { error: { ... } }

500 INTERNAL SERVER ERROR The request couldn't be evaluated because an unexpected server error happened. Ex: POST /albums { ... } -> 500 { error: { ... } }

Some comments

Be sure to use 201 when you actually create a new resource so the consumer knows the difference between creating a new object and updating an existing one.

The difference between 401 and 403 is that 401 should be returned if you are not logged in or don't have an active session while 403 should be returned if you are not the owner or permitted to access a certain resource.

JSON Response Body

There are three types of responses: instance response, collection response and error response.

Instance Response

Should be returned when you are getting an object, when you're creating a new object and when you are updating an existing object. The JSON response should contain one key which should be the name of the resource in singular form and the value should be a JSON representation of the object. Example:

GET /albums/12 -> 200 {     album: {         href: '/albums/12',         id: 12,         artist: 123         name: 'Album Name',         ...     } }

POST /albums { ... } -> 201 {     album: {         href: '/albums/12',         id: 12,         artist: 123         name: 'Album Name',         ...     } }

Notice that getting an object and creating an object should return the same type of response body. This will make it easier for the consumer to handle the response the same way.

HREF

The href key contains the path to the instance resource and helps the consumer to know where to access the specific object. With the resource format we established in PART 1 however this should not be that important since you should be able to deduce the location of the instance by following the resource convention. With that being said, this could increase the accessibility of your API. Have the href value contain the API domain if it is supposed to be accessed remotely. If the API exist on your web application domain however I prefer to just have the path in the href value.

Collection Response

Should be returned when you are getting many objects. The JSON response should contain a key with the name of the resource in plural form, containing a list of JSON representations of objects matching the query. The response should also include pagination information for easier accessibility. More details on this will be provided in the next part. Example:

GET /artists -> 200 {     hrefs: {         current: '/artists?offset=0&limit=10'         first: '/artists?offset=0&limit=10',         last: '/artists?offset=90&limit=10',         next: '/artists?offset=1&limit=10',         prev: null,     },     offset: 0,     limit: 10,     total: 100,     artists: [         {             href: '/artists/1',             id: 1,             name: 'Artist #1',             ...         },         {             href: '/artists/2',             id: 2,             name: 'Artist #2',             ...         },         ...     ] }

Error Response

The error response should contain sufficient information for the consumer of the API to know what went wrong. Returning a response with error set to TRUE or FALSE is redundant and you will already know if the request was successful or not depending on the HTTP status code. Instead a good error response should contain:

statusCode Include the status code in the response for convenience.

property Give an indication what resource threw the error.

title Have a user friendly title describing the problem which can be displayed to the end user.

message Have a user friendly message about the problem the can also be displayed to the end user.

devMessage Provide a more in-depth information about the issue that the consumer of the API may use to interpret the issue.

moreInfo (optional) Provide a link to your API documentation which may contain more information about the issue.

Example:

GET /albums/12 -> 404 {     error: {         statusCode: 404,         property: 'album',         title: 'Not Found',         message: 'Couldn't find album number 12',         devMessage: 'Make sure you have the correct id of the album.'     } }

IMPORTANT

The keys in the JSON responses should always be in camel case independent from what language your server is implemented in. Remember that JSON stands for JavaScript Object Notation and the standard in JavaScript is to use camel case. The consumer of the API shouldn't need to know which language the API was implemented in and therefore can be sure to always use camel case.

Continued in Part 3: http://materik.tumblr.com/post/101938795476/restful-json-api-design-part-3

[materik]

RESTful JSON API Design (PART 1)

PART 1. This is the first part in a series of articles on designing RESTful JSON APIs. This part will be about resources, designing routes and adding behaviour.

Most of my knowledge about RESTful API design I've gotten from Stormpath's CTO, Les Hazlewood, in his Best Practices video from 2012: https://www.youtube.com/watch?v=hdSrT4yjS1g. I really suggest you watch it as it is a good complement to this series. I've attempted to summarize some of the points he's making and present some of my own experiences into an easy to follow guide.

REST doesn't have a clear cut implementation specification but there are lots of conventions and guidelines that you may follow. The most important point which Les is making in his presentation is that an API should be designed for easy adoption and usage before catering to all ideologies of REST design out there and I completely agree. All these points I'm making in this series should make for a more accessible API design but I invite to discussion on improving the ideas presented here so that we all can benefit from it.

With that said, let's begin.

RESOURCES

Resources are representations of things, collections and/or objects and they may have relationships with other resources. A resource is what a developer interacts with through the API and may operate on it through some pre defined methods (see the behaviour section). There are three types of resources:

Collection Resource (ex: /artists) Interact with many objects

Instance Resource (ex: /artists/:id) Interact with one object

Relationship Resource (ex: /artists/:id/albums) Interact with many objects related to another object

A resource should always be a noun and should not reflect behaviour, that is, they should never be verbs. Example of bad resources are:

/addArtist /verifyArtist /getAlbum /getAlbumReleaseDate

Having behaviour in your routes will make it more difficult for a developer to instinctivly know the route to use and with time the number of endpoints will grow as you add more behaviour to your API and will quickly become unmaintainable.

Note on the Relationship Resource

You shouldn't have to string more than two relationship resources together. For example, these are generally bad:

/artists/:artistId/albums/:albumId/tracks /artists/:artistId/albums/:albumId/tracks/:trackId

Your routes will become very long and you will need to know more identifiers than you should have to. For example, if you want to interact with the tracks of an album you shouldn't need to know the artist, or, if you want a particular track you shouldn't need to know the album (This, of course, assumes that all the tracks have unique identifiers, regardless of album or artist. If this is not the case you will need to have your relationship routes like this). Replace the routes above with these instead:

/albums/:albumId/tracks /tracks/:trackId

Rule of Thumb

You shouldn't need to know more than one identifier per route.

BEHAVIOUR

To add behaviour to a resource you should use the HTTP methods: POST, GET, PUT and DELETE (I won't be talking about HEAD nor PATCH). You can think of them corresponding to CRUD (Create, Read, Update and Delete) but they are not truly one to one but it might be easier to think of them this way. Here are some more in depth explanation of the methods:

GET

Getting data from a resource. Could be applied to any of the resource types:

GET /albums - Returns a list of all albums GET /albums/23 - Returns a representation of album 23 GET /artists/111/albums - Returns a list of all albums belonging to artist 111

DELETE

Deleting a resource. Could be applied to any of the resource types:

DELETE /albums - Will delete all albums DELETE /albums/23 - Will delete album 23 DELETE /artists/111/albums - Will delete all albums belonging to artist 111

For safety you might not want to expose the multiple delete options as this could be dangerous. Instead you may only want to allow the deletion of an instance if your API is not dependent on deleting objects in bulk.

PUT

Updating a resource with the supplied request data. All fields of the resource is required in the request because the same PUT request should always return the same response (PUT is idempotent). You can only perform a PUT request on an instance resource:

PUT /albums/23 - Will update album 23 with the request data

POST

Creating or updating a resource with the supplied request data. All fields are not necessary in these requests unlink PUT. POST can be applied to all resouces but with different behaviour:

Create:

POST /albums - Will create a new album with the request data POST /artists/111/albums - Will create a new album belonging to artist 111 with the request data

Update:

POST /albums/23 - Will update album 23 with the request data

Note

The difference between the PUT update and the POST update is that PUT requires all fields to be supplied unlike POST. You can think of a POST update as a partial update.

FURTHER BEHAVIOR

For adding further behaviour that doesn't fit with the CRUD paradigm you may use query strings to perform an action, for example:

GET /albums/23?action=get-cover POST /albums/23?action=update-cover

Use GET when you want something returned and POST when you want something to happen. These routes won't be instinctivly familiar to the developer why you need to stick to the action query string key and an easy to understand action value, but otherwise this way of adding more behaviour should be infinitely expandable.

Continued in PART 2: http://materik.tumblr.com/post/99806761591/restful-json-api-design-part-2

[materik]

Restberry is a framework for setting up RESTful APIs with NodeJS, just define your models and setup your CRUD API without needing to write any code.