Optimizing React Applications for Search Engines
When I first started learning and using React I always had the same thought in the back of my mind, "How do I ensure SEO with this thing?" React is a front-end JavaScript framework, much like Angular, Backbone, or Ember, so how do ensure SEO for a dynamic web application that is primarily rendered within the browser? This question came up again when I was porting a Broadleaf eCommerce application to React. The following are a few of the things I learned through the process of ensuring my React web application was SEO-friendly.
The first part of the problem I needed to solve was to find a way that Google's web crawler could crawl the webpage without having to run JavaScript and make network calls. Many front-end single-page applications tend not to fair well in this category. The whole idea around the single-page application is that the client does the work of rendering the app. My goal was to figure out how to render the page for the initial request, as well as any of the application state, so that the first request would be fully crawlable even if JavaScript were turned off. This can be fairly complicated to implement so I won't go into too much detail as others have explained it much better than I could (see redux and react-router documentation).
Rendering head meta-data on client/server
Server-side rendering is great and all, but how do I ensure my Homepage has different metadata than my Category pages? Also, how do I ensure that both the client and the server render the same head data for a page? Typically, for a server-side template engine, you can simply include the metadata within your returned template from your controller. However, for React you need to load the metadata in through Ajax. For my application, I decided to use Redux for storing my page metadata and I managed my page-specific head data in Broadleaf.
On the Broadleaf side, I have a custom endpoint that accepts a pageType and pathname, e.g. CATEGORY and /hot-sauces. Using these two values I can determine the Category and the corresponding metadata for that category. I then send back this head response JSON to my React application. I won't go into too many details for this side since it is really just a simple ad-hoc REST endpoint.
On the React side, I have a reducer and actions that handle dispatching the request and receiving the new head data. My setup is that each route has a function responsible for loading the data for that page, including the head data. I set up this function to run both server and client-side before the app is rendered. The result is that the Redux state is populated with the correct head data both client and server-side. Rendering the data into the head of the page could be complicated if I built it all myself, but thankfully there are several npm packages built around rendering head metadata for React apps. I decided to choose react-helmet.
If you are unfamiliar with React and Redux you may want to skip to the next section, otherwise we are going to briefly go into the details on how we set this up.
First, we need to start off by writing our action creators. For this we create a new file headActions.js, and our first task is to define the our action types:
// define our action types export const HeadActionTypes = { REQUEST_HEAD : `REQUEST_HEAD`, // request our head data RECEIVE_HEAD : `RECEIVE_HEAD` // receive our head data }
Now that we have defined out action types we need to define our action creators that will create the actions for the types above.
/** * Creates an action that specifies a request for head data. * @param {String} pageType the type of the page, e.g. CATEGORY or page * @param {Object} location the location we are grabbing head data for * @return {Object} the action */ function requestHead(pageType, location) { return { type : HeadActionTypes.REQUEST_HEAD, // the pageType will be used to specify the entity type on the backend pageType, // the pathname will be used to determine the proper head metadata pathname : location.pathname, } } /** * Creates an action that handles receiving head data * @param {String} pageType the type of the page * @param {Object} location the location for the head * @param {Object} head the head data * @return {Object} the action */ function receiveHead(pageType, location, head) { return { type : HeadActionTypes.RECEIVE_HEAD, // the pageType will be used to specify the entity type on the backend pageType, // the pathname will be used to determine the proper head metadata pathname : location.pathname, head } }
Now, we need to create a more complicated action that does the following: 1. Check to see if we should fetch head data 2. Dispatch request action 2. Callout to our Head REST endpoint 3. Dispatch receive action with response
/** * Checks to see if the head data for this pathname already exists in the state or if * we are already fetching the head data for this pathname. */ function shouldFetchHead(state, pathname) { return !state.headByPath[pathname] || !state.headByPath[pathname].isFetching } /** * Creates a thunk action that dispatches the request, fetches the data, then * dispatches receiving the data for the given location. This is the primary * method that our containers will use to fetch the correct metadata. * @param {String} pageType the page type * @param {Object} location the location * @return {function} thunk function */ export function fetchHead(pageType, location) { return (dispatch, getState) => { if (!shouldFetchHead(getState(), location.pathname)) { return undefined // (1) if we don't need to fetch head data, return } // (2) dispatch the requestHead to notify we are fetching data dispatch(requestHead(pageType, location)) // (3) fetch the head data return fetch(`/head/${pageType}?pathname=${location.pathname}`, { headers : { 'Content-Type' : `application/json` } }) // convert to json .then(response => response.json()) // (4) dispatch the receiveHead to notify we are done fetching data .then(head => dispatch(receiveHead(pageType, location, {head}))) } }
This should be the last of the action creators needed, next we take on setting up the reducers.
We require two reducers for handling the head actions. The first is responsible for taking the head response for a page and reducing it to an object with attributes page type, pathname, and head data. The second calls upon the first and is responsible for organizing the head states by pathname, which will allow us to cache the head data for multiple URLs. Our goal is to have our head state organized like the following JSON structure:
headByPath : { '/' : { isFetching : false, head : { title : 'The Heat Clinic', ... }, pageType : 'PAGE', pathname : '/', }, '/hot-sauces' : {...} }
We now need to create a new file named headReducer.jsand will start by creating out first reducer.
import {HeadActionTypes} from '../actions/headActions' import merge from 'lodash/merge' /** * Handles reducing the action into the head state * @param {Object} [state={ isFetching : false, head : {}, pageType : null, pathname : null, }] the initial head state * @param {Object} action the action to be reduced * @return {Object} the reduced state */ function head(state = { isFetching : false, head : {}, pageType : null, pathname : null, }, action) { switch(action.type) { case HeadActionTypes.REQUEST_HEAD: return merge({}, state, { isFetching : true, pageType : action.pageType, pathname : action.pathname, }) case HeadActionTypes.RECEIVE_HEAD: return merge({}, state, { isFetching : false, pageType : action.pageType, pathname : action.pathname, head : action.head, }) default: return state } }
Now that we have a reducer to reduce the actions into the head state, we need to organize our head state by pathname.
/** * Handles reducing the action into the head state by pathname * @param {Object} [state={}] the initial state * @param {Object} action the action to be reduced * @return {Object} the reduced state */ export function headByPath(state = {}, action) { switch(action.type) { case HeadActionTypes.REQUEST_HEAD: case HeadActionTypes.RECEIVE_HEAD: return merge({}, state, { // we ensure the head state is organized by pathname [action.pathname] : head(state[action.pathname], action) }) default: return state } }
Now we have our reducers and our actions set up. The next step is creating our container for rendering the head data into the page.
import React, {PropTypes} from 'react' import {connect} from 'react-redux' import Helmet from 'react-helmet' import merge from 'lodash/merge' class HeadContainer extends React.Component { constructor(props) { super(props) } render() { return ( <Helmet {...this.props.head}/> ) } } /** * We map the head state to our components props. * * @param {Object} state the global state * @param {Object} ownProps our component's base props * @return {Object} our component's new props */ function mapStateToProps(state, ownProps) { let {location} = ownProps let defaultProps = { head : { defaultTitle : `The Heat Clinic` } } return merge(defaultProps, state.headByPath[location.pathname]) } export default connect(mapStateToProps)(HeadContainer)
Now that we have our container, we need to ensure all of our routes render it. This can be done using a layout component or by simply adding it to each route's component.
Now that we have our container rendering, we need to add support for fetching the head state for our routes. To do this, we just need to use connect to inject our fetchHead action into our router components and then ensure we load this data on both the server and the client.
Lastly, since we don't have a dom on the server we need to modify our index file to contain the head attributes. Here is my way of handling this:
function renderHtml(html, initialState) { let head = Helmet.rewind() return new Promise((resolve, reject) => { resolve(` <!DOCTYPE html> <html ${head.htmlAttributes.toString()}> <head> ${head.title.toString()} ${head.meta.toString()} ${head.base.toString()} ${head.link.toString()} ${head.script.toString()} ${head.style.toString()} </head> <body> <div id="mount">${html}</div> <script type="text/javascript" charset="utf-8"> window.__INITIAL_STATE__ = ${JSON.stringify(initialState)} </script> <script src="/bundle.js"></script> </body> </html> `) }) }
Above we call Helmet.rewind() to get the head attributes in JSON format and then render those attributes within the head and HTML tags.
One of the things I needed to ensure was that my React had SEO-friendly dynamic URLs. Even more than that, I had to ensure that the routing could be dynamically determined from the Broadleaf RESTful API. This would require me to create a custom endpoint to determine which routes to use by pathname in Broadleaf and setting up my router in React to allow for asynchronous routing. On the Broadleaf side this was pretty simple, I just created a custom routing endpoint that responded with a page type for a given pathname. On the React side this was a little bit more complicated. Thankfully, I was using react-router, which allows for asynchronous routing. I set up my child routes to be tied to the result of that request to my routing endpoint within Broadleaf. If the pathname matched a category route, I chose the category routes; if it was a content managed page, I chose the page routes, etc. The end result is that the routing for my application was dynamic and asynchronously loaded from Broadleaf. This gives a lot of power to the admin user to influence the URL structure of their pages without having to open up an IDE.
Learning how to ensure SEO for this project boosted my confidence in working with React. I initially had doubts about how it would meet certain criteria, but in the end I found it to be an effective solution. While there are several aspects of SEO I didn't get to, I'm confident they are all easily solvable.
