How we improved our pull request flow
One of the most quality-improving things we started doing at Q42 is using pull requests to merge chunks of work. But we also noticed that doing so was more time consuming then it needed to be. While working on my latest project we created deployment shepherd to help us with deploying pull requests so we could easily review deployed code without having to run it ourselves.
Pull Requests primer
TLDR; You start working in your own branch (feature/login-with-returnurl) and when you are done you push your changes to Github. From the web interface you create a pull request to have your changes merged into your main development branch (default: master). By doing this you have just created a really natural moment in time for others to review your changes and give feedback before they get merged in.
Visit Github Help Page for an excellent thorough explaination of what pull requests are and how to use every aspect of them.
Improvement number one; use the status flag
Reviewing a pull request which does not reach the minimum quality threshold is a less-then-optimal situation. To make this visible we use the Github commit status api to append metadata to the pull request about its fitness-to-merge. If all is clear (code compiles, tests succeed) it will receive a green checkmark.
The checkmarks are also visible in the pull request list so we can easily discern which are ready for reviews.
Introducing deployment shepherd
So now that we have a minimal quality thresshold and review process in place we wanted to quicken the cycle time of pull requests. We determined that:
Most of our pull requests contained small localized changes;
Frontend pull requests took an disproportionate amount of time to review due to the fact that we had to visually verify that it looked as expected;
The context switch for developers was tremendous; stash local changes; checkout the pull request branch, compile the code, review the working in action, give feedback and checkout original branch to continue previous work.
To mitigate these issue we concluded that it would help us if we would automatically deploy the pull requests somewhere so we could easily verify visual changes/working with a minimal mode switch for the developer. On Azure you can create 'deployment slots' of your website which are logically grouped together. These seemed like a good candidate for our deployments.
To help us with deploying our pull requests to these deployment slots we created deployment shepherd. Deployment shepherd is ran during our deployment process and does the following:
For each deployment slot on azure, call /api/status to determine if it is alive and which code lives there;
Based on the results retrieved and the id of the pull request we are going to deploy elect the slot we can use;
If we are deploying this pull request for the first time comment it with the url where we can test it;
If we are deploying this pull request over changes which have not yet been merged comment that pull request;
Returns the deployment slot name on which to deploy our changes.
Updated deployment process
So our updated deployment process now looks like:
If deployment originates from develop branch deploy the code to our main develop environment
If the branch is a pull request (pull/123) use the deployment shepherd to find the correct environment to deploy to.
Use the given environment to deploy our website with Octopus Deploy
Note; for all branches other then develop we suffix its name to the label of the deployment in octopus to differentiate its origin.
Upgraded pull requests
Now that we have enabled deployment shepherd to find us a deployment slot we receive the following messages in our pull requests:
Limitations
Because of limitations in Azure you are only allowed to have a maximum of 4 deployment slots (+ the main deployment) per website. Due to this the maximum amount of open pull requests which we can have deployed and reviewable is also limited to 4. If at any time an open pull request deployment is overwritten by a newer deployment the following is commented to the pull request:
If this happens we have to reside to checking out the pull request locally or re-deploying it with octopus.
Afterthoughts
We are generally quite happy with the improvements to our pull requests review flow. The smaller pull requests are now merged really fast and if needed can be reviewed while deployed. Bigger changes are ofcourse still locally checked out to determine possible issues with the code.
The maximum amount of open pull requests concurrently deployed to Azure is limited to 4 but for us this is not really a big issue. Most of the times we tend to keep the amount of pull requests really low. This is also due to the fact that we open the pull request only when the main developer of a feature thinks it is done.
You can find the sourcecode of Deployment Shepherd at Github. The repository contains documentation about the usage and commandline switches.