One of the most quality-improving things we started doing at Q42 is using pull requests to merge chunks of work. But we also noticed that doing so was more time consuming then it needed to be. While working on my latest project we created deployment shepherd to help us with deploying pull requests so we could easily review deployed code without having to run it ourselves.
Pull Requests primer
TLDR; You start working in your own branch (feature/login-with-returnurl) and when you are done you push your changes to Github. From the web interface you create a pull request to have your changes merged into your main development branch (default: master). By doing this you have just created a really natural moment in time for others to review your changes and give feedback before they get merged in.
Visit Github Help Page for an excellent thorough explaination of what pull requests are and how to use every aspect of them.
Improvement number one; use the status flag
Reviewing a pull request which does not reach the minimum quality threshold is a less-then-optimal situation. To make this visible we use the Github commit status api to append metadata to the pull request about its fitness-to-merge. If all is clear (code compiles, tests succeed) it will receive a green checkmark.
The checkmarks are also visible in the pull request list so we can easily discern which are ready for reviews.
Introducing deployment shepherd
So now that we have a minimal quality thresshold and review process in place we wanted to quicken the cycle time of pull requests. We determined that:
Most of our pull requests contained small localized changes;
Frontend pull requests took an disproportionate amount of time to review due to the fact that we had to visually verify that it looked as expected;
The context switch for developers was tremendous; stash local changes; checkout the pull request branch, compile the code, review the working in action, give feedback and checkout original branch to continue previous work.
To mitigate these issue we concluded that it would help us if we would automatically deploy the pull requests somewhere so we could easily verify visual changes/working with a minimal mode switch for the developer. On Azure you can create 'deployment slots' of your website which are logically grouped together. These seemed like a good candidate for our deployments.
To help us with deploying our pull requests to these deployment slots we created deployment shepherd. Deployment shepherd is ran during our deployment process and does the following:
For each deployment slot on azure, call /api/status to determine if it is alive and which code lives there;
Based on the results retrieved and the id of the pull request we are going to deploy elect the slot we can use;
If we are deploying this pull request for the first time comment it with the url where we can test it;
If we are deploying this pull request over changes which have not yet been merged comment that pull request;
Returns the deployment slot name on which to deploy our changes.
Updated deployment process
So our updated deployment process now looks like:
If deployment originates from develop branch deploy the code to our main develop environment
If the branch is a pull request (pull/123) use the deployment shepherd to find the correct environment to deploy to.
Use the given environment to deploy our website with Octopus Deploy
Note; for all branches other then develop we suffix its name to the label of the deployment in octopus to differentiate its origin.
Upgraded pull requests
Now that we have enabled deployment shepherd to find us a deployment slot we receive the following messages in our pull requests:
Limitations
Because of limitations in Azure you are only allowed to have a maximum of 4 deployment slots (+ the main deployment) per website. Due to this the maximum amount of open pull requests which we can have deployed and reviewable is also limited to 4. If at any time an open pull request deployment is overwritten by a newer deployment the following is commented to the pull request:
If this happens we have to reside to checking out the pull request locally or re-deploying it with octopus.
Afterthoughts
We are generally quite happy with the improvements to our pull requests review flow. The smaller pull requests are now merged really fast and if needed can be reviewed while deployed. Bigger changes are ofcourse still locally checked out to determine possible issues with the code.
The maximum amount of open pull requests concurrently deployed to Azure is limited to 4 but for us this is not really a big issue. Most of the times we tend to keep the amount of pull requests really low. This is also due to the fact that we open the pull request only when the main developer of a feature thinks it is done.
You can find the sourcecode of Deployment Shepherd at Github. The repository contains documentation about the usage and commandline switches.
For a project I am working on we want to retrieve a changelog from TeamCity so we can append it as releasenotes to the deploy in Octopus Deploy. Unfortunately it is currently not easily possible to retrieve the pending changes from within a build of TeamCity. To fix this I have created a small tool which will retrieve this information from the TeamCity rest API and output the changes as markdown to the console.
Code
The code is located at Github. It is build with .Net4.5 without any fancy stuff (NuGet restore && build should suffice).
## [janedoe - 2014-07-21 21:17](https://github.com/crunchie84/teamcity.releasenotesfetcher/commit/c96e213a1c54d5d0ff47b91c7c163928b4b389c4) this is my commit (oldest) ## [johndoe - 2014-07-21 22:27](https://github.com/crunchie84/teamcity.releasenotesfetcher/commit/c96e213a1c54d5d0ff47b91c7c163928b4b389c4) this is my commit (little newer) ## [johndoe - 2014-07-21 23:37](https://github.com/crunchie84/teamcity.releasenotesfetcher/commit/c96e213a1c54d5d0ff47b91c7c163928b4b389c4) this is my commit (newest)
Commandline packaging of your .Net website to the filesystem
Normally when I want to deploy an ASP.Net MVC website I create a MsDeploy package but due to the fact that the current project uses Octopus Deploy which does not natively support MsDeploy (yet). So now I was digging around in the msbuild targets to find out how to get a compiled website package ready for FTP deployments without all the non-deploy files.
After searching around a lot in the files and on the internet I found a blogpost detailing the exact issues why and how to get your appication de package to a filesystem location. The steps are as follows:
Append the following msbuild target to your {mywebsite}.csproj
<Target Name="PublishToFileSystem" DependsOnTargets="PipelinePreDeployCopyAllFilesToOneFolder"> <!-- This target is a custom build target, taken from this blog: http://www.digitallycreated.net/Blog/59/locally-publishing-a-vs2010-asp.net-web-application-using-msbuild --> <Error Condition="'$(PublishDestination)'==''" Text="The PublishDestination property must be set to the intended publishing destination." /> <MakeDir Condition="!Exists($(PublishDestination))" Directories="$(PublishDestination)" /> <ItemGroup> <PublishFiles Include="$(_PackageTempDir)\**\*.*" /> </ItemGroup> <Copy SourceFiles="@(PublishFiles)" DestinationFiles="@(PublishFiles->'$(PublishDestination)\%(RecursiveDir)%(Filename)%(Extension)')" SkipUnchangedFiles="True" /> </Target>
