It was back in August 2010 that I joined Adfero, fresh out of University, and since I started work I’ve hardly blogged, if at all, on any of the amazing things I’ve worked on or done. So I thought it was about time I brought people up to speed, and hopefully, with future planned projects write more blog posts. When I initially set out to write this blog, I was looking to write about the future not a summary of the past (I was going to save specifics for future blog posts) but in reality it’s kind of become both.

Past Projects

When I look back over the past year I can think of three major projects I have been heavily involved in with Adfero:

• Moving the News Feeds product to Amazon Web Services (AWS)
• Researching and implementing the use of NuGet across the development department
• Drawing up guidelines for the development department on versioning software

Each project deserves a blog post in it’s own right, but I thought I’d put a small summation here of each one. More as a prompt for me to write full posts on them more than anything else

Migration to AWS

When I moved to the News Feeds product, indeed when I started at Adfero, the majority of the company’s infrastructure in terms of applications was run on our own servers managed and maintained by us. The decision had been made earlier in the year that our internally developed applications to move to the cloud to leverage the massive benefits of having a specialist company managed the general infrastructure.

I arrived at the News Feeds team when it was still just one developer, and they were just about to leave to do a Masters, meaning I became the sole developer on the product. It was over the course the next four months, along with our Software Architect Dan Bradley, we came up and implemented a plan to move the product into AWS. In the most cases this involved minor tweaks to applications, but in other areas complete re-writes took place to make application more scalable and manageable. In fact since I’ve been working on the News Feeds product the entire stack has been completely re-written in one shape or form to take advantage of new technologies like MVC 4.

It was only after the migration to AWS was I joined on the team by another developer, and so I became the de facto team leader.

NuGet Implementation

With what has become a very large number of specialist applications across not only the News Feeds product but all our products we started to stroll into the world of DLL Hell. This would have been bad enough, had it not been for the fact we started iterating quite quickly on some of our internal frameworks and applications. It was about this time that I remembered about NuGet. I’ve always been a fan of making things simple and easy to use, and this is what NuGet does.

After spending some time researching NuGet, I drew up some guidelines, and incorporated it into the News Feeds product build cycle. It took a while to iron out some of the little nuances of it, but after about 2 weeks I was satisfied it worked effectively, and received backing to roll it’s use out to all our products. We still have issues from time to time, but no more are we stuck in the world of DLL Hell and reference DLL repositories.

Versioning Guidelines

One of the age old problems in software development is versioning of applications. It wasn’t until we had been using NuGet for about a month that we realised we had some issues in the way we were generating version numbers for our internal frameworks. All of a sudden the Visual Studio auto incrementing version numbers were not helpful, especially across teams of multiple developers, all building and deploying from their own machines.

In a fit of rage one day (self inflicted I must say) I decided there must be a better way. For a while we had been looking around and talking about using Semantic Versioning (SemVer), which in my view is an excellent base to come up with some guidelines. In general all the SemVer principals are sound, but for us, they needed modifying slightly to fit in with the Microsoft world of four digit version numbers. After I calmed down I drafted our versioning guidelines, to incorporate our use of Visual Studio, NuGet and Git. The guidelines have since been rolled out to all our development teams

I hope to publish a revised version of the guidelines for versioning frameworks and API’s in a world of Visual Studio, NuGet and Git soon as an expected update to NuGet will use SemVer.

Future Projects

Using the experience gained while working, I’ve noticed some areas where I really would like to work on more:

• Automating server deployment
• Automating the building of software

As you can see both involve automation. I’m a huge fan of doing as little as possible to go from application writing to it appearing on a server for all to use. Both of these areas I intend to blog on a lot in the future so I’m only going to give a brief overview.

Automating server deployment

For a long time at work we’ve been talking about ways to automate the way we deploy our servers and applications on servers. This has involved several steps in the form of things like using NuGet, proper versioning guidelines and our in house (now open-sourced) windows application deployment system built on top of S3 called Plywood.

A couple of weeks ago I had a quiet period while waiting to hear back from other departments, so I though why not take a look at actually automating server start up. Currently we click buttons in the AWS management console and occasionally use the command line to spin up servers if the console doesn’t have all the required options. After some looking around and comparisons I decided to go with Chef, partially for the growing Windows support, and partially for the ability to run your own Chef servers for free (and there is a limited free hosted version). Chef has the ability to script in Ruby repeatable parts of the deployment of servers, as well as deploy applications as and when you like it. This means we can start up vanilla flavours of Linux and let Chef deal with the rest.

While Chef deals with the configuration of servers, it doesn’t really deal with the issue automating spinning up servers; be it in creating a new stack or automating scaling of the stack to deal with demand. Fortunately, AWS has a solution to this in the form of CloudFormation, which deals with provisioning all the resources need to spin up a completely new stack and scale it for changeable demand.

Automating the building of software

Having got bored of waiting for my machine to build the applications I write, I decided to write an application do it for me on a remote machine, and so Lumberjack was born. Lumberjack aims to automate pulling in the latest commit from a source code repository, build the source code, test it and if all is well push the build package to a destination deployment system or, simply a file system like S3.

I have managed to get a prototype of the system up and running which I am in the process of converting into a fully working version. While the main core of the system will remain closed source for the time being, I have decided to write all the source control, building and output functionality in an open source project called Woodpecker. This is in line with the Github idea of “open source (almost) everything” as written by Tom Preston-Werner of Github.