Happy Cog was contracted by MTV to create a website that allowed users to socialize around the MTV Online Music Award (OMA) nominees and vote for the winners. Being an MTV event, with on-air promotion and billing, we knew we were going to get hit with some pretty heavy traffic. Knowing this, we needed to plan our infrastructure so that it could grow as demand increased. That meant scaling up from our initial server pool of four servers to a hefty twenty-two application servers during the hour long voting event. Luckily for us, Beanstalk made this entire process a breeze.

Following Branching Best Practices

To begin we employed Beanstalk’s suggested “deployment branch” approach. That meant we had two important branches: stage and master. Stage was our test bed where we committed all the soon to be live changes. This could be a copy update, a change to the legal speak or something as major as updated imagery. This worked great locally as it allowed us to work collaboratively as a development team and commit changes for others to review. What’s more, rolling in Beanstalk deployments, we were able to automatically deploy this branch out to a test server for the wider audience to review. That meant our internal QA team and our MTV clients could always see what we were working on. They could monitor the progress of updates and confirm them once they were done. This allowed us to work quickly without worrying about gaining approval for major releases. Instead, we gained approval every few minutes as individual updates were made.

Once we had a good batch of updates to push live we simply merged the stage branch into our master branch and manually deployed the changes. This way we were able to resolve conflicts and test merges before anything went live.

Beanstalk Saves the Day: Scaling Servers over a 3G Hotspot

While out in LA for the live event our site was built for, we were met with two potentially catastrophic changes, both of which Beanstalk turned from show-stoppers into non-issues.

The first was that our internet connection was no where near as reliable as we had hoped. Plans change, and we went from an expected wired+broadband connection down to a mobile WiFi hostspot. At 2kbps we were not going to be able to deploy changes from our local development machines. Luckily, with everything in Beanstalk, this wasn’t an issue at all. We simply needed enough bandwidth to load up the Beanstalk web app and click deploy. After that Beanstalk’s hefty connection out of Rackspace handled the rest.

The second change was a switch from four app servers to twenty-two. We had always planned to ramp up our app servers, but the specifics of keeping all those servers in sync was never discussed. The more we learned from our host the more difficult things became. From configuring on-demand shared filesystems to setting up periodic syncs everything involved downtime and risk. In the end we simply added each additional app server to Beanstalk and forgot about the rest. With Beanstalk controlling our deployments we simply needed to click deploy and all 4, 5, 6, 7 … 22 app servers were updated without fail.

Mark Huot Technology & Development Director, Happy Cog