Here is the write up on how to Use Git, GitHub and TeamCity to push-build-test your Windows Azure Web Sites (WAWS) and by means of Continuous Delivery (CD) to Windows Azure. Because the Windows Azure Platform is so openly manageable and supports open standards and protocols Not Only Microsoft (noMicrosoft) products are enabled to take full advantage of the power of no-friction, no-hassle publication to Windows Azure! After reading this blog post you can remove any deployment work from your teams and enable you to put full focus on building your Applications to provide the fastest Business Value too! We are also going to run TeamCity on a Windows Azure Virtual Machine (VM or WAVM).

This scenario is for those who already have and/or want to have complete and full control of their Continuous Integration environment and like to use noMicrosoft products. The rest of us can just very happily go on and use Team Foundation Service TFService. (You can if you want have Gangnam Style for your CD! Sorry couldn’t resist.)

Shameless plug and “comical” side note

The Windows Azure Fast (AzureFast) contest is still on-going it seems and you can check (and if you like support with votes and comments) my video “Windows Azure ‘Super Fast’ Web Sites” here:

• Windows Azure ‘Super Fast’ Web Sites on the Windows Azure Fast Web Site
• Windows Azure ‘Super Fast’ Web Sites on the Windows Azure Facebook Page
• If both of those have moved away here is Windows Azure ‘Super Fast’ Web Sites on my YouTube account

This is a true story and I hope you like it as much as I enjoyed producing it!

Back to order with a gentle disclaimer: There is serious risk this post will run long since it contains quite a few steps. I am redoing the whole thing as I type this, in order to make sure it’s reproducible and to be able to get the screen shots for this post.

Background

At the recent Windows Azure Conf (AzureConf) I did a session entitled Continuous Delivery Zen with Windows Azure:

Html 5 embedded version here (or go to the link above):

I wanted to prove that any kind of third party tooling-interaction works well with Windows Azure. When I asked my friends at JetBrains if it was possible to use TeamCity to do Continuous Delivery they naturally said ‘yes’ and some digging and researching ensued. It was not completely easy to nudge everything into place but most of the steps were really powerful and easy to perform.

In the video I make a promise at 46:18 that I would write this blog post after the conference. There was just one more thing I wanted to fix in my scenario ;~) before this blog post. That thing – pushing to Git in Windows Azure from TeamCity turned out to be a little tricky for a relative Git n00b such as myself so that, naturally, took a while. But with out any (further) ado here are the steps to reproduce for your projects:

What we are going to do

1. Create a Windows Azure Virtual Machine (VM) with Windows Server 2012.
2. Remote to our server and install TeamCity on it.
3. During the installation I will add a data disk to the machine stored in Windows Azure Storage. This disk will be used by TeamCity to store data and configuration related to the running of my TeamCity Services.
4. Also I will configure port forwarding on port 80 in the Windows Azure Portal to my VM and configure the local firewall on my Windows Server to allow traffic on this port. The same port will naturally be configured in the TeamCity setup wizard.
5. Use temporary storage for the TeamCity Build Agent.
6. Enable MSBuild on the VM
7. Install Git on my VM.
8. Create a new Windows Azure Web Site where our site will be deployed.
9. Set up my Web Site to support Git publishing. I will NOT set up GitHub integration for the Web Site. TeamCity will be monitoring my GitHub repository for any changes and pull that code when I push.
10. Create a repository on GitHub to store my web site code.
11. Clone this repository to my local machine.
12. Create a brand new template ASP.NET MVC 4.0 Web Site in the local repository with a Unit Test project.
13. Push my web site to the GitHub repository.
14. Configure TeamCity to pull any changes from my Repository, build the code, run the Unit Tests and push my Web Site on to Git for my Web Site in Windows Azure.

After all of this, each time TeamCity pushes a new version to Git on Windows Azure that server will (once again build my Web Site) version my push and deploy my new version of the Web Site.

As you can see there are a LOT of steps to be done here. So let’s get going!

Taking advantage of the redeploy old version feature in the Windows Azure portal

A quick note that compared to my demo at AzureConf the last steps have changed slightly. That was the tricky change I wanted to get done before this write up. At AzureConf, I mentioned that I use WebDeploy. This version below does not. Instead I push my code using a TeamCity Command Line task to push my site to my Git repository in Windows Azure. The advantage of this approach is that the Windows Azure Git sever will keep a record of my old deployments and enable me to “step back in time” by redeploying an old version of my site. This process takes seconds only and is very powerful. I will show you this near the end of my post. (Don’t scroll down just read on, dear reader.)

Step 1) Create a Windows Azure Virtual Machine with Windows Server 2012

Nothing could be simpler than getting full control of your very own VM in Windows Azure! In the Windows Azure Portal you simply click New => Compute => Virtual Machine => Quick Create. Set a VM name, which is the dns name. Mine is teamcityrocks.cloudapp.net. Select Windows Server 2012, a small will due. Set the Administrator password. Choose the location. I picked North Europe, aka. Dublin, Ireland. Hit “Create Virtual Machine”!

It takes Windows Azure a few minutes to allocate a slot in a physical VM hosting server, download a fresh image of our server to it, launch it, configure it and hook it into the network and load balancers to supply it with it’s DNS name. Great time to fetch coffee. Or beer.

Step 2) Remote to the VM and install TeamCity

In fact I will not click the “Finish” step in the TeamCity installation wizard until step 4.

A newly created VM in Windows Azure has only one endpoint configured on it. That’s the Remote Desktop endpoint. If that was not in there it would be impossible to connect to the server.

All you need to do now is hit connect at the bottom of the page and log in using the credentials you set up for ‘Administrator’ when you created the VM.

Et voilà! ;~) (I know just a few phrases of French.)

Next download the latest release of TeamCity on the JetBrains Web Site. In my case that turned out to be TeamCity 7.1.2. TeamCity has a 60 day evaluation period which is perfect for this demo. *

* Update: JetBrains are really cool in that they are able to bet their business on that users want to continue using their product(s). It is in fact the Enterprise License that has a 60 day trial. If you don’t have more than 20 Build Configurations in your TeamCity server the Proffesional license is not limited for free usage or time. Very cool JetBrains and thank you for your input

Note: Naturally you can install TeamCity any way you please or you can use a pre-existing already installed TeamCity Server and simply skip this step.

Follow all of the default installation steps until “Specify Server Configuration Directory”.

Step3) Add a Data Disk to the VM for TeamCity data storage

You are able to specify where you want to store data for TeamCity during this setup step. If you open up Windows Explorer on the VM you’ll note that you have two disks by default on the VM:

• C: the OS disk on which you are currently installing TeamCity.
• D: Labeled ‘Temporary Storage’. According to the WAVM documentation “Each VHD also has a temporary storage disk, which is labeled as drive D. The applications and processes running in the VM can use this disk for transient and temporary storage.” We will use this disk later as a temporary work location for TeamCity.

We want one more disk. This dis will be for persistent storage of data TeamCity will want to keep. Two really cool thing about Cloud features that you can take advantage of in Windows Azure in this case are “self-provisioning” and “pay-as-you-go”. We are simply going to requisition a data disk which will be stored as a “Page Blob” in Windows Azure Storage (WA Storage blog). The size of this page blob as we define it isn’t really very important because even if I define a disk of say 1 TB and only use a fraction of this for storage I will only pay for what I use in Windows Azure! Amazing.

To add a data disk to a VM you go to the Windows Azure portal, browse to the VM and select “Attach” => “Attach empty disk”. I went with a 100 GB data disk for this demo.

It is also possible to later detach data disks from VMs right here in the portal if you want to copy that data somewhere else. Or you can upload a pre-existing data disk as a VHD to Windows Auzre Storage and migrate your current TeamCity server to a VM in Windows Azure. Possibilities are endless.

Once the adding of the disk completes you head back to your Remote Desktop connection and open up “Disk Management”. Immediately your system recognizes the new disk and prompts you to “Initialize Disk”. Initialize it and then right click the disk in Disk Management and select to create a “New Simple Volume…”. Follow that wizard and perhaps label your disk “Data”. Once that completes you have a brand new empty data disk attached to your VM:

We return now to the TeamCity installation wizard we are following and specify our new data disk for “server configuration directory”. In my case the drive was “F:\”. The TeamCity installation now commences.

Step 4) Set port 80 in the Windows Azure portal, in the server fire wall and in TeamCity

After a while you get to specify ‘80’ for the “TeamCity server port”. In order for traffic to flow from the Internet to TeamCity you have to configure and open up that path through the data center and into your Windows Server.

In the Windows Azure portal select the VM and go to “Endpoints” => “Add Endpoint”.

I configured it to be “TeamCityHttp” with external and internal port 80. Our Windows Server does not have the Internet Information Server Service activated by default so that port should be available to us.

Once that port is configured in the Windows Azure Load Balancers (by defining it in the Windows Azure Portal) we return to the Remote Desktop and open up “Windows Firewall with Advanced Security”. Got to “Inbound Rules” => “New Rule…”. Select “Port”, “Specific local port” = 80 and “Allow the Connection”. Finish the Wizard by naming your Firewall rule aptly:

Step 5) Use temporary storage for the TeamCity Build Agent

After the TeamCity Server has installed the next step is to configure the default Build Agent. TeamCity works in such a way that the Service will control a set of Build Agents which will perform the actual builds for the Service. The only thing I have configured here is to use our VM Temporary storage for Build Agent “tempDir” and “workDir”. They are set to ‘d:\temp’ and ‘d:\work’ respectively.

Next step is to configure the accounts for running the TeamCity Services. I selected “SYSTEM account” for both of them; Build Agent and TeamCity Server.

The installation completes and you get to browse to the TeamCity server on the local machine, the VM over Remote Desktop.

Hit “Proceed” and let TeamCity create it’s database, initialize and start up. Accept the “License Agreement for JetBrains® TeamCity™”.

Create the TeamCity administrator account.

And so the installation of TeamCity on a Windows Azure Virtual Machine with Windows Server 2012 concludes!

You can now browse to the TeamCity Services using the VM:

Or even better in your browser as usual:

We will configure the TeamCity Services a little bit later. There is one more thing we need to do on our VM and that is to install Git.

Step 6) Enable MSBuild on the VM

At this step you have to make sure you can run the latest version of MSBuild on the build server. You will also need the correct Visual Studio Build Process Templates on the VM. This might mean you have to install Visual Studio on the Build Server. What you need to do to make sure you live up to the Microsoft License requirements is up to you. For this demo I will copy my msbuild folder from C:\Program Files (x86)\MSBuild to the same location on the VM. This can be accomplished with a copy from local machine and paste to the build server. Please note that this is probably not the way you need to set this up for a production build environment, but I’ll leave this step up to you.

Step 7) Install Git on the VM

When we (finally get to) push our Web Site to Windows Azure we will do so using Git. Therefor we need to download Git for Windows and install it on the VM. I happened to get the Git-1.8.0-preview20121022 version.

The only thing I changed in the Git installation Wizard was to “Run Git from the Windows Command Prompt” because that is what we are going to do from TeamCity when we deploy our WAWS.

Step through and finish the rest of the installation. Verify that your server is property Git-Enabled by opening up a command window and typing the command ‘Git’.

You should get a response saying that git offers you a set of commands to run.

Step 8) Create a new Windows Azure Web Site where our site will be deployed

Now it’s time to start creating the site we want to deploy to in Windows Azure. This task is accomplished in the Windows Azure Portal under New => Compute => Web Site => Quick Create. Enter a url and a region and hit “Create Web Site”.

Seconds later the WAWS is created:

And you can browse to a nice template Web Site with a public Url in Windows Azure: (My site was called continuousdeliverywithteamcity.azurewebsites.net.)

Step 9) Set up the Web Site to support Git publishing

Click on the Web Site in the portal and then on “Set up Git publishing”. A Git repository for your WAWS is created in Windows Azure.

And finishes:

Right now you can go and integrate with a project you already have on CodePlex, GitHub or BitBucket. We are NOT going to do that here. Instead we want TeamCity to monitor changes in our Souce Control system and take action as we push/commit.

Another option is to not use Git at all but instead opt to integrate with the Team Foundation Service that Microsoft provides. This may be a better option for teams that are more used to running TFS.

Step 10) Create a repository on GitHub to store my web site code

For this demo I am creating a new empty GitHub repository. Naturally you can use any repository with a Web Site you already own. For the demo feel free to follow along and just create a new demo repository.

I selected to add a .gitignore file for csharp. We will make one modification to this file later on.

Step 11) Clone this repository to my local machine

I like to use GitHub for Windows and the Git Shell.

Copy the Git Http Read+Write Access Url from the newly created GitHub repository. Mine was https://github.com/MagnusDemo/teamcitydemowebsite.git

Open up the Git Shell in a new directory and use the git clone command:

git clone https://github.com/MagnusDemo/teamcitydemowebsite.git

You now have the Git repository cloned from GitHub to your local system.

Step 12) Create your Web Site in the Git repository

Naturally, again, you can use any Web Site you like here. It can be ASP.NET or PHP out of the box and you can use your favorite (Visual Studio) IDE. Honestly use notepad if you want! I’m going to go with Visual Studio and do File => New => Project => ASP.NET MVC 4.0 (and .NET Framework 4.5) Web Application:

I also opt to create a Unit Test project:

When the project has been created I make just a simple modification to the template on the index page: “”

You can naturally configure your VM to run Microsoft Unit Tests framework but we will swap that for NUnit in this demo. Remove the reference to Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll in your Unit Test Project. Then right click on “References” and select “Manage NuGet packages…” Search for “NUnit” and install it in your project. Then go into your HomeControllerTests.cs and correct it to use NUnit:

Now you should hit F5 in Visual Studio to ensure your Web Application runs locally. Also you should run the Unit Tests in your Solution to see that those run OK.

Step 13) Push my web site to the GitHub repository

When you use a modern project template in Visual Studio it is going to use the NuGet Pagage Manager and install some packages as library references along with your template. For the default setup I chose there are 30 (!) such packages. When you build your code it is possible to “Enable NuGet Package Restore” for your solution. We are not going to use that method here though I’m certain that is also a good option.

Do NOT do this:

What happens when you do this is the build process when you build your code will include a step for downloading any missing NuGet packages before building your code. Again we are not doing this.

Instead we are going to make one small change to our .gitignore file which was added when we created the GitHub repository. We will comment out the ignore for “packages” which will make it so that we will commit all of our packages along with our Solution.

The default is “packages” and we change that to “#packages” effectively removing that ignore.

Now it’s time to run the following Git commands at our Git Shell prompt:

• git add . (Adds all of our files to the Git Repository.)
• git commit –am “Initial Commit.” (Confirms or Commits that my changes – all of the adds – are correct.)
• git push (Pushes the local commit to the remote repository – the GitHub source.)

323 files just got pushed to GitHub.

(Can I just comment that I think that’s a LOT of files for being just a template web application? But that’s a side note and not really important right now.)

Over in your repository on GitHub in the browser you can now verify that indeed the code has been pushed there.

Intermission

Wow – that’s a lot of work that we have just done! Time to just step back and see what we’ve done so far, where we are at, and what remains:

• We have a WAVM running Windows Server 2012, TeamCity and Git.
• We have a WAWS configured with Git.
• We have a GitHub repository.
• We have a Solution with a Web Application and a Unit Test project pushed to that repository.

The thing we have not done is configure TeamCity to tie all of this together. That’s our next and final big step. The last step is done automatically by Windows Azure and then we have some verifications to do. Plus I promised you I’d cover Windows Azure integrated Web Application Version Control and the “stepping back in time” feature.

Step 14) Configure TeamCity

Here is what we want TeamCity to do:

• Notice when we push changes to the GitHub repo.
• Pull those changes down locally on the VM.
• Build the Solution.
• Run the Unit Tests.
• If everything succeeds go ahead and push the new Web Application to the Git Server on Windows Azure.

Go back to the browser and the TeamCity tab and click on “Create a project”. Name the project appropriately.

On the next screen select “Create build configuration” and enter the following data:

Then click on “VCS Settings >>”

On the screen “Version Control Settings” I select “VCS Checkout Mode” = “Automatically on Agent (if supported by VCS roots)” and then click above that on “Create and attach a new VCS root”.

Select “Git” from the drop down “Type of VCS”.

The “Fetch Url” is the copied from the GitHub Repository under the “Git Read-Only” link. Mine is git://github.com/MagnusDemo/teamcitydemowebsite.git

I made one more change here. For some reason I had problems with the TeamCity git version for pulling code from GitHub in this demo. Instead of figuring out why that is I set the setting “Path to Git” = “C:\Program Files (x86)\Git\bin\git.exe”.

No other settings or changes are required in this screen. At the bottom “Test Connection” to make sure TeamCity can read your GitHub repository. Then hit “Save”. Back in “Version Control Settings” click “Add Build Step >>” and then select “Runner Type” = “Visual Studio (sln)” since we are building a Visual Studio Solution.

Name your Build step and select the “Solution File Path”:

The result should be something like this:

Add another build step and make it an “NUnit“ step.

In this screen the most important part is to set the path to the .dll containing your tests. Mine was “TeamCityDemoWebApplication.Tests\bin\Release\TeamCityDemoWebApplication.Tests.dll”. Also set .NET Runtime Version to 4.0.

Your result after this should be something like this: Two Build Steps 1) Build in Release mode. 2) Run Unit Tests.

Next go back to the TeamCity Project overview screen (one step up) and “Create build configuration” again. This configuration is for pushing the Solution to Git in Windows Azure.

Select our existing “VCS root” and check “Display options” for “Show changes from snapshot dependencies”. Then “Add Build Step” = “Command Line”. On this screen there are a few specifics we need to control.

First we need to set the path to the .exe we want to use. That would be the path to the git.exe installed on the VM previously. In my case it’s “C:\Program Files (x86)\Git\bin\git.exe”.

Second we need to supply Command Parameters to make a git push. The base for this command is

git push <destination> <branch>

The command is “git push” the destination will be figured out below and the branch will be ‘master’ which is the default branch and the only one we have.

Regarding the destination we have to figure out how to git push with user credentials in one go. There are a few things we need to learn here:

A) supplying username and password in a git request means prepending them to the destination url on the following format:

https://<username>:<password>@<destination url>/<repositoryname>.git

If we don’t supply the credentials we will be prompted for username and/or password and this won’t work when running a command line on a build server. We have to push with no extra prompts.

B) We need to find the credentials to use for access to our Git Repository in Windows Azure for our WAWS.

Each user has ONE (and only one) set of personal credentials for accessing deployments in Windows Azure. You can set and reset those in the portal. The same credentials will be used for all of your WAWS. These are NOT the credentials we will be using here.

Additionally each WAWS you create also gets a set of Web Site specific credentials. This is one special set of credentials for one WAWS. There is a really neat trick to how you can find the base for this url in the WA Portal.

Go to the WAWS in the Windows Azure Portal and click into the Configure section. Copy the Git “Deployment Trigger Url”. Mine is:

https://$continuousdeliverywithteamcity:K6bTXnAxtwNDbGDpuNC8k3yBcJyw4y2x6lrSZKJP6G3hKmCvb6Wr6XQ0j0g4@continuousdeliverywithteamcity.scm.azurewebsites.net/deploy

Yes I am showing you my username and password here but it’s not a problem. By the time I publish this blog post I will have deleted this demo WAWS so you won’t be able to use the credentials for anything.

The only thing we need to do now is modify the url slightly at the end and replace “deploy” with our WAWS name:

https://$continuousdeliverywithteamcity:K6bTXnAxtwNDbGDpuNC8k3yBcJyw4y2x6lrSZKJP6G3hKmCvb6Wr6XQ0j0g4@continuousdeliverywithteamcity.scm.azurewebsites.net/continuousdeliverywithteamcity.git

Putting this Url together with the rest of the command makes the whole thing this in my case:

git push https://$continuousdeliverywithteamcity:K6bTXnAxtwNDbGDpuNC8k3yBcJyw4y2x6lrSZKJP6G3hKmCvb6Wr6XQ0j0g4@continuousdeliverywithteamcity.scm.azurewebsites.net/continuousdeliverywithteamcity.git master

Paste your version into “Command Parameters” and hit Save.

This should be similar to your result:

There are two more thing we want to configure in the TeamCity Deploy Step: 1) Build Triggers and 2) Dependencies

Start by clicking on “Dependencies“ => “Add new Snapshot Dependency” and select the “Full Build” do depend on.

Next click on “Add new artifact dependency” and set that to the same “Full Build” (selected by default for me). Configure the rest of the settings same as this screen shot:

Now click on “5 Build Triggers” => “Add new trigger” => “VCS Trigger” and configure it accordingly:

This should be the result of adding the Build Trigger:

Now you have your TeamCity Project set up with two Build Configurations.

Installation and configuration is DONE! I know there has been a lot of configuration in this step of the blog post but NOW it’s time to test this bad girl!

Test the result

Click on “Projects” in the top left and you have a screen like this. Click on “Run” for the “Full Build” project and hopefully, fingers crossed, some magic will happen!

When the Full Build and Deploy Build Configurations both have run it should look like this:

The last thing TeamCity does is push to the Git repository in Windows Azure for our WAWS.

Next the Windows Azure Git Server will take over and do the following with the Web Site:

• Build it. (Again)
• Store the new version for the future.
• Deploy the new version.

All this “just happens” for us automatically. It’s the default setup in Windows Azure.

Go back to the tab you had open to your site (or reopen one if you have closed it). My site url was http://continuousdeliverywithteamcity.azurewebsites.net/. Here is my result:

Make the following change to one of your unit tests and push that change to GitHub:

Assert.Fail(“Making sure TeamCity runs my tests.”);

This should be similar to your result. As you can see the Full Build failed on the build server because one test failed.

Remove the fail from the test and make a new change to your index page. Now we will deploy a newer version of the code and see that this is automatically deployed to our WAWS:

After pushing these TWO changes to GitHub the build passes through my TeamCity Service and deploys to my WAWS:

The “Step Back in Time” feature

After having done all of this it’s nice to get a little treat! We have now deployed twice to our WAWS. One time with the message “TeamCity Deployed this!” and the second time with the message “TeamCity Deployed this! New Change!”. If you go to the portal and click on Deployments for the WAWS you will see them both:

The mind-blowing treat is that you can click on the previous deployment and select “Redeploy” at the bottom of the page. This will redeploy your previous Web Site version in a matter of seconds!

When you browse back to the site and refresh you can see the result:

Yes that’s right you are back to the previous version of the site! But wait, there’s more! We are running this Web Site in one single free instance. Even if you scale this Web Site out to multiple instances, say three instances serving your Web Application for three times the throughput, the steps back and forth in history are just as fast. All of your Web Sites, even scaled out, will redeploy any version you pick in sync! That is just #awesome!

(This is the reason I did not want to use WebDeploy from our TeamCity server because then I’d effectively bypass this built in functionality. Sure I can do the same from TeamCity by running a previous Deploy build step but that’s not integrated is it!)

Summary

Rounding off I re-remind myself to write shorter blog posts. #EpicFail

Most of this setup is really simple and straight forward. TeamCity is an excellent tool for those who love to take 100% control of their build environment, but I’m sorry to say it’s advanced nature takes some learning and getting used to. Once you master that tool though you have awesome power! (And awesome responsibility.)

Before I sign off I’d like to take this opportunity to thank the people who helped out in my research with git and TeamCity stumblings. I want to call them out and give credit where credit is due:

• Maxim Manuylov and Hadi Hariri at JetBrains for helping me set up and configure TeamCity.
• David Ebbo for assisting me with git and that narly one-liner I needed to push from TeamCity to WAWS.
• Brady Gaster for cheering on and giving helpful hints and encouragement.

This is it folks – Every time my code is pushed it gets built, tested and deployed automagically using TeamCity in Windows Azure to my Windows Azure Web Site. Build fails and Unit Test fails will stop the deployment. If some other kind of logical error gets deployed I can step back in time and redeploy an old version of my site in a matter of seconds!

HTH – Cheers,

Magnus