Upgrading Azure Storage Client Library to v2.0 from 1.7

I upgraded Azure Storage to version 2.0 from 1.7 and I've found a number of differences when using storage. I thought how I'd document how I upgraded these more awkward bits of Azure Storage in version 2.0.

DownloadByteArray has gone missing

For whatever reason DownloadByteArray has been taken from me. So has DownloadToFile, DownloadText, UploadFromFile, UploadByteArray, and UploadText

Without too much whinging I'm just going to get on and fix it. This is what was working PERFECTLY FINE in v1.7:

public byte[] GetBytes(string fileName)
{
    var blob = Container.GetBlobReference(fileName);
    return blob.DownloadByteArray();
}

And here is the code modified to account for the face that DownloadByteArray no longer exists in Azure Storage v2.0:

public byte[] GetBytes(string fileName)
{
    var blob = Container.GetBlockBlobReference(fileName);
    using (var ms = new MemoryStream())
    {
        blob.DownloadToStream(ms);
        ms.Position = 0;
        return ms.ToArray();
    }
}

How to get your CloudStorageAccount

Another apparently random change is that you can't get your storage account info in the same way as you used to. You used to be able to get it like this in Storage Client v1.7:

var storageAccountInfo = CloudStorageAccount.FromConfigurationSetting(configSetting);
var tableStorage = storageAccountInfo.CreateCloudTableClient();

But in Azure Storage v2.0 you must get it like this:

var storageAccountInfo = CloudStorageAccount.Parse(
            CloudConfigurationManager.GetSetting(configSetting));
var tableStorage = storageAccountInfo.CreateCloudTableClient();

Why?.. not sure. I have had problems with getting storage account information before so maybe this resolve that.

What happened to CreateTableIfNotExist?

Again, it's disappeared but who cares.. Oh you do? Right well let's fix that up. So, in Azure Storage Client v1.7 you did this:

var tableStorage = storageAccountInfo.CreateCloudTableClient();
tableStorage.CreateTableIfNotExist(tableName);

But now in Azure Storage Client Library v2.0 you must do this:

var tableStorage = storageAccountInfo.CreateCloudTableClient();
var table = tableStorage.GetTableReference(tableName);
table.CreateIfNotExists();

Attributes seem to have disappeared and LastModifiedUtc has gone

Another random change that possibly doesn't achieve anything other than making you refactor your code. This was my old code from Storage Library Client v1.7:

var blob = BlobService.FetchAttributes(FileName);
if (blob == null || blob.Attributes.Properties.LastModifiedUtc < DateTime.UtcNow.AddHours(-1))
{
    ...
}

But now it should read like this because thought it looks prettier (which it does in fairness).

var blob = BlobService.FetchAttributes(FileName);
if (blob == null || blob.Properties.LastModified < DateTimeOffset.UtcNow.AddHours(-1))
{
    ...
}

Change your development storage connection string

This is just a straight bug so that's excellent. I was getting a useless exception stating "The given key was not present in the dictionary" when trying to create a CloudStorageAccount reference. To resolve this change your development environment connection string from UseDevelopmentStorage=true to UseDevelopmentStorage=true;DevelopmentStorageProxyUri=http://127.0.0.1 then it will magically work.

Bitch and moan

Apologies for the whingy nature of this post, I'm quite a fan of Azure but I have wasted about 3-4 hours with this "upgrade" from Azure Storage Client Library 1.7 to 2.0. It's been incredibly frustrating particularly since there seems to be no obvious reason why these changes were made. I just can't believe the amount of breaking changes when I haven't really written that much Azure storage code.

Randomly taking out nice methods like DownloadByteArray and DownloadText is surely a step backwards no? Or randomly renaming CreateIfNotExist() to CreateIfNotExists()... what is the point in that?!

I remember when upgrading to ASP.NET 4 from 3.5, I spent very little time working through breaking changes and I have 100 times more .NET code than I do Azure Storage code. As well as that, I was well aware of the many improvements with that .NET version update, with this Azure Storage update I have no idea what I'm getting. No matter the improvements, it is just an Azure storage API and this number of breaking changes, often for the benefit of syntax niceties is just unnacceptable.

Oh, if you are still in pain doing this I have found a complete list of breaking changes in this update along with minimal explanations here.

Export and back up your SQL Azure databases nightly into blob storage

With Azure I have always believed, if you can do it with the Azure Management Portal then you can do it with a REST API. So I thought it would be a breeze to make an automated job to run every night to export and back up my SQL Azure database into a BACPAC file in blob storage. I was suprised to find scheduling bacpac exports of your SQL Azure databases is not documented in the Azure Service Management API. Maybe it is because the bacpac exporting and importing is in beta? Nevermind. I successfully have a worker role backing up my databases and here's how:

It is a REST API so you can't use nice WCF to handle all your POST data for you but there is a trick still to avoid writing out all your XML parameters by hand and instead strong typing a few classes.

Go to your worker role or console application and add a service reference to your particular DACWebService (it varies by region):

• North Central US: https://ch1prod-dacsvc.azure.com/DACWebService.svc
• South Central US: https://sn1prod-dacsvc.azure.com/DACWebService.svc
• North Europe: https://db3prod-dacsvc.azure.com/DACWebService.svc
• West Europe: https://am1prod-dacsvc.azure.com/DACWebService.svc
• East Asia: https://hkgprod-dacsvc.azure.com/DACWebService.svc
• Southeast Asia: https://sg1prod-dacsvc.azure.com/DACWebService.svc

Once you import this Service Reference you will have some new classes that will come in handy in the following code:

//these details are passed into my method but here is an example of what is needed
var dbServerName = "qwerty123.database.windows.net";
var dbName = "mydb";
var dbUserName = "myuser";
var dbPassword = "Password!";

//storage connection is in my ServiceConfig
//I know these the CloudStorageConnection can be obtained in one line of code
//but this way is necessary to be able to get the StorageAccessKey later
var storageConn = RoleEnvironment.GetConfigurationSettingValue("Storage.ConnectionString");
var storageAccount = CloudStorageAccount.Parse(storageConn);

//1. Get your blob storage credentials
var credentials= new BlobStorageAccessKeyCredentials();
//e.g. https://myStore.blob.core.windows.net/backups/mydb/2012-05-22.bacpac
credentials.Uri = string.Format("{0}backups/{1}/{2}.bacpac",
    storageAccount.BlobEndpoint,
    dbName,
    DateTime.UtcNow.ToString("yyyy-MM-dd"));
credentials.StorageAccessKey = ((StorageCredentialsAccountAndKey)storageAccount.Credentials)
                                   .Credentials.ExportBase64EncodedKey();

//2. Get the DB you want to back up
var connectionInfo = new ConnectionInfo();
connectionInfo.ServerName = dbServerName;
connectionInfo.DatabaseName = dbName;
connectionInfo.UserName = dbUserName;
connectionInfo.Password = dbPassword;

//3. Fill the object required for a successful POST
var export = new ExportInput();
export.BlobCredentials = credentials;
export.ConnectionInfo = connectionInfo;

//4. Create your request
var request = WebRequest.Create("https://am1prod-dacsvc.azure.com/DACWebService.svc/Export");
request.Method = "POST";
request.ContentType = "application/xml";
using (var stream = request.GetRequestStream())
{
    var dcs = new DataContractSerializer(typeof(ExportInput));
    dcs.WriteObject(stream, export);
}

//5. make the POST!
using (var response = (HttpWebResponse)request.GetResponse())
{
    if (response.StatusCode != HttpStatusCode.OK)
    {
        throw new HttpException((int)response.StatusCode, response.StatusDescription);
    }
}

This code would run in a scheduled task or worker role to be set for 2am each night for example. It is important you have appropriate logging and notifications in the event of failure.

Conclusion

This sends off the request to start the back up / export of database into a bacpac file. The success of this is no indication that the back up was successful, only the request submition. If your credentials are wrong you will get a 200 OK response but it the back up will fail silently later.

To see if it has been successful you can check on the status of your exports via the Azure Management Portal, or by waiting a short while and having a look in your blob storage.

I have not covered Importing because, really, exporting is the boring yet important activity that must happen regularly (such as nightly). Importing is the one you do on the odd occassion when there has been a disaster and the Azure Management Portal is well suited to such an occassion.

Add Cloud and Local service configurations when there is only a Default

I would like a cloud and local versions of my service configurations to change some of the configuration settings in different deployments to either Local or Cloud. Sometimes you have only one service configuration called "Default" which is not enough. This guide shows how to add more.

So you only have one Service configuration file? You would be better off with more than one since it is likely you would want to have different configuration setting defined for different deployments e.g. storage emulator used for local deployments and Azure for cloud deployments.

Right click on one of your roles and go to properties. On the Service Configuration drop down select <Manage...>

On the Manage Service Configurations dialogue box you want to click the Default configuration (for example) and click 'Create copy' and rename it to 'Local'. Do the same again for 'Cloud' and then remove 'Default'.

You should now have two versions of the service configuration, both Local and Default.

These two versions of ServiceConfiguration, Local and Cloud, are now in existence which conforms with the standard set up when you have a new Azure Service. Of course, these service configurations will be duplicates of each other so you will need to make the appropriate changes in them to reflect that, e.g. set the storage account to point to Azure on Cloud and storage emulator on Local service configurations.

Interesting little Azure points

I often discover little interesting facts about Azure while I am using it that I would blog about but would only last one line/paragraph so I don't bother. This is what I should tweet about I suppose but I get the feeling my tweets (@britishdev) get lost within 10mins surrounded by your little cousin Kate reporting, "@KatezLulz: OMG I just ate broccoli!!!1! :p" and I can't compete with that. So here is a blog post which I will try to keep updates with short and sharp points about Windows Azure.

#1 What happens to your Azure deployment if your machine is turned off halfway through?

It depends where you are in your deployment. As long as the uploading phase is done you will be fine. Sort of. Visual Studio and the management portal make it look a lot more simple than it is to deploy an Azure package but it does it in steps like: Upload, Create, Start instances etc (or along those lines anyway). So if you switch your computer off after you have uploaded you will not lose everything and have to start again but it wouldn't have deployed fully either. Next time you log in to the management portal you will find your instances are there but "Stopped" so you simply need to start them using the great big Start button in the Azure management portal and it is done. Trivial.

#2 You can filter your Azure Table Storage viewer in Visual Studio

Type filters into the bar such as Timestamp gt datetime'2012-01-19T10:00:00Z'. This will get every record after 19^th Jan 2012 10am. Here is a full list of table filters.

Case sensitive Azure storage

Azure storage URLs are case sensitive as you may have noticed. If you have not noticed then: OMG AZURE STORAGE URL ARE CASE SENSITIVE!!1! This is most likely because URL specification states that URLs should be case sensitive so as to ensure different casings should represent different locations. Admittedly, this is rather odd when compared with IIS and subsequently web sites running in Azure.

I am a developer and architect by trade so I am more than aware of the importance in maintaining consistent URLs throughout a web site. Search engines index sites in a case sensitive way and so it is important not to accidentally display duplicate web content on differently cased URLs. To handle this in the past I have always insisted that my teams adopt a lower case policy where by every URL written in code, links, references etc are always written in lower case. This avoids such an SEO disaster. I also create a redirect rule using IIS Rewrite Module to 301 redirect any URLs containing uppercase characters to its lowercase version.

With this policy in mind it should not be difficult to now use Azure storage by simply maintaining these standards. It is wise to create a repository class that will handle all interaction with Blob, Tables and Queue storage that can abstract common rules away from the developer each time they wish to use them. One of these rules would be to ensure that when saving to these repositories the filenames and containers are lower cased using .ToLower() (in C#). Also when getting an object from storage you could also ensure that the file name requested is also lower cased in the same way.

This does not however prevent users accessing the URL using uppercase but really according to the URL specs they should not and using smart code you can most likely avoid this from happening. For example, if links are only ever displayed in lowercase someone if very unlikely to ever access it using uppercase.

Azure case sensitivity conclusion

So in summary, although it is an odd inconsistency between IIS and storage it is only a trivial programming exercise to enforce and a minor coding standard to communicate to your team. This will ensure that storage is always used as intended allowing you to reap the great benefits of using Windows Azure Storage.

Using the Azure API to see a deployment status using .NET

See the status of your Windows Azure deployments using the Windows Azure Service Management REST API. Since this is REST based you can use any framework or programming language that can make web requests. Python, Java etc here is .NET.

Here is a short bit of C# code that will allow you to call the part of the Windows Azure Service Management REST API that deals with getting the status of your hosted service in Azure.

You can see how the REST API is expected to be used here at Get Hosted Service Properties. This code accesses that API:

static void Main(string[] args)
{
    var subsctiptionId = "f62e5e87-5c76-4a94-9136-794fae3eff16";
    var hostedService = "colintest";
    //I have another post that details how GetCertificateByThumbprint method works:
    //http://www.britishdeveloper.co.uk/2012/01/adding-certificate-to-request-in-net.html
    var certificate = GetCertificateByThumbprint("23A43AE81F15CB000000000000000000000000000");

    var statusApiUrl = string.Format(
       "https://management.core.windows.net/{0}/services/hostedservices/{1}?embed-detail=true",
       subsctiptionId, hostedService);
    var hostedServiceStatus = new Uri(statusApiUrl);
    Console.WriteLine("Hosted service status");
    MakeApiRequest(hostedServiceStatus, certificate);
    
    Console.ReadKey();
}

private static void MakeApiRequest(Uri requestUri, X509Certificate2 certificate)
{
    var request = (HttpWebRequest)HttpWebRequest.Create(requestUri);
    request.Headers.Add("x-ms-version", "2011-10-01");
    request.Method = "GET";
    request.ContentType = "application/xml";
    request.ClientCertificates.Add(certificate);

    try
    {
        using (var response = (HttpWebResponse)request.GetResponse())
        {
            Console.WriteLine("Response status code: " + response.StatusCode);

            using (var responseStream = response.GetResponseStream())
            using (var reader = new StreamReader(responseStream))
            {
                Console.WriteLine("Response output:");
                Console.WriteLine(reader.ReadToEnd());
            }
            Console.WriteLine("");
        }
    }
    catch (Exception e)
    {
        Console.WriteLine(e.Message);
        throw e;
    }
}

Since you have used ?embed-detail=true in the querystring this add extra detail. From here you can get all sorts of useful information such as: Status e.g. Running or DeploymentSlot e.g. Production.

Note: The GetCertificateByThumbprint(string thumbprint) method I used is of course simplifying attaching a certificate to the request for the sake of conciseness. You can have a look at what this method is doing here at attaching a certificate to a WebRequest.

Do not use iisreset in Azure

So you have remote desktop in to one of you Azure instances and you are free to do anything you like right? Wrong! Do not change things! And as I found do not use IISReset.

I have heard many times that remoting in to an Azure instance is for looking and debugging only, NOT for changing things. In fact I have even given this advice to many clients I have spoken to about Azure. But who am I to practice what I preach?

Really, I give the advice of not changing things when RDPing into an Azure instance because any changes you make will at some point be lost when your instances are automatically updated for you and then redistributed to other machines. Any changes you wish to be permanent on your machine will need to be part of your package.

Anyway, you can see why I avoid making changes, because of a lack of persistence but that doesn't mean I shouldn't do a cheeky IIS reset when trying to fix an issue right? Hmm, wrong.

Do not restart IIS on an Azure instance

I do not know why, there must be some black magic Azure voodoo that goes on after IIS is initialised that doesn't happen when you restart it yourself manually. Anyway, I learnt that it will completely destroy your instance. The site will not respond any longer from that instance. The best way to effectively do an IIS Reset is to Reboot your instance from the Azure Management Portal.

How to run Crystal Reports on Azure

Here is a step by step guide on how to make an ASP.NET project that uses Crystal Reports run successfully on Azure. If you try to run a Crystal Report in your ASP.NET site without the Crystal Reports runtime installed you will receive a "System.Runtime.InteropServices.COMException" with description "The Report Application Server failed".

The problem is that you need to install the Crystal Reports runtime. This isn't a problem with regular hosting since you can just install Crystal Reports on each of your servers and off you go.

With Azure, though, if you remote into the machine and install it, it will work fine until your deployment is redistributed to another machine which it will do at some point due to the nature of cloud computing.

How to install Crystal Reports on your Azure web role

Fortunately it is still easy with Azure. Easy when you know how anyway. Here are the steps you will need to take:

First of all you will need to download the SAP Crystal Reports runtime engine for .NET Framework 4 (64-bit). This should extract as a msi file called CRRuntime_64bit_13_0_2.msi.

In your web application in Visual Studio you should paste this msi file at the route of your web project and include it in the project. Right click it in the Solution Explorer and set its 'Build Action' to 'None' and also set its 'Copy to Output Directory' to 'Always Copy'.

Next you will create a command file to execute this msi file. Create a new text file, call it StartUp.cmd and then save it in the root of your web project (next to the msi). In that file write the following:

@ECHO off

ECHO "Starting CrystalReports Installation" >> log.txt
msiexec.exe /I "CRRuntime_64bit_13_0_2.msi" /qn
ECHO "Completed CrystalReports Installation" >> log.txt

Set the properties of StartUp.cmd to 'Build Action' = 'None' and 'Copy to Output Directory' = 'Always Copy'.

Now in your ServiceDefinition.csdef make this cmd file a start up task by adding the following lines:

<WebRole name="Web" vmsize="Small">
  ...
  <Startup>
    <Task commandLine="StartUp.cmd" executionContext="elevated" taskType="background" />
  </Startup>
</WebRole>

You are now instructing each instance that starts up with your package to run the Crystal Reports msi file that installs the runtime on the instance ready for its use in Azure.

A few Crystal Reports on Azure tips

I ran into a few bits and bobs which caused me unnecessary pain along the seemingly clean process outlined above. I will share them with you in case you do too in no particular order.

• Make sure your .rpt Crystal Report files are set to Build Action: Content and Copy to Output Directory: Copy always.
• Don't be alarmed with how long it takes to deploy. It will take much longer to upload than usual because you are now uploading an extra 77MB of installation files. It took me an hour to deploy on my home connection!
• Ignore all the warnings about how your web project is dependent on various Crystal Report assemblies since Azure will have them just as soon as your installation file runs.
• Configure a remote desktop connection when you do your deployments since it will be invaluable should anything go wrong and at an hour per deployment you don't want to be messing about.
• Visual Studio may have added a load of random assemblies in your web.config you are not aware of and don't need and may even cause problems like log4net.

That is all. Good luck - it's very satisfying when you get it going.

Azure AppFabric Cache billing

I have started using Windows Azure AppFabric Cache service recently and I was confused at how much money it was costing.

I started using a new subscription yesterday and I seem to have already used 53.68 MB of my 128MB cache. In one day?!

However, if I look at my cache in the Azure management portal I am not using it at all!

So where has this 53MB of AppFabric Cache come from? Since I've only put my site live for less than 24hrs I was worried I was blazing through my 128MB/month allowance but then I thought about it logically for a moment. You don't use up a cache you just use it. It doesn't go anywhere.

How AppFabric Cache usage is calculated and billed

With this in mind it becomes obvious what has happened. I set my AppFabric cache up before I got around to deploying my site. So really it has been used since 22^nd December, which is 13 days ago. It is irrelevant that I have deployed a site onto it only yesterday.

Look at the maths: 128MB/31days = 4.13MB/day * 13days = 53.68MB. It is a confusing way of displaying it but still, it makes sense.

Understanding Windows Azure spending limits

If you have logged into the Azure billing portal since 12^th December 2011 you will have noticed it has had a facelift as well as some changes to its functionality. Particularly a billing spending cap.

I just wanted to share my findings on the new feature of spending limits. Now there will be a $0 spending cap on your account when you sign up for a free trial. This ensures you will not be charged during your free trial. Beware though that once this cap is met your free trial service will stop working and you know that will happen just before you prototype demo.

Let me just clear up some things because I was a bit confused with exactly what had changed until I did some digging about. Mainly because my imagination just invented how the spending caps should work and partly because it is difficult to find documentation on this.

If you set up a new free trial FROM NOW you will get the $0 spending limit. If you had one before (presumably 12^th Dec) you have a little note saying you have removed the spending limit. You didn't, you just never got the limit because you registered for a free trial before that the spending cap facility was available. Confusing notification at fault really.

So here is what I found on the payment caps:

• If you are not on a free trial you have no spending cap and you cannot create one ($0 or otherwise)
• If you are on a free trial since before 12^th December 2011 it will say you removed the spending limit
• If you joined a free trial on or after 12^th December 2011 you will automatically have a $0 billing limit
• If you remove the $0 spending charge limit you cannot put it back

My thoughts

I think it is a nice idea to keep people reassured that they will not be billed during a free trial. I understand some people felt hard done by when they were charged during this evaluation period (despite how well communicated and fair the free trial usage limits were... but anyway).

However, I think the Azure team has missed a trick here. Why not allow people to set custom spending limits on their accounts? It'd be very nice to be reassured that I would not be billed more than £x a month if I had a fixed budget. It seems like the infrastructure for such a feature is there but just missing the implementation at this time.

Maybe in time the ones of us willing to invest in Azure will get the same reassurances as the free trial users?

Where are my .NET certificates and their thumbprints? Ask Powershell

I ran into a confusing situation where I was trying to pass a certificate with a PowerShell command but found it incredibly confusing to find my certificates.

I was trying to use a certificate on my local machine that I created at an earlier time but where the hell are they? If you open Microsoft Management Console (type mmc into the start bar) and then add the certificates snap in. Choose either Current User or Local computer or whichever is most relevant for you.

Let's dispel some weirdness. My = Personal. For some hilarious reason there is no 'My' folder and it is actually called 'Personal'. This is the place you should be looking for for your own certificates you have created.

So if you have found the certificates you were after you will probably need the thumbprint... but it is encrypted. Awesome.

If you open up PowerShell and type the command

get-childitem -path cert:\CurrentUser\My

this will list all of the certificates in the Current User\Personal folder for you by thumbprint. Very handy. For the ones in Local machine simply replace CurrentUser with LocalMachine.

Hopefully this will save you some time when dealing with this confusing certificate nightmare.

Service 'SQL Azure Data Sync Preview' failed to start

While trying to install and start the SQL Azure Data Sync Preview Service I ran across the error message "Service 'SQL Azure Data Sync Preview' (SQL Azure Data Sync Preview) failed to start. Verify that you have sufficient privileges to start system services." Here are the step I had to take to make it start working.

You have to get all of these right to be able to install and start the service successfully. Most are not very obvious and the final step I seem to be the only person on the internet with this problem (hence the blog post)! Anyway, here goes:

Get your username and domain right

Although it says you can enter the account with examples of "domain\user, .\localuser", you can't; it is a lie! You should ensure that you fill out the "User Name" field like MACHINENAME\Username. So MYMACHINE\Colin if you were me.

Verify you have sufficient permissions

Your user may not have permission to log in to Windows Services so you need to make sure you can.

Go to:

• Control Panel > Administrative Tools > Local Security Policy
• Open Local Policies > User Rights Assignment
• Click on Log on as a service
• Click "Add User or Group..."
• Set location to your machine
• Type your user name into the box and click OK then Apply

Ensure you have a password

This was the problem I ran into. The installation just didn't seem to complete because my Windows user did not have a password. I Ctrl + Alt + Del > Change password. I gave myself an actual password rather than leaving it blank which I had done in the past. It then installed without a hitch.

Conclusion

So quite a few snags you may run into when installing the Data Sync service but hopefully with my little check list it wont take you as long as it took me. I'm guessing these sort of bits are just a symptom of the service being in Preview not RTM so they are understandable really.

For what it's worth the preview of SQL Azure Data Sync is a great service, installing this Windows Service was the only issue I had in an otherwise simple process. The benefits of using the service are great, you can sync multiple databases both inside or outside of the Azure data centres. You will only get charged for data leaving the data centre too so any SQL Azure to SQL Azure synchronisation within the same data centre is free! Well, free for now anyway since it is still in preview so I'd keep an eye out when it becomes production ready. Happy syncing!

Share admin login with LiveID in the Azure Management Portal

Has this happened to you? Someone else signed up to use Windows Azure and you are the one who is working on it. You want to access this Azure Portal with a different LiveID? Your LiveID?

Let me guess - your CTO/director decided that Azure is awesome and signed up for its free trial. He had a click around and then quickly passed it on to you, the developer, to work on it. Now you are getting irritated having to remember the new weird LiveID and password and you are tired having to log in and out of your other applications that use your LiveID.

Then the way forward is for you to add a Co-Admin to the Azure Portal. This will allow you to log in with your LiveID that you are used to.

Add new Co-Admin to your Azure account

In your Azure Management portal:

• Go to the "Hosted Services, Storage Accounts & CDN" tab.
• Then click "User Management".
• Click "Add New Co-Admin".
• Type in your LiveID and select the subscription they can administrate.

Your new LiveID is now added to the list of users and should be a Co-Administrator which has all the power you need to manage the technical side of your available subscriptions.

Two host names appearing on Google for the same site

If you have an application hosted in Window's Azure your application will have a host name such as mysite.cloudapp.net. You will probably also have a CNAME that points www.mysite.com to the cloudapp. So what if both domains have been indexed by Google? How do you get cloudapp.net URL off of Google?

This is not a problem with Windows Azure; this can happen to any site that has two hostnames/IPs that point to the same site. It is just a recurring issue I see during my Azure consultancy work and worth a quick explanation and solution.

Both mysite.cloudapp.net and mysite.com are pointing to the exact same application and servers so there is no way you can switch one off since they are one and the same, just accessible through two paths. (Well, more than two since they will at least have IP address paths too.)

This issue can be solved in your application code. Your application will need to determine what host name the application has been accessed through and either display the page if the host name is correct or 301 redirect to the correct hostname if the host name is wrong. So how to do it?

301 Redirect cloudapp.net to your correct domain

This can be done in your application by setting a redirect rule using the IIS URL Rewrite Module. There is already a template in IIS's Rewrite Module for doing this called "Canonical domain name" under the SEO section, however I don't particularly like this one since it redirects everything that isn't the correct hostname (e.g. your development machine). Still, you should have a look at it to see what it is doing.

The key steps of what your rule should be doing are:

1. Check to see if the host name is incorrect
2. If it is it should issue a redirect to the correct domain name
3. The redirect should be a 301
4. The page should remain e.g. http://mysite.cloudapp.net/hello.html -> http://www.mysite.com/hello.html
5. If the host name is not in violation the page rendering should continue without further action

So you can make your own however you feel is best but here is my example to get you going.

<rewrite>
    <rules>
        <rule name="CanonicalHostNameRule1">
            <match url="(.*)" />
            <conditions>
                <add input="{HTTP_HOST}" pattern="^mysite\.cloudapp\.net$" negate="false" />
            </conditions>
            <action type="Redirect" url="http://www.mysite.com/{R:1}" />
        </rule>
    </rules>
</rewrite>

This code goes in the <system.webServer>...</system.webServer> section of your web.config. Again this is only an example, you may wish to add another condition for handling your IP address for example.

This redirect will solve the problem of any users visiting the incorrect site. Also, because it is a 301 redirect, over a week or two Google will remove the cloudapp.net URL from its search results and attribute its existing value to the correct URL.

VIP swap with different endpoints in Azure

Attempting to Swap VIPs from the Azure Management Portal can return an message "Windows Azure cannot perform a VIP swap between deployments that specify different endpoint ports" when trying to Swap VIPs between a staging and production environment with different endpoints.

Say you have a web role deployed to Azure and it is currently in production. It has one instance and one endpoint of http on port 80. You then publish a new web role to staging that is also one instance but this time it has an https endpoint on port 443.

Both will work as usual however when you come to swap the VIPs over you will recieve the error "Windows Azure cannot perform a VIP swap between deployments that have a different number of endpoints" preventing you from putting your new site into production.

Is there a way around this?

I tried a few different tactics. I tried doing a straight upgrade to production (since I have tested the same deployment in staging) but this failed for the same reason as above.

I tried changing my new package to have the original http endpoint on port 80 as well as the new endpoint on port 443. This didn't work either. All this experimenting takes time so I thought I would save you the hassle and just tell you that neither of these work!

The only way(?)

The only way I found to get your new site into production is (you guessed it) delete the old site that is to be replaced (scary!), and then click "Swap VIP". You will experience down time for somewhere between 2 and 3 minutes so pick the least destructive time to do this.

Note: Please please test the new site that on staging thoroughly as you just deleted the production package and so it will take a long time to get back!

Cannot create database 'DevelopmentStorageDb20110816' in Storage Emulator Azure SDK

I have uncovered this problem before and this time I thought I would spend the time to crack it. When running the Storage Emulator from the Azure SDK or running DSINIT.exe for the first time it needs to initialise; part of this involves creating a new database. This was not running correctly due to a permissions problem which stated "Cannot create database 'DevelopmentStorageDb20110816' : CREATE DATABASE permission denied in database 'master'".

The full report is as follows:

Added reservation for http://127.0.0.1:10000/ in user account MSFT-123\MyUser.
Added reservation for http://127.0.0.1:10001/ in user account MSFT-123\MyUser.
Added reservation for http://127.0.0.1:10002/ in user account MSFT-123\MyUser.

Creating database DevelopmentStorageDb20110816...
Cannot create database 'DevelopmentStorageDb20110816' : CREATE DATABASE permission denied in database 'master'.

One or more initialization actions have failed. Resolve these errors before attempting to run the storage emulator again. These errors can occur if SQL Server was installed by someone other than the current user. Please refer to http://go.microsoft.com/fwlink/?LinkID=205140 for more details.

Grant permissions

The user this program is running as must be a sa with full permissions on the database. If this is not the case you can either change the user or GRANT permissions to the current user.

To change the user run the Windows Azure SDK Command Prompt (as administrator) and type 'DSINIT /?'. This will give you details on how to change user, which is to use the /user: argument.

Alternatively you could GRANT the permissions to your default user like so:

USE master

GRANT CREATE DATABASE TO "MYDOMAIN\MyUser"

Either of these solutions should solve your permissions problem with DSINIT.

Cannot GRANT permissions

Unfortunately my problems went further than this still. I believe this is to do with how many times I had installed a SQL Server Express in the past. How irritating is that?! You own the machine and yet you don't seem to have the permissions that reflect that!

Anyway, the solution to make this work once again was (unfortunately) to uninstall SQL Server Express and reinstall it again. This way you will be the owner of the SQL Server Express database engine and you will be able to create all the databases you wish, including your long awaited Storage Emulator database.

A better way to regain admin access

A better way to regain admin access was pointed out to me by a colleague, Michael Coates. You can either solve this loss of administrator on a SQL server by following this troubleshooting guide from MSDN (this worked for one commenter). Or you can run a batch script that will magically do it for you (this didn't work for one commenter). I have not done either of these so take this advice at your own risk. Remember, my way was to delete and reinstall the server so this cannot be more risky, surely?

Cross database joins in SQL Azure

Currently cross database joins are not supported in SQL Azure. Also you cannot change database mid query so you cannot, for example, put a USE [MyDB] in your query either. As a side note, please vote for it to be a priority feature for the SQL Azure team to develop soon.

So, since cross database joins are not supported at this time you must find a workaround. I will give you two possible solutions I would recommend and you can hopefully choose the one that is best for your application.

Combine your databases

If you have tables that are frequently used together, i.e. they are joined in queries or the rows are inserted in the same transactions, then it would be a good idea to move the similar tables into the same database. This of course eliminates the need to traverse databases. SQL Azure has very recently increased the maximum database size from 50GB to 150GB, which potentially makes this a more viable option than perhaps it once was.

Join your data in your application

Two separate queries could be run on the two separate databases and then these results could be joined within that application. Obvious downsides to this will be the potential for large DB I/O, large network transfer and large memory usage in your app. This is not something to consider if the amount of data that is likely to be returned is large (e.g. 1000+ rows) but it is fine if the data will be manageable.

Conclusion

Personally I would much rather settle for bringing all my similar tables that are likely to be used within the same queries together into one database so there is no longer a need for cross database solutions. This makes for cleaner application code and more efficient use of your resources. However, if this option is not available to you then perhaps the second option may have to be the one you choose.

Surely soon in the future the SQL Azure team will address this issue though and your cross database code can stay clean! Although in fairness cross database querying isn't even available in Entity Framework yet (but still workaround-able) either. I wonder why this is so difficult for Microsoft? Just shows you should always try and combine similar DBs in any database designs where possible.

"The Web Platform Installer could not start" fix

I was trying to download the Azure SDK version 1.5 and I was experiencing issues preventing the install because the Windows Platform installer could not load up and run properly. It instead opened a error message saying "The Web Platform Installer could not start. Please report the following error on the Web Platform Installer Forum. http:/forums.iis.net/1155.aspx".

The problem was that the Web Platform Installer needed to update but could not seem to manage it. This was causing the error to display and then crash without allowing me to install the original Azure SDK I was actually trying to install.

Microsoft Fix It fixed it

Microsoft Fix It is a tool from Microsoft that automatically finds the particular one you want in this scenario is the Diagnose and fix program installing and uninstalling problems automatically.

To do this download it and use it as follows:

• When the option to select "Detect problems and let me select the fixes to apply." appears, do that.
• say you are having a problem with "Installing"
• After a short wait scroll down to find "Microsoft Web Platform Installer 3.0" (or which ever version number you have) and select that.
• Next you should click "Yes, try uninstall" then click next and follow the instructions.

This should solve your issue with the Microsoft Web Platform Installer and allow you to install the Azure SDK or whatever it is you're trying to install.

Unable to remove directory. Access to the path 'mswasri.dll' is denied when packaging an Azure project

When trying to build a package using the Azure SDK built into Visual Studio 2010 I sometimes get the error message "Unable to remove directory "bin\Release\CloudPackage.csx\". Access to the path 'mswasri.dll' is denied."

This stopped me from being able to build Azure cloud package ready for deployment. I tried changing my cloud package .csx file to read only but it just changed back. I tried deleting it but it was in use.

This gave me a clue. I run my local copy of my Azure site through my local IIS. I think this was locking the site and thus preventing it from being packaged.

Solution

The way to solve this problem that I found to work best was to find the Application Pool that is running the web application in my local development environment and stop it. This allows me to successfully package my web application ready from deployment to the Azure cloud!

Azure TableServiceContext does not contain CreateQuery Azure

I have done it loads of times before but this time when trying to call CreateQuery, Add Object, DeleteObject, UpdateObject or SaveChanges on my TableServiceContext I got an error saying "'Microsoft.WindowsAzure.StorageClient.TableStorageServiceContext' does not contain a definition for 'CreateQuery'".

I keep forgetting that as well as including the Microsoft.WindowsAzure.StorageClient assembly I also need to include System.Data.Services.Client.

This is because, although TableStorageServiceContext comes in Microsoft.WindowsAzure.StorageClient.dll, it inherits from DataServiceContext which is part of another assembly, System.Data.Services.Client.dll. So this must be included in you Azure storage project.